We are overjoyed to announce the release of: magnum 6.0.1: Container Management project for OpenStack This release is part of the queens release series. The source is available from: ** http://git.openstack.org/cgit/openstack/magnum Download the package from: https://tarballs.openstack.org/magnum/ Please report issues through launchpad: ** http://bugs.launchpad.net/magnum For more details, please see below. 6.0.1 ^^^^^ Known Issues ************ * Kubernetes client is incompatible with evenlet and breaks the periodic tasks. After kubernetes client 4.0.0 magnum is affected by the bug below. https://github.com/eventlet/eventlet/issues/147 Magnum has three periodic tasks, one to sync the magnum service, one to update the cluster status and one send cluster metrics The send_metrics task uses the kubernetes client for kubernetes clusters and it crashes the sync_cluster_status and send_cluster_metrics tasks. https://bugs.launchpad.net/magnum/+bug/1746510 Upgrade Notes ************* * In magnum configuration, in [drivers] set send_cluster_metrics = False to to avoid collecting metrics using the kubernetes client which crashes the periodic tasks. Changes in magnum 5.0.0..6.0.1 ------------------------------ 710192a Add issue to reno for the incompatible k8s client 198fce7 Support accessing all clusters/templates across projects c57bc86 Deprecate usage of tenant and user in context 53d386d Add label availability_zone cf84683 Add send_cluster_metrics configuration parameter d11f87d Start RPC service before waiting 0e4ec17 Remove broken job magnum-non-functional-tox-migration e825e43 Zuul: Remove project name be0609c Support soft-anti-affinity policy for nodes 674b905 ci: Add redirection from /v2 to /identity/v2 65dfb20 Add openstack_ca_file configuration option 192dc8b [k8s] Add missing verify_ca in minion_wc_notify 8287661 fix url for versioned objects docs in code 34754f3 federation api: federation table and db layer 2a60e56 Change the name of kubernetes-dashboard deployment. 0053570 [k8s] Take container_infra_prefix from cluster if specified df0dad9 Don't run functional jobs on api-ref changes 130ebbe Fix policies for quotas 54cd3ca Use barbicanclient.v1 instead of barbicanclient f85cf55 Fix image list and usage in contributor quickstart 553316e Fix: functional CI Jobs 2c3af23 doc: Use os_distro instead of os-distro 604bbe5 Fix Usage of cliff commandmanager 65b47b5 Update docs to use openstack client commands 1be24f3 Update Fedora Atomic image name ba1e7b8 Add missing translation for verify_ca cae5a1d Updated from global requirements 2996385 [k8s] Take kube_tag from cluster if specified 273405c Leverage heat-container-agent for monitoring 84006f6 Allow flavor_id on cluster create 28fff80 Make docker_storage_driver a str instead of enum 53fa3ac Remove intree magnum tempest plugin 4527311 [doc-migration] Consolidate install guide 9e0b444 The os_distro of image is case sensitive 2f69309 k8s_atomic: Remove kubelet and kube-proxy from master 73b7250 Updated from global requirements 336404c Generate lower case stack name b07b6f3 Add verify_ca configuration parameter f89cc4c k8s_atomic: Add server to kubeconfig 282c3b5 Add app.wsgi to target of pep8 d71f6c3 Remove setting of version/release from releasenotes 4371d21 Updated from global requirements c609524 Fix: magnum devstack installation with tls-proxy 0165244 Updated from global requirements 616b2ee Updated from global requirements 5036eb3 Redundant alias in import statement 8a875af Do not use “-y” for package install 9899aa1 Using --option ARGUMENT 283c093 Generate stack name as a valid hostname d78c115 Zuul: add file extension to playbook path 7482e65 Doc Fix for Alembic multiple heads error b262050 Add sample policy configuration to doc c77e0c7 Register default magnum service and stat policies in code b7ca578 Register default certificate policies in code c14af5c Register default quota policies in code 38a8fed Register default cluster template policies in code 0d36f8b Register default cluster policies in code 1cbb1d4 Register default baymodel policies in code ac0756e Register default bay policies in code e06004d Implement basic policy module in code 0258528 use keystoneauth1 session in functional test 0366512 Fix use of irrelevant-files parameter b5531dd Add /etc/environment to flannel/etcd/kubelet f51fbba Updated from global requirements 4fc1588 Add labels to api-ref cluster create 0b49f38 Migrate to Zuul v3 2e36a0c Fix user-guide formatting 67a4874 Fix magnum TLS cert generation 532fc26 Fix to use the correct hyperlink. 010a267 Swarm: Incorrect reference to Flannel variables 8f74906 [swarm-fedora-atomic] fix cluster etcd_lb protocol definition 446b159 Allow master_flavor_id on cluster create 7de5264 Add kube_dashboard_enabled label to user guide 1e3fa7c Updated from global requirements 4aade35 Fix prometheus scrape configuration c4ffa7b writing convention: do not use “-y” for package install 69bb03f k8s_fedora: Add container_infra_prefix label 3e6884a Add default configuration files to data_files 74677ad Remove SCREEN_LOGDIR from devstack setting c86bffe Updated from global requirements 8ce15c4 Avoid running periodic processes inside each worker process 68d7b87 Update CoreDNS to 011 ce0c039 Updated from global requirements fe6286e k8s: Fix node-exporter manifest 640217d Use newer location for iso8601 UTC 7e16d77 Updated from global requirements 71beb1d Imported Translations from Zanata 1d9101b writing convention set to use "." to source script files c47caad Updated from global requirements 244fe10 Imported Translations from Zanata b17959f Update reno for stable/pike 048400b Remove TENANT_NAME from /etc/sysconfig/heat-params 35dc923 Fix no_proxy evaluation for Swarm clusters 3d04ed4 Allow labels on cluster create c22f9cd Deal with db_exc.DBDuplicate of conductor startup 136820c Remove unused config periodic_global_stack_list 8e36613 Fix usage of --kubelet-preferred-address arg for apiserver 5153d91 Clean-up server names in drivers debdc71 Stop using deprecated 'message' attribute in Exception 644bf34 Add attribute 'disabled' for service-list Diffstat (except docs and test files) ------------------------------------- .gitignore | 3 + .zuul.yaml | 287 +++++++++++ api-ref/source/clusters.inc | 3 + api-ref/source/parameters.yaml | 7 +- api-ref/source/samples/cluster-create-req.json | 6 +- contrib/drivers/heat/dcos_centos_template_def.py | 2 +- devstack/lib/magnum | 60 ++- devstack/plugin.sh | 1 + devstack/settings | 1 - .../common/configure_2_edit_magnum_conf.rst | 114 +++++ .../common/configure_3_populate_database.rst | 5 + etc/magnum/magnum-policy-generator.conf | 3 + etc/magnum/policy.json | 52 -- .../source/common/configure_2_edit_magnum_conf.rst | 111 ----- .../common/configure_3_populate_database.rst | 5 - install-guide/source/common/prerequisites.rst | 191 ------- install-guide/source/conf.py | 301 ----------- install-guide/source/get_started.rst | 20 - install-guide/source/index.rst | 24 - install-guide/source/install-debian-manual.rst | 32 -- install-guide/source/install-obs.rst | 36 -- install-guide/source/install-rdo.rst | 45 -- install-guide/source/install-ubuntu.rst | 32 -- install-guide/source/install.rst | 52 -- install-guide/source/launch-instance.rst | 453 ----------------- install-guide/source/next-steps.rst | 9 - install-guide/source/verify.rst | 33 -- magnum/api/app.wsgi | 1 - magnum/api/attr_validator.py | 7 +- magnum/api/controllers/v1/bay.py | 27 +- magnum/api/controllers/v1/baymodel.py | 2 +- magnum/api/controllers/v1/cluster.py | 63 ++- magnum/api/controllers/v1/cluster_template.py | 39 +- magnum/api/controllers/v1/magnum_services.py | 3 +- magnum/cmd/conductor.py | 8 + magnum/cmd/driver_manage.py | 2 +- .../common/cert_manager/barbican_cert_manager.py | 2 +- magnum/common/clients.py | 2 +- magnum/common/context.py | 2 +- magnum/common/exception.py | 8 + magnum/common/policies/__init__.py | 39 ++ magnum/common/policies/base.py | 52 ++ magnum/common/policies/bay.py | 91 ++++ magnum/common/policies/baymodel.py | 106 ++++ magnum/common/policies/certificate.py | 60 +++ magnum/common/policies/cluster.py | 125 +++++ magnum/common/policies/cluster_template.py | 141 ++++++ magnum/common/policies/magnum_service.py | 36 ++ magnum/common/policies/quota.py | 80 +++ magnum/common/policies/stats.py | 36 ++ magnum/common/policy.py | 3 + magnum/common/rpc_service.py | 5 +- magnum/common/utils.py | 10 + magnum/conductor/k8s_api.py | 4 +- magnum/conf/__init__.py | 2 + magnum/conf/cluster.py | 6 + magnum/conf/drivers.py | 48 ++ magnum/conf/rpc.py | 7 - magnum/db/api.py | 96 ++++ .../041d9a0f1159_add_flavor_id_to_cluster.py | 30 ++ ...04c625aa95ba_change_storage_driver_to_string.py | 33 ++ ...52bcaf58fecb_add_master_flavor_id_to_cluster.py | 30 ++ .../versions/9a1539f1cd2c_add_federation_table.py | 47 ++ .../versions/a0e7c8450ab1_add_labels_to_cluster.py | 30 ++ magnum/db/sqlalchemy/api.py | 120 ++++- magnum/db/sqlalchemy/models.py | 21 + .../common/image/heat-container-agent/Dockerfile | 49 ++ .../heat-container-agent/config.json.template | 373 ++++++++++++++ .../common/image/heat-container-agent/launch | 5 + .../image/heat-container-agent/manifest.json | 4 + .../scripts/50-heat-config-docker-compose | 116 +++++ .../heat-container-agent/scripts/55-heat-config | 194 ++++++++ .../scripts/configure_container_agent.sh | 36 ++ .../scripts/heat-config-notify | 163 ++++++ .../heat-container-agent/scripts/hooks/atomic | 115 +++++ .../scripts/hooks/docker-compose | 127 +++++ .../heat-container-agent/scripts/hooks/script | 96 ++++ .../scripts/write-os-apply-config-templates.sh | 71 +++ .../image/heat-container-agent/service.template | 11 + .../image/heat-container-agent/tmpfiles.template | 10 + .../fragments/atomic-install-openstack-ca.sh | 12 + .../fragments/configure-docker-storage.sh | 12 +- .../configure_docker_storage_driver_atomic.sh | 10 +- .../fragments/configure-kubernetes-master.sh | 25 +- .../fragments/configure-kubernetes-minion.sh | 49 +- .../kubernetes/fragments/core-dns-service.sh | 7 +- .../kubernetes/fragments/enable-monitoring.sh | 139 ------ .../kubernetes/fragments/enable-node-exporter.sh | 9 +- .../fragments/enable-prometheus-monitoring | 434 ++++++++++++++++ .../kubernetes/fragments/kube-dashboard-service.sh | 8 +- .../kubernetes/fragments/make-cert-client.sh | 18 +- .../templates/kubernetes/fragments/make-cert.sh | 12 +- .../kubernetes/fragments/start-container-agent.sh | 16 + .../kubernetes/fragments/wc-notify-master.sh | 8 +- .../fragments/write-grafana-service.yaml | 67 --- .../fragments/write-heat-params-master.yaml | 3 +- .../kubernetes/fragments/write-heat-params.yaml | 3 +- .../kubernetes/fragments/write-kube-os-config.sh | 4 +- .../kubernetes/fragments/write-kubeconfig.yaml | 24 - .../fragments/write-prometheus-configmap.yaml | 82 --- .../fragments/write-prometheus-service.yaml | 60 --- .../common/templates/swarm/fragments/add-proxy.sh | 33 +- .../common/templates/swarm/fragments/cfn-signal.sh | 8 +- .../common/templates/swarm/fragments/make-cert.py | 21 +- .../swarm/fragments/network-config-service.sh | 4 +- .../fragments/write-cluster-failure-service.yaml | 2 +- .../swarm/fragments/write-heat-params-master.yaml | 1 + .../swarm/fragments/write-heat-params-node.yaml | 1 + .../swarm/fragments/write-swarm-agent-service.sh | 12 +- .../swarm/fragments/write-swarm-master-service.sh | 8 +- magnum/drivers/heat/driver.py | 16 +- magnum/drivers/heat/k8s_fedora_template_def.py | 15 +- magnum/drivers/heat/k8s_template_def.py | 6 +- magnum/drivers/heat/swarm_fedora_template_def.py | 10 +- magnum/drivers/heat/swarm_mode_template_def.py | 8 +- magnum/drivers/heat/template_def.py | 3 + .../templates/fragments/add-ext-ca-certs.yaml | 32 ++ .../templates/fragments/configure-etcd.yaml | 1 + .../templates/fragments/enable-kubelet-master.yaml | 1 + .../templates/fragments/enable-kubelet-minion.yaml | 1 + .../fragments/enable-network-service-client.yaml | 7 + .../fragments/enable-network-service.yaml | 7 + .../templates/fragments/make-cert-client.yaml | 12 +- .../templates/fragments/make-cert.yaml | 12 +- .../templates/fragments/wc-notify.yaml | 2 +- .../fragments/write-heat-params-master.yaml | 1 + .../templates/fragments/write-heat-params.yaml | 1 + .../k8s_coreos_v1/templates/kubecluster.yaml | 42 ++ .../k8s_coreos_v1/templates/kubemaster.yaml | 32 ++ .../k8s_coreos_v1/templates/kubeminion.yaml | 33 ++ .../templates/kubecluster.yaml | 70 ++- .../k8s_fedora_atomic_v1/templates/kubemaster.yaml | 117 +++-- .../k8s_fedora_atomic_v1/templates/kubeminion.yaml | 58 ++- .../templates/kubecluster.yaml | 62 ++- .../k8s_fedora_ironic_v1/templates/kubemaster.yaml | 110 ++-- .../k8s_fedora_ironic_v1/templates/kubeminion.yaml | 10 + .../templates/kubeminion_software_configs.yaml | 41 +- magnum/drivers/mesos_ubuntu_v1/template_def.py | 8 +- .../templates/fragments/add-ext-ca-certs.sh | 27 + .../templates/mesos_slave_software_configs.yaml | 22 +- .../mesos_ubuntu_v1/templates/mesoscluster.yaml | 41 ++ .../mesos_ubuntu_v1/templates/mesosmaster.yaml | 32 ++ .../mesos_ubuntu_v1/templates/mesosslave.yaml | 10 + .../swarm_fedora_atomic_v1/templates/cluster.yaml | 45 +- .../templates/swarmmaster.yaml | 51 +- .../templates/swarmnode.yaml | 48 +- .../fragments/write-heat-params-master.yaml | 1 + .../fragments/write-swarm-master-service.sh | 14 +- .../fragments/write-swarm-worker-service.sh | 12 +- .../templates/swarmcluster.yaml | 59 ++- .../templates/swarmmaster.yaml | 38 +- .../templates/swarmnode.yaml | 39 +- magnum/objects/cluster.py | 8 +- magnum/objects/cluster_template.py | 6 +- magnum/objects/fields.py | 16 - magnum/service/periodic.py | 4 + .../functional/api/v1/test_cluster_template.py | 230 --------- .../api/v1/test_cluster_template_admin.py | 86 ---- .../unit/api/controllers/v1/test_certificate.py | 8 +- .../api/controllers/v1/test_cluster_template.py | 28 +- .../unit/api/controllers/v1/test_magnum_service.py | 14 +- .../unit/common/cert_manager/test_barbican.py | 6 +- .../handlers/test_k8s_cluster_conductor.py | 99 ++-- .../handlers/test_mesos_cluster_conductor.py | 29 +- .../handlers/test_swarm_cluster_conductor.py | 39 +- playbooks/magnum-buildimages-base.yaml | 102 ++++ playbooks/magnum-functional-base.yaml | 96 ++++ playbooks/post/upload-images.yaml | 21 + playbooks/post/upload-logs.yaml | 14 + playbooks/pre/prepare-workspace-images.yaml | 12 + playbooks/pre/prepare-workspace.yaml | 23 + ...dd-container_infra_prefix-516cc43fbc5a0617.yaml | 11 + .../notes/availability_zone-2d73671f5ea065d8.yaml | 6 + ...broken-kuberenetes-client-d2d1da6029825208.yaml | 17 + .../notes/bug-1580704-32a0e91e285792ea.yaml | 7 + .../notes/bug-1663757-198e1aa8fa810984.yaml | 12 + .../notes/bug-1718947-0d4e67529e2817d7.yaml | 6 + .../notes/bug-1722522-d94743c6362a5e48.yaml | 10 + ...ort-all-tenants-for-admin-a042f5c520d35837.yaml | 5 + ...rt-policy-and-doc-in-code-0c19e479dbd953c9.yaml | 21 + ...ort_nodes_affinity_policy-22253fb9cf6739ec.yaml | 7 + releasenotes/source/conf.py | 11 +- releasenotes/source/index.rst | 1 + .../source/locale/fr/LC_MESSAGES/releasenotes.po | 64 +++ .../source/locale/ja/LC_MESSAGES/releasenotes.po | 15 +- releasenotes/source/pike.rst | 6 + requirements.txt | 56 +-- setup.cfg | 8 +- test-requirements.txt | 15 +- tox.ini | 5 + 245 files changed, 7378 insertions(+), 4275 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 845022a..cf3e3d1 100644 --- a/requirements.txt +++ b/requirements.txt @@ -11 +11 @@ SQLAlchemy!=1.1.5,!=1.1.6,!=1.1.7,!=1.1.8,>=1.0.10 # MIT -WSME>=0.8 # MIT +WSME>=0.8.0 # MIT @@ -14 +14 @@ alembic>=0.8.10 # MIT -cliff>=2.8.0 # Apache-2.0 +cliff!=2.9.0,>=2.8.0 # Apache-2.0 @@ -16,2 +16,2 @@ decorator>=3.4.0 # BSD -docker>=2.0.0 # Apache-2.0 -enum34;python_version=='2.7' or python_version=='2.6' or python_version=='3.3' # BSD +docker>=2.4.2 # Apache-2.0 +enum34>=1.0.4;python_version=='2.7' or python_version=='2.6' or python_version=='3.3' # BSD @@ -20,20 +20,20 @@ iso8601>=0.1.11 # MIT -jsonpatch>=1.1 # BSD -keystoneauth1>=3.1.0 # Apache-2.0 -keystonemiddleware>=4.12.0 # Apache-2.0 -kubernetes>=1.0.0 # Apache-2.0 -marathon>=0.8.6 # MIT -netaddr!=0.7.16,>=0.7.13 # BSD -oslo.concurrency>=3.8.0 # Apache-2.0 -oslo.config!=4.3.0,!=4.4.0,>=4.0.0 # Apache-2.0 -oslo.context>=2.14.0 # Apache-2.0 -oslo.db>=4.24.0 # Apache-2.0 -oslo.i18n!=3.15.2,>=2.1.0 # Apache-2.0 -oslo.log>=3.22.0 # Apache-2.0 -oslo.messaging!=5.25.0,>=5.24.2 # Apache-2.0 -oslo.middleware>=3.27.0 # Apache-2.0 -oslo.policy>=1.23.0 # Apache-2.0 -oslo.serialization!=2.19.1,>=1.10.0 # Apache-2.0 -oslo.service>=1.10.0 # Apache-2.0 -oslo.utils>=3.20.0 # Apache-2.0 -oslo.versionedobjects>=1.17.0 # Apache-2.0 -oslo.reports>=0.6.0 # Apache-2.0 +jsonpatch!=1.20,>=1.16 # BSD +keystoneauth1>=3.3.0 # Apache-2.0 +keystonemiddleware>=4.17.0 # Apache-2.0 +kubernetes>=4.0.0 # Apache-2.0 +marathon!=0.9.1,>=0.8.6 # MIT +netaddr>=0.7.18 # BSD +oslo.concurrency>=3.20.0 # Apache-2.0 +oslo.config>=5.1.0 # Apache-2.0 +oslo.context>=2.19.2 # Apache-2.0 +oslo.db>=4.27.0 # Apache-2.0 +oslo.i18n>=3.15.3 # Apache-2.0 +oslo.log>=3.30.0 # Apache-2.0 +oslo.messaging>=5.29.0 # Apache-2.0 +oslo.middleware>=3.31.0 # Apache-2.0 +oslo.policy>=1.30.0 # Apache-2.0 +oslo.serialization!=2.19.1,>=2.18.0 # Apache-2.0 +oslo.service!=1.28.1,>=1.24.0 # Apache-2.0 +oslo.utils>=3.33.0 # Apache-2.0 +oslo.versionedobjects>=1.28.0 # Apache-2.0 +oslo.reports>=1.18.0 # Apache-2.0 @@ -45 +45 @@ python-glanceclient>=2.8.0 # Apache-2.0 -python-heatclient>=1.6.1 # Apache-2.0 +python-heatclient>=1.10.0 # Apache-2.0 @@ -47 +47 @@ python-neutronclient>=6.3.0 # Apache-2.0 -python-novaclient>=9.0.0 # Apache-2.0 +python-novaclient>=9.1.0 # Apache-2.0 @@ -51 +51 @@ setuptools!=24.0.0,!=34.0.0,!=34.0.1,!=34.0.2,!=34.0.3,!=34.1.0,!=34.1.1,!=34.2. -six>=1.9.0 # MIT +six>=1.10.0 # MIT @@ -54 +54 @@ taskflow>=2.7.0 # Apache-2.0 -cryptography!=2.0,>=1.6 # BSD/Apache-2.0 +cryptography!=2.0,>=1.9 # BSD/Apache-2.0 diff --git a/test-requirements.txt b/test-requirements.txt index 3edc3f0..12535af 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -9 +9 @@ bandit>=1.1.0 # Apache-2.0 -bashate>=0.2 # Apache-2.0 +bashate>=0.5.1 # Apache-2.0 @@ -11 +11 @@ coverage!=4.4,>=4.0 # Apache-2.0 -doc8 # Apache-2.0 +doc8>=0.6.0 # Apache-2.0 @@ -15 +15 @@ mock>=2.0.0 # BSD -openstackdocstheme>=1.16.0 # Apache-2.0 +openstackdocstheme>=1.17.0 # Apache-2.0 @@ -18,3 +18,3 @@ osprofiler>=1.4.0 # Apache-2.0 -os-api-ref>=1.0.0 # Apache-2.0 -os-testr>=0.8.0 # Apache-2.0 -python-subunit>=0.0.18 # Apache-2.0/BSD +os-api-ref>=1.4.0 # Apache-2.0 +os-testr>=1.0.0 # Apache-2.0 +python-subunit>=1.0.0 # Apache-2.0/BSD @@ -23 +22,0 @@ sphinx>=1.6.2 # BSD -tempest>=16.1.0 # Apache-2.0 @@ -26 +25 @@ testscenarios>=0.4 # Apache-2.0/BSD -testtools>=1.4.0 # MIT +testtools>=2.2.0 # MIT