We are thrilled to announce the release of:
magnum 7.0.0: Container Management project for OpenStack
This release is part of the rocky release series.
The source is available from:
** http://git.openstack.org/cgit/openstack/magnum
Download the package from:
https://tarballs.openstack.org/magnum/
Please report issues through launchpad:
** http://bugs.launchpad.net/magnum
For more details, please see below.
7.0.0 ^^^^^
New Features ************
* k8s_fedora_atomic clusters are deployed with RBAC support. Along with RBAC Node authorization is added so the appropriate certificates are generated.
* This release introduces 'federations' endpoint to Magnum API, which allows an admin to create and manage federations of clusters through Magnum. As the feature is still under development, the endpoints are not bound to any driver yet. For more details, please refer to bp /federation-api [1].
[1] https://review.openstack.org/#/q/topic:bp/federation-api
* This is allowing no floating IP to be usable with a multimaster configuration in terms of load balancers.
* Add new label 'cert_manager_api' enabling the kubernetes certificate manager api.
* Embed certificates in kubernetes config file when issuing 'cluster config', instead of generating additional files with the certificates. This is now the default behavior. To get the old behavior and still generate cert files, pass --output-certs.
* Add 'cloud_provider_enabled' label for the k8s_fedora_atomic driver. Defaults to true. For specific kubernetes versions if 'cinder' is selected as a 'volume_driver', it is implied that the cloud provider will be enabled since they are combined.
* Add new labels 'ingress_controller' and 'ingress_controller_role' enabling the deployment of a Kubernetes Ingress Controller backend for clusters. Default for 'ingress_controller' is '' (meaning no controller deployed), with possible values being 'traefik'. Default for 'ingress_controller_role' is 'ingress'.
* In the OpenStack deployment with Octavia service enabled, the Octavia service should be used not only for master nodes high availability, but also for k8s LoadBalancer type service implementation as well.
* Update kubernetes dashboard to *v1.8.3* which is compatible via kubectl proxy. Addionally, heapster is deployed as standalone deployemt and the user can enable a grafana-influx stack with the *influx_grafana_dashboard_enabled* label. See the kubernetes dashboard documenation for more details. https://github.com/kubernetes/dashboard/wiki
* Update k8s_fedora_atomic driver to the latest Fedora Atomic 27 release and run etcd and flanneld in system containers which are removed from the base OS.
Known Issues ************
* Adding 'calico' as network driver for Kubernetes so as to support network isolation between namespace with k8s network policy.
* Currently, the replicas of coreDNS pod is hardcoded as 1. It's not a reasonable number for such a critical service. Without DNS, probably all workloads running on the k8s cluster will be broken. Now Magnum is making the coreDNS pod autoscaling based on the nodes and cores number.
Upgrade Notes *************
* Using the queens (>=2.9.0) python-magnumclient, when a user executes openstack coe cluster config, the client certificate has admin as Common Name (CN) and system:masters for Organization which are required for authorization with RBAC enabled clusters. This change in the client is backwards compatible, so old clusters (without RBAC enabled) can be reached with certificates generated by the new client. However, old magnum clients will generate certificates that will not be able to contact RBAC enabled clusters. This issue affects only k8s_fedora_atomic clusters and clients <=2.8.0, note that 2.8.0 is still a queens release but only 2.9.0 includes the relevant patch. Finally, users can always generate and sign the certificates using this [0] procedure even with old clients since only the cluster config command is affected. [0] https://docs.openstack.org/magnum/latest/user/index.html #interfacing-with-a-secure-cluster
* New clusters should be created with kube_tag=v1.9.3 or later. v1.9.3 is the default version in the queens release.
* New clusters will be created with kube_tag=v1.11.1 or later. v1.11.1 is the default version in the Rocky release.
Security Issues ***************
* k8s_fedora Remove cluster role from the kubernetes-dashboard account. When accessing the dashboard and skip authentication, users login with the kunernetes-dashboard service account, if that service account has the cluster role, users have admin access without authentication. Create an admin service account for this use case and others.
Bug Fixes *********
* Add *region* parameter to the Global configuration section of the Kubernetes configuration file. Setting this parameter will allow Magnum cluster to be created in the multi-regional OpenStack installation.
* Add *trustee_keystone_region_name* optional parameter to the *trust* section. This parameter is useful for multi-regional OpenStack installations with different Identity service for every region. In such installation it is necessary to specify a region when searching for *auth_url* to authenticate a trustee user.
* Now user can update labels in cluster-template. Previously string is passed as a value to labels, but we know that labels can only hold dictionary values. Now we are parsing the string and storing it as dictionary for labels in cluster-template.
* Fix etcd configuration in k8s_fedora_atomic driver. Explicitly enable client and peer authentication and set trusted CA (ETCD_TRUSTED_CA_FILE, ETCD_PEER_TRUSTED_CA_FILE, ETCD_CLIENT_CERT_AUTH, ETCD_PEER_CLIENT_CERT_AUTH). Only new clusters will benefit from the fix.
* When creating a multi-master cluster, all master nodes will attempt to create kubernetes resources in the cluster at this same time, like coredns, the dashboard, calico etc. This race conditon shouldn't be a problem when doing declarative calls instead of imperative (kubectl apply instead of create). However, due to [1], kubectl fails to apply the changes and the deployemnt scripts fail causing cluster to creation to fail in the case of Heat SoftwareDeployments. This patch passes the ResourceGroup index of every master so that resource creation will be attempted only from the first master node. [1] https://github.com/kubernetes/kubernetes/issues/44165
* Create admin cluster role for k8s_fedora_atomic, it is defined in the configuration but it wasn't applied.
* Fix bug #1758672 [1] to protect kubelet in the k8s_fedora_atomic driver. Before this patch kubelet was listening to 0.0.0.0 and for clusters with floating IPs the kubelet was exposed. Also, even on clusters without fips the kubelet was exposed inside the cluster. This patch allows access to the kubelet only over https and with the appropriate roles. The apiserver and heapster have the appropriate roles to access it. Finally, all read-only ports have been closed to not expose any cluster data. The only remaining open ports without authentication are for healthz. [1] https://bugs.launchpad.net/magnum/+bug/1758672
* Strip signed certificate. Certificate (ca.crt) has to be striped for some application parsers as they might require pure base64 representation of the certificate itself, without empty characters at the beginning nor the end of file.
* Multi master deployments for k8s driver use different service account keys for each api/controller manager server which leads to 401 errors for service accounts. This patch will create a signed cert and private key for k8s service account keys explicitly, dedicatedly for the k8s cluster to avoid the inconsistent keys issue.
Changes in magnum 6.0.1..7.0.0 ------------------------------
cf12bb7 Bump k8s version to v1.11.1 feed29d Using cgroupfs as default cgroup-driver 2535915 [k8s] Fix docker volume issue 9d147fa Reno for embed certs in kubernetes config b991838 Using simple public/private key for k8s service account keys eb930a3 Create /etc/kubernetes/manifests on k8s master 1b0fbc2 Change Kubelet flexvolume directory a127c66 Trustee: provide region_name to auth_url searching 4751d32 Fix the heat-container-agent docker image 797f015 Resolve stack outputs only on COMPLETE d417395 Add etcd_volume_size parameter in coreos template ae158f3 Update the default admission control list 974399a k8s_fedora: Add cloud_provider_enabled label ff67ac8 Switch to stestr f6d1c0d Fix etcd race condition issue b7bfee5 Support disabling floating IPs in swarm mode c069cfb Add release notes link in README a28e560 Provide a region to the K8S Fedora Atomic config cff4823 Rename scripts 80f806f Make service account private key hidden 2521afa Pass in `region_name` to get correct heat endpoint 97f086c Revert "Rename scripts" 591a2dc Rename scripts 393e70f Allow multimaster lb with no floating ip option 043c57d Sync service account keys for multi masters 003e27f Added error handling for discoveryurl 4c5d38a k8s_fedora: Create admin cluster-role 4292b86 k8s_fedora: enable tls in traefik ingress 1710f3a k8s_fedora: set ingress traefik log level to INFO e8ac68b Use HostAddressOpt for opts that accept IP and hostnames 3c72d7b Fix race condition issue for k8s multi masters ec58c23 Add option to specify Cgroup driver for Kubelet da910ac fix tox python3 overrides edee703 Strip signed certificate d66b4f2 Revert "Strip signed certificate" 9e97c8b Devicemapper storage driver need specified volume 8996e35 Release note for supporting Octavia as LoadBalancer type service backend 69ffdae Strip signed certificate 2cc57c5 Use Octavia for LoadBalancer type service 38878fd k8s_fedora: Make CoreDNS config a SoftwareDeployment a015193 Update ca related magnum comands to osc b319f72 [doc] fix coredns correct image verison b8fb70d [doc] Correct the non-existent link for the Fedora image 8729997 Open the 8472 port of master for vxlan 91d5229 k8s_fedora: Add admin user 6215887 Follow the new PTI for document build 9eac5ed Imported Translations from Zanata 338b045 Fix incompatible requirement f6ed7d3 Add and improve tests for certificate manager 3d1ee57 Stop using slave_scripts/install-distro-packages.sh 449c8f2 Add bindep.txt file 79c002c Add calico-node on k8s master node 54a4ac9 Make DNS pod autoscale 1c0b972 fix a typo d179c37 Adding documentations about network in vms 16ce059 Adding glossary.rst 405b0c2 k8s_fedora: Add flannel to master nodes d9e590b Cache barbican certs for periodic tasks a1fb448 k8s_fedora: Explicitly set etcd authentication 205e8ad k8s_fedora: Add kubelet authentication/authorization c0f8db9 Updated from global requirements 96e8e6c add lower-constraints job 79f4cc0 Add service account to daemonset in traefik d07fb7f Add missing RBAC config for Prometheus a56c14d TrivialFix: Correcting JSON syntax dd7ed64 Update minimum version of docker in unit tests 1431be0 Add reno for RBAC and client incompatibility 57b9457 Add minimum system requirements to docs 14bf2b2 Use pip_check_reqs module 3721e6b Specify grafana version 21b50f6 Imported Translations from Zanata 30785ac Update kubernetes dashboard to v1.8.3 1eb9bf1 kuberntes: Disable the scale_manager for scale down 4efb58b k8s: allow passing extra options to kube daemons 0b18989 [kubernetes] add ingress controller 6aac363 Admin can now delete clusters in any project d95ba4d Run etcd and flanneld in a system container 838b8da Support calico as network driver 05c7f35 Add disabled_drivers config option 1e36980 Using v1.9.3 as default k8s version 0e6e27e Enables MySQL Cluster Support for Magnum 5a34d7d Check CERT_MANAGER_API if True or False 0e95314 Add missed space in k8s template file 3c8edd4 Add support for Octavia resources in Heat faa9e90 [k8s] allow enabling kubernetes cert manager api 11e7f33 Document use of kube_tag label 1caa885 Change swarm ClusterTemplate coe to swarm-mode 21c87f3 Now user can update label values in cluster-template ec950be federation api: api endpoints 4d39559 Driver's name are case sensitive 38e1051 Update reno for stable/queens 5b45639 Replace CentOS package mysql-devel > mariadb-devel 2329cb7 k8s: Fix kubelet, add RBAC and pass e2e tests 7142ba7 Corrected some misspellings in magnum
Diffstat (except docs and test files) -------------------------------------
.gitignore | 6 +- .stestr.conf | 4 + .zuul.yaml | 2 + README.rst | 1 + .../source/samples/certificates-ca-show-resp.json | 2 +- api-ref/source/samples/cluster-get-one-resp.json | 2 +- bindep.txt | 4 + devstack/lib/magnum | 15 +- devstack/plugin.sh | 2 +- lower-constraints.txt | 172 +++++ magnum/api/attr_validator.py | 27 + magnum/api/controllers/v1/__init__.py | 12 + magnum/api/controllers/v1/cluster.py | 8 +- magnum/api/controllers/v1/cluster_template.py | 6 + magnum/api/controllers/v1/federation.py | 454 ++++++++++++ magnum/api/utils.py | 9 + magnum/api/validation.py | 41 +- magnum/cmd/conductor.py | 2 + magnum/common/exception.py | 9 + magnum/common/keystone.py | 29 + magnum/common/policies/__init__.py | 2 + magnum/common/policies/cluster.py | 11 + magnum/common/policies/cluster_template.py | 11 + magnum/common/policies/federation.py | 91 +++ magnum/common/x509/operations.py | 52 +- magnum/conductor/api.py | 23 + magnum/conductor/handlers/common/cert_manager.py | 91 ++- magnum/conductor/handlers/federation_conductor.py | 32 + magnum/conf/cluster.py | 4 + magnum/conf/drivers.py | 8 +- magnum/conf/services.py | 13 +- magnum/conf/trust.py | 5 +- .../versions/05d3e97de9ee_add_volume_driver.py | 7 +- .../versions/2581ebaf0cb2_initial_migration.py | 8 +- ...35cff7c86221_add_private_network_to_baymodel.py | 8 +- ...65537a94a_add_network_driver_baymodel_column.py | 7 +- .../versions/4956f03cabad_add_cluster_distro.py | 7 +- .../4ea34a59a64c_add_discovery_url_to_bay.py | 8 +- .../versions/5518af8dbc21_rename_cert_uuid.py | 9 +- .../592131657ca1_add_coe_column_to_baymodel.py | 10 +- .../5d4caa6e0a42_create_trustee_for_each_bay.py | 11 +- .../alembic/versions/966a99e70ff_add_proxy.py | 14 +- ...71_add_fixed_subnet_column_to_baymodel_table.py | 8 +- ...47f5931da8_add_insecure_registry_to_baymodel.py | 8 +- .../fcb4efee8f8b_add_version_info_to_bay.py | 11 +- magnum/db/sqlalchemy/models.py | 40 +- magnum/drivers/common/driver.py | 18 +- .../common/image/heat-container-agent/Dockerfile | 26 +- .../scripts/heat-config-notify | 6 +- .../scripts/write-os-apply-config-templates.sh | 2 + .../environments/disable_floating_ip.yaml | 6 + .../environments/disable_lb_floating_ip.yaml | 4 + .../templates/environments/enable_floating_ip.yaml | 6 + .../environments/enable_lb_floating_ip.yaml | 4 + .../templates/environments/with_master_lb.yaml | 1 - .../environments/with_master_lb_octavia.yaml | 13 + .../configure_docker_storage_driver_atomic.sh | 12 +- .../kubernetes/fragments/calico-service.sh | 465 ++++++++++++ .../kubernetes/fragments/configure-etcd.sh | 13 +- .../fragments/configure-kubernetes-master.sh | 115 ++- .../fragments/configure-kubernetes-minion.sh | 169 +++-- .../kubernetes/fragments/core-dns-service.sh | 172 ++++- .../fragments/enable-cert-api-manager.sh | 15 + .../fragments/enable-ingress-controller.sh | 23 + .../kubernetes/fragments/enable-ingress-traefik.sh | 165 +++++ .../fragments/enable-prometheus-monitoring | 434 ----------- .../fragments/enable-prometheus-monitoring.sh | 484 +++++++++++++ .../kubernetes/fragments/enable-services-master.sh | 18 +- .../kubernetes/fragments/flannel-config-service.sh | 73 ++ .../kubernetes/fragments/flannel-service.sh | 82 +++ .../fragments/kube-apiserver-to-kubelet-role.sh | 86 +++ .../kubernetes/fragments/kube-dashboard-service.sh | 491 ++++++++++--- .../fragments/kube-system-namespace-service.sh | 71 -- .../kubernetes/fragments/make-cert-client.sh | 115 ++- .../templates/kubernetes/fragments/make-cert.sh | 124 ++-- .../kubernetes/fragments/network-config-service.sh | 79 -- .../kubernetes/fragments/network-service.sh | 82 --- .../kubernetes/fragments/wc-notify-master.sh | 7 +- .../kubernetes/fragments/write-flannel-config.sh | 28 + .../fragments/write-heat-params-master.yaml | 24 + .../kubernetes/fragments/write-heat-params.yaml | 6 + .../kubernetes/fragments/write-kube-os-config.sh | 7 + .../kubernetes/fragments/write-network-config.sh | 28 - .../common/templates/swarm/fragments/make-cert.py | 42 +- magnum/drivers/heat/driver.py | 20 +- magnum/drivers/heat/k8s_fedora_template_def.py | 45 +- magnum/drivers/heat/k8s_template_def.py | 14 +- magnum/drivers/heat/swarm_mode_template_def.py | 53 +- magnum/drivers/heat/template_def.py | 28 +- magnum/drivers/k8s_coreos_v1/driver.py | 6 +- .../fragments/write-heat-params-master.yaml | 1 + .../templates/fragments/write-heat-params.yaml | 1 + .../k8s_coreos_v1/templates/kubecluster.yaml | 22 +- .../k8s_coreos_v1/templates/kubemaster.yaml | 12 + .../k8s_coreos_v1/templates/kubeminion.yaml | 7 + magnum/drivers/k8s_fedora_atomic_v1/driver.py | 6 +- .../templates/kubecluster.yaml | 170 ++++- .../k8s_fedora_atomic_v1/templates/kubemaster.yaml | 267 ++++++- .../k8s_fedora_atomic_v1/templates/kubeminion.yaml | 49 +- magnum/drivers/k8s_fedora_ironic_v1/driver.py | 6 +- .../templates/kubecluster.yaml | 18 +- .../k8s_fedora_ironic_v1/templates/kubemaster.yaml | 38 +- .../templates/kubeminion_software_configs.yaml | 13 +- .../templates/swarmcluster.yaml | 14 +- .../templates/swarmmaster.yaml | 2 +- .../templates/swarmnode.yaml | 2 +- magnum/objects/__init__.py | 6 +- magnum/objects/federation.py | 215 ++++++ magnum/objects/fields.py | 26 + .../api/controllers/v1/test_cluster_template.py | 32 + .../unit/api/controllers/v1/test_federation.py | 415 +++++++++++ .../conductor/handlers/common/test_cert_manager.py | 234 +++++- .../conductor/handlers/test_cluster_conductor.py | 2 +- .../handlers/test_federation_conductor.py | 38 + .../handlers/test_k8s_cluster_conductor.py | 200 ++++- .../handlers/test_mesos_cluster_conductor.py | 78 +- .../handlers/test_swarm_cluster_conductor.py | 92 ++- playbooks/pre/prepare-workspace-images.yaml | 8 +- ...nd-client-incompatibility-fdfeab326dfda3bf.yaml | 20 + .../notes/add-federation-api-cf55d04f96772b0f.yaml | 11 + .../allow-multimaster-no-fip-b11520485012d949.yaml | 5 + ...broken-kuberenetes-client-d2d1da6029825208.yaml | 4 + ...284-k8s-fedora-admin-user-e760f9b0edf49391.yaml | 8 + ...02728-kube-os-conf-region-46cd60537bdabdb2.yaml | 6 + ...-trustee-auth-region-name-37796a4e6a274fb8.yaml | 8 + .../calico-network-driver-0199c2459041ae81.yaml | 5 + .../notes/cert-manager-api-ee0cf7f3b767bb5d.yaml | 5 + .../notes/client-embed-certs-322701471e4d6e1d.yaml | 6 + ...er_template_update_labels-10ce66c87795f11c.yaml | 7 + ...ure-etcd-auth-bug-1759813-baac5e0fe8a2e97f.yaml | 7 + .../notes/dns-autoscale-90b63e3d71d7794e.yaml | 8 + ...able_cloud_provider_label-ed79295041bc46a8.yaml | 7 + ...ion-for-k8s-multi-masters-29bd36de57df355a.yaml | 13 + .../notes/ingress-controller-552ea956ceabdd25.yaml | 8 + ...atomic_apply_cluster_role-8a46c881de1a1fa3.yaml | 5 + ...8s_fedora_protect_kubelet-8468ddcb92c2a624.yaml | 12 + .../strip-ca-certificate-a09d0c31c45973df.yaml | 7 + ...t-octavia-for-k8s-service-d5d7fd041f9d76fa.yaml | 6 + ...nt-keys-for-multi-masters-71217c4cf4dd472c.yaml | 9 + ...date-kubernetes-dashboard-5196831c32d55aee.yaml | 8 + .../notes/update-to-f27-cc8aa873cdf111bc.yaml | 10 + .../upgrade-to-k8s-v1.11.1-8065fd768873295d.yaml | 5 + releasenotes/source/index.rst | 1 + .../locale/en_GB/LC_MESSAGES/releasenotes.po | 806 +++++++++++++++++++++ .../source/locale/fr/LC_MESSAGES/releasenotes.po | 64 -- releasenotes/source/queens.rst | 6 + requirements.txt | 28 +- setup.cfg | 8 - specs/containers-service.rst | 2 +- test-requirements.txt | 9 +- tools/pretty_tox.sh | 16 - tox.ini | 59 +- 175 files changed, 7977 insertions(+), 1389 deletions(-)
Requirements updates --------------------
diff --git a/requirements.txt b/requirements.txt index cf3e3d1..0e5ffaa 100644 --- a/requirements.txt +++ b/requirements.txt @@ -9 +9 @@ Babel!=2.4.0,>=2.3.4 # BSD -PyYAML>=3.10 # MIT +PyYAML>=3.12 # MIT @@ -18 +18 @@ enum34>=1.0.4;python_version=='2.7' or python_version=='2.6' or python_version== -eventlet!=0.18.3,!=0.20.1,<0.21.0,>=0.18.2 # MIT +eventlet!=0.18.3,!=0.20.1,>=0.18.2 # MIT @@ -21 +21 @@ jsonpatch!=1.20,>=1.16 # BSD -keystoneauth1>=3.3.0 # Apache-2.0 +keystoneauth1>=3.4.0 # Apache-2.0 @@ -23 +23 @@ keystonemiddleware>=4.17.0 # Apache-2.0 -kubernetes>=4.0.0 # Apache-2.0 +kubernetes>=5.0.0 # Apache-2.0 @@ -26,2 +26,2 @@ netaddr>=0.7.18 # BSD -oslo.concurrency>=3.20.0 # Apache-2.0 -oslo.config>=5.1.0 # Apache-2.0 +oslo.concurrency>=3.26.0 # Apache-2.0 +oslo.config>=5.2.0 # Apache-2.0 @@ -31 +31 @@ oslo.i18n>=3.15.3 # Apache-2.0 -oslo.log>=3.30.0 # Apache-2.0 +oslo.log>=3.36.0 # Apache-2.0 @@ -38 +38 @@ oslo.utils>=3.33.0 # Apache-2.0 -oslo.versionedobjects>=1.28.0 # Apache-2.0 +oslo.versionedobjects>=1.31.2 # Apache-2.0 @@ -43 +43 @@ pycadf!=2.0.0,>=1.1.0 # Apache-2.0 -python-barbicanclient!=4.5.0,!=4.5.1,>=4.0.0 # Apache-2.0 +python-barbicanclient>=4.5.2 # Apache-2.0 @@ -46 +46 @@ python-heatclient>=1.10.0 # Apache-2.0 -python-neutronclient>=6.3.0 # Apache-2.0 +python-neutronclient>=6.7.0 # Apache-2.0 @@ -50 +50 @@ requests>=2.14.2 # Apache-2.0 -setuptools!=24.0.0,!=34.0.0,!=34.0.1,!=34.0.2,!=34.0.3,!=34.1.0,!=34.1.1,!=34.2.0,!=34.3.0,!=34.3.1,!=34.3.2,!=36.2.0,>=16.0 # PSF/ZPL +setuptools!=24.0.0,!=34.0.0,!=34.0.1,!=34.0.2,!=34.0.3,!=34.1.0,!=34.1.1,!=34.2.0,!=34.3.0,!=34.3.1,!=34.3.2,!=36.2.0,>=21.0.0 # PSF/ZPL @@ -53,3 +53,3 @@ stevedore>=1.20.0 # Apache-2.0 -taskflow>=2.7.0 # Apache-2.0 -cryptography!=2.0,>=1.9 # BSD/Apache-2.0 -Werkzeug>=0.7 # BSD License +taskflow>=2.16.0 # Apache-2.0 +cryptography>=2.1 # BSD/Apache-2.0 +Werkzeug>=0.9 # BSD License diff --git a/test-requirements.txt b/test-requirements.txt index 12535af..209b0c4 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -15,2 +15 @@ mock>=2.0.0 # BSD -openstackdocstheme>=1.17.0 # Apache-2.0 -oslotest>=1.10.0 # Apache-2.0 +oslotest>=3.2.0 # Apache-2.0 @@ -18 +16,0 @@ osprofiler>=1.4.0 # Apache-2.0 -os-api-ref>=1.4.0 # Apache-2.0 @@ -22 +20 @@ pytz>=2013.6 # MIT -sphinx>=1.6.2 # BSD +sphinx!=1.6.6,!=1.6.7,>=1.6.2 # BSD @@ -23,0 +22 @@ testrepository>=0.0.18 # Apache-2.0/BSD +stestr>=2.0.0 # Apache-2.0 @@ -26,2 +24,0 @@ testtools>=2.2.0 # MIT -# releasenotes -reno>=2.5.0 # Apache-2.0