We are happy to announce the release of: kuryr-kubernetes 2.0.0: Kubernetes integration with OpenStack networking This release is part of the ussuri release series. The source is available from: https://opendev.org/openstack/kuryr-kubernetes Download the package from: https://tarballs.openstack.org/kuryr-kubernetes/ Please report issues through: https://bugs.launchpad.net/kuryr-kubernetes/+bugs For more details, please see below. Changes in kuryr-kubernetes 1.1.0..2.0.0 ---------------------------------------- 4ed3177 Make _create_ports keys coherent with Neutron API. 6c39f94 Upgrade centos in dockerfiles and fix OVN gate ca46976 Ensure NP are enforced on SVC with different port and target port 83b4504 Update dockerfiles to use stable/ussuri uc 9307df0 Update TOX_CONSTRAINTS_FILE for stable/ussuri 21c881f Update .gitreview for stable/ussuri 52b216d [ussuri][goal] Update contributor docs 92e1b7f Add option cafile default value in kuryr-controller's config 20f7d24 Ensure LB state annotation sg matches the SG on the LB 5ee781f Use unittest.mock instead of third party mock 92d13c4 Do not configure l3 for VIFVHostUser VIFs c4e47c1 Ensure SG rule is deleted from CRD upon Namespace deletion faa59b8 Remove pod_ip_caching from _kuryr_k8s_opts cc360f4 Add init containers logs ecda137 Remove MEMOIZE from get_pod_ip 9553f78 Disable urllib3 warnings 01ef4ee Fix sg rules creations for LB sg when applying members sg 1b48b76 Add gcc-c++ into containers to fix grpcio build c188427 Remove unused method 24ea735 Change default listening address for health checks. 3588860 Use the right variable in debug message 737e34c Ensure no attempt to deleted sg rules owned by Octavia happens 20eea68 Run on_finalize() for ADDED events 8d162a3 Remove excess physnet to device mapping 62673df Delete neutron ports for Failed/Succeeded pods ec88d2a Fix IPv6 enabled devstack and namespace subnet plugin. fe1627e Preserve unknown in remote_ip_prefixes in NP CRD eeee83d Add IPv6 support to namespace subnet driver 172b256 Autogenerate path for vhostuserclient socket 25fb761 do not die if cannot remove socket file a4e4280 ignore docker files for better docker image reuse 04b6e9b Support DPDK application on bare-metal host 780c4df Namespace event handling through KuryrNet CRD cacd080 Make node annotations with pci addresses optional 7fb7d96 Ensure LB sg rules use IPv6 when enabled ca0c7a8 Remove unrelated health check for sriov driver a0a6e65 Nested CNI: Remove interfaces on DEL requests da1e0aa Log CNI_ARGS and CNI_NETNS in kuryr-cni ede6e82 Update class hierarchy for dpdk driver 93657a9 Remove sgId from KuryrNet CRD 05599e8 Update class hierarchy for sriov driver 2e18045 Make package repo configurable 2c2b927 Disable coredns deployment fd3f4c5 Raise K8sResourceNotFound for all client methods ded6b6d Removing six library. 7707f83 Remove remaining notions of python-neutronclient 6e3e331 Add IPv6 devstack support for namespace subnet driver b214618 Remove _post_lb_resource() c10468c Update CRDs from apiextensions.k8s.io/v1beta1 to v1 9cdd1c8 Ensures accurate quota calculation during the readiness checks bf50385 Move OpenShift job to experimental 18a495a Ensure list of pods items is retrieved 6679558 Enable trace on kuryr-kubernetes plugin. 986fdc4 Remove namespace_security_groups from opts.py 574f5ea Nested: Detect MTU mismatch 1045bcb Bump hacking to newer version 3303955 Accept non-IP as KUBERNETES_SERVICE_HOST 123b932 Get driver for resource but not physnet ca77073 Basic IPv6 support in DevStack ffe5962 Add info about required RBAC permissions to docs d37f34d Remove ingress permissions from ServiceAccount 6714f21 Deprecate usage of neutron-client in kuryr-kubernetes. 6d46466 Revert "Ensure namespace network resources are deleted even on subnet not found" edc6597 Add DPDK support for nested pods 9acfd8f Remove namespace isolation support 85e542e Ensure no sg rule is repeated on the Network Policy CRD d1a12bf Deprecate unmaintaned features 0814cca Remove openshift routes(Ingress) support d49fd12 Ensure LB with error status is only recreated after deleted f2eb7ef Fix port creation with generated payload. f2a0cf9 Refactor neutron tag resources for macvlan driver. 19d5515 Update exceptions handling for openstacksdk. 7691e94 Ensure namespace network resources are deleted even on subnet not found 6cc473b Remove _get_trunks method for openstacksdk client. c047547 Drop use of USE_SYSTEMD var in devstack plugin 09b0799 Move neutron related code to macvlan driver. be132b1 Use openstacksdk for setting the tags. 3a35761 Use openstacksdk for update_port_pci_info function. b41460c Make OVN job voting 29e4961 Remove get_ports_by_attrs by simply use args in query ports. a4bc6e8 Update neutron_vif driver to use OpenStackSDK. e54ff62 Refactor of os_vif_util module. 8efde71 Ensure lb SG is not updated on member creation b287d33 Remove old CNI handlers 3d32182 Update sriov driver to use OpenStackSDK. c55e3fb Update nested_vlan_vif driver to use OpenStackSDK. c6ddf4f Update nested_vif driver to use OpenStackSDK. b5244a1 Update network_policy driver to use OpenStackSDK. ac74acc Update public_ip driver to use OpenStackSDK. 097780a Update namespace_subnet driver to use OpenStackSDK. 88e5775 Update vif_pool driver to use OpenStackSDK. 7a51dd9 Add ability to query for trunks with specified tags. 705bc44 Stop passing around neutron client object. cb2f8fe Update namespace_security_groups driver to use OpenStackSDK. 687642e Update lbaasv2 driver to use OpenStackSDK. c7b8816 Fix more problem of changing size during dict iteration 453e0d8 Avoid KeyError when deleting NPs 3196021 Add support for listeners on the same port but different protocol 14dbf5b Gather debug info when NetlinkError EEXIST happens 0e581ca Add support to Octavia ACLs 11b7bd4 Make OpenShift gates to use NPs a538eae Fix problem of changing size during dict iteration 674344b Fix iteration over remote_ip_prefixes field c05de06 Ensure only lb with no corresponding svc is cleaned up c581a73 Fix Octavia version detection 75a91cf Add missing parameter Controller in the log message ed6e9c1 Protect from sg Not Found on multiple np enforcement 8c62ce5 Ensure LB member is removed upon pod removal db3fcbb Ensure network leftovers without kuryrnet CRD obj are deleted 1e3ebc9 Set defaults for certs and token on the k8s client fb602cd Bump openstacksdk to 0.36.0 4c34068 Ensure LB sg update is retried when NP is enforced f0b0826 Fix catched exception after transition to OpenStackSDK. 20e14a9 Allow to use Mellanox smart NICs for DPDK 64490e5 Ensure namespace are recreated upon a k8s client failure d2e3aea Ensure leftover LBaaS are deleted upon Controller start 018cbf8 Update the community page. 0e949c6 Fix lower-constraints gate e63ba2b Ensure egress rules include VIP IPs. 64249af Nested CNI: Look for leftover ifaces to remove c963efa Make kuryr-kubernetes-tempest-containerized voting b5fdeea Switch to use Ubuntu Bionic Octavia Amphora a473a54 Upgrade OpenShift on gates to 3.11 630217f Reap zombie child processes in CNI daemon service 7bc2cd2 Bump OpenStackSDK version to 0.17 0660413 Fix misprints in sriov binding driver 14ee23d Ensure pools population does not happen until pools are recovered 1410394 Add support for different loadbalancer algorithms 28b7999 Handle not found subport on Trunk 66b85a6 Fix pool population when namespace isolation is enforced 3703285 Make openshift-dns deployment Python 3 compatible 07e09ce Avoid race between member addition and namespace deletion 22f3030 Add requests.ConnectionError to watcher retries f24d824 Bump pool_maxsize for K8sClient to 1000 2449917 Ensure not found exception is ignored when LB is not present 7ed6e86 Basic Python 3 compatibility fixes c33089c Do not crash on failing to add default route 481f600 Have configurable additional vifs prefix e507d71 Use ifname from CNI request for default vif 11ccc66 Prevent Kuryr restart due to connection issue 20f6a3c Documentation for nested-dpdk case b008a59 Only delete pod from CNI registry after unplugging the vif 1067b9b Fix deletion of ports on a pool 3eba2d3 Fix multinode gate after switch to Python 3 d7d8e9f Ensure last_update is initialized before used 2c41671 Protection from subport with wrong ACTIVE status 3223940 Fix lower-constraints gate after switch to py36 28b27c5 Remove Python 2 support 5a8681a Set explicitly type of language in code-block directive. ad4c460 Correct ordered/unordered lists. 80b5ecd Change inline hyperlinks to link-target pairs. fd440fc Explicitly use code-block. feff260 Fix directives formatting. c3c270c Fix text blocks formatting e32d796 Fix inconsistency in headlines format. 9a214d2 Remove comment about Amphora building being broken bf64e95 Better log message for pools re-population 971b2ee Fix py3 gate 0530ddc Modify Network Policy gate to run all the tests b0b78b4 Ensure LB SG is not updated for egress only policy db1b24f Ensure Network Policy handles egress traffic to a SVC e1a7ddf Added support for bulk create ports. 1b97158 Move from Neutron client to OpenStackSDK. 1fc9cdc Switch to Ussuri jobs d98c2ed Set configmap_modifiable=True in pools gate b53ea87 Use pyroute2 to tweak vf in sriov 6d43ad4 Remove namespace leftover upon kuryr-controller restart d442760 Catch right exception for ConnectFailure at activating vif 998be3b Avoid race between Retries and Deletion actions 5fb1f59 NestedVIFPool: React when status.hostIP is missing 5f77e77 Update K8s version support matrix b95a0c3 Use upper-constraints in docs builds 1f3ee02 Just check k8s 1.16 9c9eac0 Add not-ready tolerations to kuryr pods 9cd4ad3 Avoid controller crash upon unexpected neutron error handling ports 48f2d28 Add a loadbalancer CRD 19580e3 Improve LOG messaging 76fa78c Fix CNI_COMMAND=VERSION case 1183446 Remove unused property from DaemonServerService a5dafba Get rid of v1beta1 API version in ds/deployment f85866d Avoid race between pod creation retry and namespace deletion efae1f5 Avoid namespace deletion error if processing a duplicated event bbf6c29 Avoid race between activating vif and pod deletion 9172f37 Make sure subports are unique when removing them 5a4c7f2 Ensure vif handler retry in case of missing ns resources 9b5fae1 Set subnet_per_namespace d9d81d4 Ensure ports from pool do not reference deleted SGs/NPs 474495f Ensure no KuryrNet addition is tried when namespace is not present 59b84a7 Fix incorrect link to k8s api-conventions 232509e Ensure lb sg rules are updated upon namespace label updates 68145b9 Ensure no LBaaS SG update is triggered for SVCs without selectors and ports 9dfc5a1 Ensure pod IP is retrieved from annotation 3208b19 Fixes race cond. during updates of neutron resources 663300b Update the constraints url 6d56ffc DevStack: Adapt `prefixes` processing 444096e DevStack: Adapt fixed_ips processing 1f3cb1f Update master for stable/train 467d5fc Add region_name to Octavia API connections fcf0d8d Fix the misspelling of "openvswitch" Diffstat (except docs and test files) ------------------------------------- .dockerignore | 2 + .gitreview | 1 + .zuul.d/base.yaml | 3 +- .zuul.d/multinode.yaml | 3 +- .zuul.d/octavia.yaml | 61 +- .zuul.d/project.yaml | 20 +- .zuul.d/sdn.yaml | 25 +- CONTRIBUTING.rst | 27 +- HACKING.rst | 1 + README.rst | 10 +- cni.Dockerfile | 14 +- cni_py3.Dockerfile | 35 - contrib/devstack-heat/README.rst | 45 +- contrib/devstack-heat/hot/node.yaml | 2 +- contrib/kubectl_plugins/README.rst | 4 + contrib/pools-management/README.rst | 7 +- contrib/pools-management/subports.py | 2 +- controller.Dockerfile | 15 +- controller_py3.Dockerfile | 26 - devstack/lib/kuryr_kubernetes | 225 ++-- devstack/local.conf.openshift.sample | 14 - devstack/local.conf.ovn.sample | 1 + devstack/local.conf.pod-in-vm.undercloud.df.sample | 9 +- .../local.conf.pod-in-vm.undercloud.ovn.sample | 9 +- devstack/local.conf.pod-in-vm.undercloud.sample | 9 +- devstack/local.conf.sample | 9 +- devstack/plugin.sh | 210 ++-- devstack/settings | 19 +- .../devref/kuryr_kubernetes_ingress_design.rst | 98 +- .../devref/kuryr_kubernetes_ocp_route_design.rst | 158 --- .../installation/devstack/dragonflow_support.rst | 159 ++- .../installation/devstack/nested-macvlan.rst | 45 +- .../installation/multi_vif_with_npwg_spec.rst | 164 +-- .../installation/testing_nested_connectivity.rst | 71 +- .../installation/testing_sriov_functional.rst | 475 ++++---- kubernetes_crds/kuryr_crds/kuryrloadbalancer.yaml | 197 ++++ kubernetes_crds/kuryr_crds/kuryrnet.yaml | 45 + kubernetes_crds/kuryr_crds/kuryrnetpolicy.yaml | 135 +++ kubernetes_crds/kuryr_crds/kuryrnetwork.yaml | 59 + kubernetes_crds/kuryrnet.yaml | 42 - kubernetes_crds/kuryrnetpolicy.yaml | 118 -- .../network_attachment_definition_crd.yaml | 23 +- kuryr_cni/main.go | 2 +- kuryr_kubernetes/clients.py | 107 +- kuryr_kubernetes/cmd/status.py | 2 - kuryr_kubernetes/cni/api.py | 9 +- kuryr_kubernetes/cni/binding/base.py | 42 +- kuryr_kubernetes/cni/binding/bridge.py | 14 +- kuryr_kubernetes/cni/binding/dpdk.py | 194 ++++ kuryr_kubernetes/cni/binding/nested.py | 97 +- kuryr_kubernetes/cni/binding/sriov.py | 117 +- kuryr_kubernetes/cni/binding/vhostuser.py | 131 +++ kuryr_kubernetes/cni/daemon/service.py | 47 +- kuryr_kubernetes/cni/handlers.py | 97 +- kuryr_kubernetes/cni/health.py | 4 +- kuryr_kubernetes/cni/main.py | 9 +- kuryr_kubernetes/cni/plugins/base.py | 4 +- kuryr_kubernetes/cni/plugins/k8s_cni_registry.py | 28 +- kuryr_kubernetes/cni/utils.py | 5 + kuryr_kubernetes/config.py | 53 +- kuryr_kubernetes/constants.py | 19 +- kuryr_kubernetes/controller/drivers/base.py | 170 +-- .../controller/drivers/default_security_groups.py | 9 - kuryr_kubernetes/controller/drivers/lbaasv2.py | 733 +++++-------- .../drivers/namespace_security_groups.py | 182 --- .../controller/drivers/namespace_subnet.py | 242 ++-- .../controller/drivers/nested_dpdk_vif.py | 75 ++ .../controller/drivers/nested_macvlan_vif.py | 113 +- kuryr_kubernetes/controller/drivers/nested_vif.py | 29 +- .../controller/drivers/nested_vlan_vif.py | 91 +- .../controller/drivers/network_policy.py | 175 ++- .../drivers/network_policy_security_groups.py | 113 +- kuryr_kubernetes/controller/drivers/neutron_vif.py | 58 +- kuryr_kubernetes/controller/drivers/public_ip.py | 81 +- kuryr_kubernetes/controller/drivers/sriov.py | 14 +- kuryr_kubernetes/controller/drivers/utils.py | 185 ++-- kuryr_kubernetes/controller/drivers/vif_pool.py | 485 ++++---- .../controller/handlers/ingress_lbaas.py | 213 ---- .../controller/handlers/kuryrnetwork.py | 161 +++ .../{kuryrnet.py => kuryrnetwork_population.py} | 55 +- kuryr_kubernetes/controller/handlers/lbaas.py | 135 ++- kuryr_kubernetes/controller/handlers/namespace.py | 282 ++--- kuryr_kubernetes/controller/handlers/pipeline.py | 9 +- kuryr_kubernetes/controller/handlers/pod_label.py | 12 +- kuryr_kubernetes/controller/handlers/policy.py | 66 +- kuryr_kubernetes/controller/handlers/vif.py | 98 +- kuryr_kubernetes/controller/ingress/__init__.py | 0 kuryr_kubernetes/controller/ingress/ingress_ctl.py | 159 --- kuryr_kubernetes/controller/managers/health.py | 8 +- kuryr_kubernetes/controller/managers/pool.py | 15 +- kuryr_kubernetes/controller/service.py | 14 +- kuryr_kubernetes/exceptions.py | 14 +- kuryr_kubernetes/handlers/asynchronous.py | 6 +- kuryr_kubernetes/handlers/base.py | 4 +- kuryr_kubernetes/handlers/dispatch.py | 7 +- kuryr_kubernetes/handlers/k8s_base.py | 18 + kuryr_kubernetes/handlers/retry.py | 35 +- kuryr_kubernetes/k8s_client.py | 134 ++- kuryr_kubernetes/objects/base.py | 5 +- kuryr_kubernetes/objects/lbaas.py | 58 - kuryr_kubernetes/objects/route.py | 43 - kuryr_kubernetes/objects/vif.py | 12 + kuryr_kubernetes/opts.py | 11 - kuryr_kubernetes/os_vif_plug_noop.py | 4 + kuryr_kubernetes/os_vif_util.py | 241 ++-- kuryr_kubernetes/platform/__init__.py | 0 kuryr_kubernetes/platform/constants.py | 17 - kuryr_kubernetes/platform/ocp/__init__.py | 0 .../platform/ocp/controller/__init__.py | 0 .../platform/ocp/controller/handlers/__init__.py | 0 .../platform/ocp/controller/handlers/route.py | 255 ----- .../unit/cni/plugins/test_k8s_cni_registry.py | 83 +- .../controller/drivers/test_default_project.py | 2 +- .../drivers/test_default_security_groups.py | 2 +- .../unit/controller/drivers/test_default_subnet.py | 2 +- .../unit/controller/drivers/test_lb_public_ip.py | 294 ++--- .../unit/controller/drivers/test_multi_vif.py | 2 +- .../drivers/test_namespace_security_groups.py | 292 ----- .../controller/drivers/test_namespace_subnet.py | 261 ++--- .../unit/controller/drivers/test_nested_dpdk.py | 228 ++++ .../controller/drivers/test_nested_macvlan_vif.py | 305 +++--- .../unit/controller/drivers/test_nested_vif.py | 30 +- .../controller/drivers/test_nested_vlan_vif.py | 208 ++-- .../unit/controller/drivers/test_network_policy.py | 72 +- .../drivers/test_network_policy_security_groups.py | 24 +- .../unit/controller/drivers/test_neutron_vif.py | 125 +-- .../unit/controller/drivers/test_public_ip.py | 93 +- .../unit/controller/handlers/test_ingress_lbaas.py | 187 ---- .../unit/controller/handlers/test_kuryrnetwork.py | 281 +++++ ...kuryrnet.py => test_kuryrnetwork_population.py} | 70 +- .../unit/controller/handlers/test_namespace.py | 266 +---- .../unit/controller/handlers/test_pipeline.py | 2 +- .../unit/controller/handlers/test_pod_label.py | 2 +- .../unit/controller/ingress/test_ingress_ctl.py | 135 --- .../platform/ocp/controller/handlers/__init__.py | 0 .../platform/ocp/controller/handlers/test_route.py | 428 -------- kuryr_kubernetes/utils.py | 71 +- lower-constraints.txt | 17 +- playbooks/get_amphora_tarball.yaml | 4 +- ...eprecate-handlers-caching-9cdfd772aba9a7ce.yaml | 9 + .../notes/drop-ingress-d78a7a9be8f20da1.yaml | 9 + releasenotes/notes/drop-py27-60f55b6bc1d082bc.yaml | 6 + .../k8s-client-token-default-882ec49d1faffc29.yaml | 11 + .../notes/octavia-acls-7452d3406d75ea15.yaml | 9 + ...e-physical-device-mapping-15d614b70c68fc73.yaml | 8 + releasenotes/source/README.rst | 20 +- releasenotes/source/conf.py | 16 +- releasenotes/source/index.rst | 4 +- releasenotes/source/queens.rst | 6 +- releasenotes/source/rocky.rst | 6 +- releasenotes/source/stein.rst | 6 +- releasenotes/source/train.rst | 6 + requirements.txt | 13 +- setup.cfg | 13 +- test-requirements.txt | 3 +- tools/gate/copy_k8s_logs.sh | 3 +- tools/generate_k8s_resource_definitions.sh | 9 + tox.ini | 16 +- 240 files changed, 10879 insertions(+), 10520 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 00aa883..33d32e2 100644 --- a/requirements.txt +++ b/requirements.txt @@ -9,2 +9,2 @@ pbr!=2.1.0,>=2.0.0 # Apache-2.0 -requests>=2.14.2 # Apache-2.0 -eventlet!=0.18.3,!=0.20.1,!=0.21.0,>=0.18.2 # MIT +requests>=2.18.0 # Apache-2.0 +eventlet>=0.22.0 # MIT @@ -12 +12 @@ netaddr>=0.7.19 # BSD -openstacksdk>=0.13.0 # Apache-2.0 +openstacksdk>=0.36.0 # Apache-2.0 @@ -14 +14 @@ oslo.cache>=1.26.0 # Apache-2.0 -oslo.config>=5.2.0 # Apache-2.0 +oslo.config>=6.1.0 # Apache-2.0 @@ -20 +20 @@ oslo.utils>=3.33.0 # Apache-2.0 -os-vif!=1.8.0,>=1.7.0 # Apache-2.0 +os-vif>=1.12.0 # Apache-2.0 @@ -22 +22 @@ PrettyTable<0.8,>=0.7.2 # BSD -pyroute2>=0.5.3;sys_platform!='win32' # Apache-2.0 (+ dual licensed GPL2) +pyroute2>=0.5.7;sys_platform!='win32' # Apache-2.0 (+ dual licensed GPL2) @@ -24 +23,0 @@ retrying!=1.3.0,>=1.2.3 # Apache-2.0 -six>=1.10.0 # MIT diff --git a/test-requirements.txt b/test-requirements.txt index ffb5526..c1a26c6 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -5 +5 @@ -hacking!=0.13.0,<0.14,>=0.12.0 # Apache-2.0 +hacking>=2.0.0 # Apache-2.0 @@ -10 +9,0 @@ docutils>=0.11 # OSI-Approved Open Source, Public Domain -mock>=2.0.0 # BSD