We are excited to announce the release of: octavia 6.1.0: OpenStack Octavia Scalable Load Balancer as a Service This release is part of the ussuri stable release series. The source is available from: https://opendev.org/openstack/octavia Download the package from: https://pypi.org/project/octavia Please report issues through: https://storyboard.openstack.org/#!/project/908 For more details, please see below. 6.1.0 ^^^^^ New Features ************ * Add a new configuration option to define the default connection_limit for new listeners that use the Amphora provider. The option is [haproxy_amphora].default_connection_limit and its default value is 50,000. This value is used when creating or setting a listener with -1 as connection_limit parameter, or when unsetting connection_limit parameter. Upgrade Notes ************* * The failover improvements do not require an updated amphora image, but updating existing amphora will minimize the failover outage time for standalone amphora on subsequent failovers. Security Issues *************** * If you are using the admin_or_owner-policy.yaml policy override file you should upgrade your API processes to include the unscoped token fix. The default policies are not affected by this issue. Bug Fixes ********* * Fixed an issue with failing over an amphora if the pair amphora in an active/standby pair had a missing VRRP port in neutron. * Fixed an issue where setting of SNI containers were not being applied on listener update API calls. * Fixed an Octavia API validation on listener update where SNI containers could be set on non-TERMINATED_HTTPS listeners. * Fixed an issue where some columns could not be used for sort keys in API list calls. * Fix an issue when the barbican service enable TLS, we create the listerner failed. * Fixed an issue where amphora load balancers fail to create when Nova anti-affinity is enabled and topology is SINGLE. * Fixed an issue where listener "insert_headers" parameter was accepted for protocols that do not support header insertion. * Fixed an issue where UDP only load balancers would not bring up the VIP address. * Fixes an issue when using the admin_or_owner-policy.yaml policy override file and unscoped tokens. * With haproxy 1.8.x releases, haproxy consumes much more memory in the amphorae because of pre-allocated data structures. This amount of memory depends on the maxconn parameters in its configuration file (which is related to the connection_limit parameter in the Octavia API). In the Amphora provider, the default connection_limit value -1 is now converted to a maxconn of 50,000. It was previously 1,000,000 but that value triggered some memory allocation issues when quickly performing multiple configuration updates in a load balancer. * Significantly improved the reliability and performance of amphora and load balancer failovers. This is especially true when the Nova service is experiencing failures. Changes in octavia 6.0.1..6.1.0 ------------------------------- 879b08d3 Ignore DELETED amphorae when performing certificate rotation 8de7e1ec Fixes API list handling of unscoped tokens 2dbd8f69 Fix amphora failover when VRRP port is missing d2fb5e0d Set Grub timeout to 0 for fast boot times 6bc9c4d8 Update amphora v2 for the failover refactor e8282012 Fix memory consumption issues with default connection_limit 8ae3f857 Remove haproxy_check_script for UDP-only LBs 43db3477 Update grenade job to run one smoke test e5cc2493 Fix missing dependency in amphora create flow 9b826fca Fix missing params in amphora base and noop driver 1e2aec16 Add missing reload method in amphora noop driver bfa44b75 Refactor the failover flows 39eb3286 Fix API sort key for complex columns 5e377fa8 Pin pylint for stable branches 17c22145 Increase the devstack secuirty group rules quota f923ef09 Switch back to using git for the agent ref 19557664 Set amphora-agent build-only packages 621acb42 Fix accepting 'insert_headers' when unsupported c3f61aec Fix haproxy user flow log format substitution 274393fd Prioritize policy validation 7c6fef82 Fix listener update with SNI certificates fc6e0b0a Migrate grenade job to native Zuul v3 12e30394 add the verify for the session b4bc92eb Fix neutron subnet lookup ignoring endpoint_type 209726d9 Fix netcat option in udp_check.sh for CentOS/RHEL bd2987ba Fix batch member create for v1 amphora driver Diffstat (except docs and test files) ------------------------------------- .gitignore | 1 + api-ref/source/parameters.yaml | 6 +- devstack/plugin.sh | 5 +- devstack/upgrade/settings | 6 - diskimage-create/diskimage-create.sh | 3 + .../75-amphora-agent-install | 3 + elements/amphora-agent/package-installs.yaml | 8 +- .../post-install.d/90-remove-build-deps | 5 +- etc/octavia.conf | 41 + .../backends/agent/api_server/amphora_info.py | 74 +- .../backends/agent/api_server/keepalived.py | 4 + .../backends/agent/api_server/keepalivedlvs.py | 6 +- .../backends/agent/api_server/loadbalancer.py | 35 +- octavia/amphorae/backends/agent/api_server/util.py | 79 +- .../amphorae/backends/utils/ip_advertisement.py | 183 ++++ .../amphorae/backends/utils/network_namespace.py | 50 + octavia/amphorae/backends/utils/network_utils.py | 83 ++ octavia/amphorae/backends/utils/udp_check.sh | 14 +- octavia/amphorae/drivers/driver_base.py | 67 +- octavia/amphorae/drivers/haproxy/exceptions.py | 7 +- .../amphorae/drivers/haproxy/rest_api_driver.py | 60 +- .../drivers/keepalived/vrrp_rest_driver.py | 79 +- octavia/amphorae/drivers/noop_driver/driver.py | 42 +- octavia/api/common/pagination.py | 15 +- octavia/api/common/types.py | 8 + octavia/api/drivers/amphora_driver/v1/driver.py | 7 +- octavia/api/drivers/amphora_driver/v2/driver.py | 7 +- octavia/api/drivers/utils.py | 35 +- octavia/api/v2/controllers/base.py | 4 + octavia/api/v2/controllers/health_monitor.py | 16 +- octavia/api/v2/controllers/l7policy.py | 27 +- octavia/api/v2/controllers/l7rule.py | 31 +- octavia/api/v2/controllers/listener.py | 11 +- octavia/api/v2/controllers/load_balancer.py | 56 +- octavia/api/v2/controllers/member.py | 14 +- octavia/api/v2/controllers/pool.py | 33 +- octavia/api/v2/types/listener.py | 1 + octavia/certificates/common/auth/barbican_acl.py | 4 +- octavia/common/clients.py | 4 +- octavia/common/config.py | 54 +- octavia/common/constants.py | 62 +- octavia/common/data_models.py | 8 + octavia/common/exceptions.py | 20 +- .../jinja/haproxy/combined_listeners/jinja_cfg.py | 11 +- .../jinja/haproxy/split_listeners/jinja_cfg.py | 4 +- octavia/common/utils.py | 23 + octavia/compute/drivers/noop_driver/driver.py | 8 +- octavia/compute/drivers/nova_driver.py | 40 +- octavia/controller/worker/v1/controller_worker.py | 449 +++++---- .../controller/worker/v1/flows/amphora_flows.py | 766 ++++++++------- .../worker/v1/flows/load_balancer_flows.py | 450 ++++++++- .../worker/v1/tasks/amphora_driver_tasks.py | 220 +++-- .../controller/worker/v1/tasks/compute_tasks.py | 84 +- .../controller/worker/v1/tasks/database_tasks.py | 51 +- .../controller/worker/v1/tasks/network_tasks.py | 219 ++++- octavia/controller/worker/v1/tasks/retry_tasks.py | 74 ++ octavia/controller/worker/v2/controller_worker.py | 423 +++++--- .../controller/worker/v2/flows/amphora_flows.py | 786 ++++++++------- octavia/controller/worker/v2/flows/flow_utils.py | 47 +- .../controller/worker/v2/flows/listener_flows.py | 14 +- .../worker/v2/flows/load_balancer_flows.py | 503 ++++++++-- octavia/controller/worker/v2/flows/pool_flows.py | 9 +- .../worker/v2/tasks/amphora_driver_tasks.py | 267 ++++-- .../controller/worker/v2/tasks/compute_tasks.py | 92 +- .../controller/worker/v2/tasks/database_tasks.py | 124 ++- .../controller/worker/v2/tasks/network_tasks.py | 254 +++-- octavia/controller/worker/v2/tasks/retry_tasks.py | 73 ++ octavia/db/models.py | 4 +- octavia/db/repositories.py | 25 +- octavia/network/base.py | 52 + octavia/network/data_models.py | 16 +- .../drivers/neutron/allowed_address_pairs.py | 289 ++++-- octavia/network/drivers/neutron/base.py | 17 +- octavia/network/drivers/neutron/utils.py | 45 +- octavia/network/drivers/noop_driver/driver.py | 69 ++ octavia/opts.py | 1 + .../backend/agent/api_server/test_keepalivedlvs.py | 12 +- .../backend/agent/api_server/test_server.py | 22 +- .../agent/api_server/test_haproxy_compatibility.py | 6 +- .../backends/agent/api_server/test_loadbalancer.py | 48 +- .../backends/agent/api_server/test_util.py | 128 +++ .../backends/utils/test_ip_advertisement.py | 212 ++++ .../backends/utils/test_network_namespace.py | 116 +++ .../amphorae/backends/utils/test_network_utils.py | 140 +++ .../amphorae/drivers/haproxy/test_exceptions.py | 52 + .../drivers/haproxy/test_rest_api_driver_0_5.py | 34 +- .../drivers/haproxy/test_rest_api_driver_1_0.py | 34 +- .../drivers/haproxy/test_rest_api_driver_common.py | 83 ++ .../drivers/keepalived/test_vrrp_rest_driver.py | 48 +- .../test_noop_amphoraloadbalancer_driver.py | 19 +- .../haproxy/combined_listeners/test_jinja_cfg.py | 137 ++- .../sample_configs/sample_configs_combined.py | 20 +- .../worker/v1/flows/test_amphora_flows.py | 213 +++-- .../worker/v1/flows/test_load_balancer_flows.py | 236 ++++- .../worker/v1/tasks/test_amphora_driver_tasks.py | 286 ++++-- .../worker/v1/tasks/test_compute_tasks.py | 70 +- .../worker/v1/tasks/test_database_tasks.py | 34 +- .../worker/v1/tasks/test_network_tasks.py | 416 +++++++- .../controller/worker/v1/tasks/test_retry_tasks.py | 47 + .../controller/worker/v1/test_controller_worker.py | 899 +++++++++++++---- .../worker/v2/flows/test_amphora_flows.py | 167 ++-- .../worker/v2/flows/test_listener_flows.py | 8 +- .../worker/v2/flows/test_load_balancer_flows.py | 274 +++++- .../controller/worker/v2/flows/test_pool_flows.py | 3 +- .../worker/v2/tasks/test_amphora_driver_tasks.py | 313 ++++-- .../worker/v2/tasks/test_compute_tasks.py | 67 +- .../worker/v2/tasks/test_database_tasks.py | 37 +- .../worker/v2/tasks/test_network_tasks.py | 436 ++++++++- .../controller/worker/v2/tasks/test_retry_tasks.py | 47 + .../controller/worker/v2/test_controller_worker.py | 1011 +++++++++++++++----- .../drivers/neutron/test_allowed_address_pairs.py | 478 ++++++++- .../unit/network/drivers/neutron/test_utils.py | 1 + .../network/drivers/test_network_noop_driver.py | 56 ++ .../legacy/grenade-devstack-octavia/post.yaml | 19 - playbooks/legacy/grenade-devstack-octavia/run.yaml | 62 -- ...ailover-missing-vrrp-port-9b5f13b9951b7edb.yaml | 5 + ...ner-update-sni-containers-6595c52e2de1f621.yaml | 8 + .../notes/fix-api-sort-key-337f342d5cdce432.yaml | 5 + ...ix-barbican-client-verfiy-689be1b9389bd1d8.yaml | 5 + ...amphora-create-dependency-a954ded0d260d462.yaml | 5 + ...header-insertion-mismatch-e3aeb5f5fee0348b.yaml | 5 + .../notes/fix-udp-only-lbs-c4ca42106fc1e2bb.yaml | 5 + .../get-all-unscoped-token-61da95856bc662e0.yaml | 10 + ...ction_limit-config-option-3ed9f0ed6ec2b514.yaml | 18 + .../refactor_failover_flow-9efcd854240f71ad.yaml | 11 + test-requirements.txt | 2 +- tools/create_flow_docs.py | 45 +- tools/flow-list-v2.txt | 32 + tools/flow-list.txt | 3 +- zuul.d/jobs.yaml | 69 +- zuul.d/projects.yaml | 15 +- 149 files changed, 11196 insertions(+), 3022 deletions(-) Requirements updates -------------------- diff --git a/test-requirements.txt b/test-requirements.txt index 7d827764..6fc42870 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -11 +11 @@ oslotest>=3.2.0 # Apache-2.0 -pylint>=2.2.0 # GPLv2 +pylint>=2.2.0,<=2.5.3 # GPLv2