Open Stack

We enthusiastically announce the release of:

octavia 9.1.0: OpenStack Octavia Scalable Load Balancer as a Service

This release is part of the xena stable release series.

The source is available from:

    https://opendev.org/openstack/octavia

Download the package from:

    https://pypi.org/project/octavia

Please report issues through:

    https://storyboard.openstack.org/#!/project/908

For more details, please see below.

9.1.0
^^^^^


Known Issues
************

* When using a distribution with a recent SELinux release such as
  CentOS 8 Stream, PING health-monitor does not work as shell_exec_t
  calls are denied by SELinux.

* Fixed configuration issue which allowed authenticated and
  authorized users to inject code into HAProxy configuration using API
  requests. Octavia API no longer accepts unencoded whitespace
  characters in url_path values in update requests for healthmonitors.


Upgrade Notes
*************

* The fix that updates the Netfilter Conntrack Sysfs variables
  requires rebuilding the amphora image in order to be effective.


Bug Fixes
*********

* Increased the TCP buffer memory maximum and enabled MTU ICMP black
  hole detection.

* The generated RSyslog configuration on the amphora supports now
  RSyslog failover with TCP if multiple RSyslog servers were
  specified.

* In order to avoid hitting the Neutron API hard when batch update
  with creating many new members, we cache the subnet validation
  results in batch update members API call. We also change to validate
  new members only during batch update members since subnet ID is
  immutable.

* Ensure that the provided rsyslog configuration file is used by the
  rsyslog by restarting the service, it fixes the log offloading
  feature on distributions that start rsyslog before cloud-init.

* Ensure that the provided rsyslog configuration file is used by
  rsyslog in the amphora by restarting the service when using the
  amphorav1 provider, it fixes the log offloading feature on
  distributions that start rsyslog before cloud-init.

* Fixed issues when building amphora image for Centos Stream 9.

* Fixed issues when building amphora image for RHEL 9.

* Fix an authentication error with Barbican when creating a
  TERMINATED_HTTPS listener with application credential tokens or
  trust IDs.

* Correctly detect the member operating status "drain" when querying
  status data from HAProxy.

* Fix the shutdown of the driver-agent, the process might have been
  stuck while waiting for threads to finish. Systemd would have killed
  the process after a timeout, but some children processes might have
  leaked on the controllers.

* Enable required SELinux booleans for CentOS or RHEL amphora image.

* Fixed backwards compatibility issue with the feature that
  preserves HAProxy server states between reloads. HAProxy version 1.5
  or below do not support this feature, so Octavia will not to
  activate it on amphorae with those versions.

* Fix a bug that prevented the provisioning_state of a health-
  monitor to be set to ERROR when an error occurred while creating,
  updating or deleting a health-monitor.

* Fix an issue with IPv6 members that could have been set in
  operating_status "ERROR" just after being added.

* Fix an issue with amphorav2 and persistence, some long tasks
  executed by a controller might have been released in taskflow and
  rescheduled on another controller. Octavia now ensures that a task
  is never released early by using a keepalive mechanism to notify
  taskflow (and its redis backend) that a job is still running.

* Fixed an issue with members in ERROR operating status that may
  have been updated briefly to ONLINE during a Load Balancer
  configuration change.

* Netfilter Conntrack Sysfs variables net.netfilter.nf_conntrack_max
  and nf_conntrack_expect_max get set to sensible values on the
  amphora now. Previously, kernel default values were used which were
  much too low for the configured net.netfilter.nf_conntrack_buckets
  value. As a result packets could get dropped because the conntrack
  table got filled too quickly. Note that this affects only UDP and
  SCTP protocol listeners. Connection tracking is disabled for TCP-
  based connections on the amphora including HTTP(S).

* Fix a bug when adding a member on a subnet that belongs to a
  network with multiple subnets, an incorrect subnet may have been
  plugged in the amphora.

* Fix a bug when deleting the last member plugged on a network, the
  port that was no longer used was not deleted.

* Fix a bug when updating a load balancer with a QoS policy after a
  failover, Octavia attempted to update the VRRP ports of the deleted
  amphorae, moving the provisioning status of the load balancer to
  ERROR.

* Fix a potential race condition when updating a resource in the
  amphorav2 worker. The worker was not waiting for the resource to be
  set to PENDING_UPDATE, so the resource may have been updated with
  old data from the database, resulting in a no-op update.

* Fixed issue with SELinux and the lvs-masquerade.sh script on the
  amphora. The script already runs with root permissions, so the use
  of sudo inside the script is unneeded.

* Fix an issue when Octavia performs a failover of an ACTIVE-STANDBY
  load balancer that has both amphorae missing. Some tasks in the
  controller took too much time to timeout because the timeout value
  defined in "[haproxy_amphora].active_connection_max_retries" and
  "[haproxy_amphora].active_connection_rety_interval" was not used.

* Fix a bug that could have triggered a race condition when
  configuring a member interface in the amphora. Due to a race
  condition, a network interface might have been deleted from the
  amphora, leading to a loss of connectivity.

* Fixed "Could not retrieve certificate" error when
  updating/deleting the client_ca_tls_container_ref field of a
  listener after a CA/CRL was deleted.

* Fixed validations in L7 rule and session cookie APIs in order to
  prevent authenticated and authorized users to inject code into
  HAProxy configuration. CR and LF (\r and \n) are no longer allowed
  in L7 rule keys and values. The session persistence cookie names
  must follow the rules described in https://developer.mozilla.org/en-
  US/docs/Web/HTTP/Headers/Set-Cookie.

* Fix load balancers stuck in PENDING_UPDATE issues for some API
  calls (POST /l7rule, PUT /pool) when a provider denied the call.

* Validate that the creation of L7 policies is compatible with the
  protocol of the listener in the Amphora driver. L7 policies are
  allowed for Terminated HTTPS or HTTP protocol listeners, but not for
  HTTPS, TCP or UDP protocols listeners.


Other Notes
***********

* The string representation of data base model objects has been
  improved. Calling str() on them will return a certain subset of
  fields and calling repr() on them will return all fields. This is
  helpful for debugging, but it may also change some of the log
  messages that Octavia emits.

Changes in octavia 9.0.1..9.1.0
-------------------------------

fc55d6cf Increase TCP buffer maximum and MTU discovery
bc2bded8 Handle feature compatibility of HAProxy server-state-file option
e6dad97a Restart rsyslog from cloud-init in amphorav1
980b8e79 Change FIPS jobs to centos-9-stream
851510f4 Fix barbican client with application credentials/trusts
c87ff96d Add *.orig to .gitignore
9c8e5e03 Fix sporadic unit test failure
cde823eb Update zuul queue configuration
16fa7f8e Fix plugging member subnets on existing networks
c513c9e7 Reconfigure amphora network interfaces seamlessly
9311c825 Ignore status update on deleted objects in driver-agent
15852b00 Cache subnets validation for batch member update
81cb9d5e Fix bug when rolling back prov and op status for some API calls
8b28293d Fix PortNotFound exception when updating a LB after a failover
279bc0b6 Catch exceptions on I/O in driver-agent
044de8e5 Apply openstack-selinux policies in Centos amphorae
aa1c69a5 Fix update/delete listener CA/CRL error
e33414db Fix driver-agent cleanup
11b9d8ae Move system scoped secure-RBAC to separate file
c7aa79cd Fix HealthMonitorToErrorOnRevertTask revert method
9ed984f8 Improve string representation of DB models
4b87554b Fix potential race conditions on update requests in the v2 worker
5647db94 Fix duplicate object error messages
33eaff55 Set sensible nf_conntrack_max value in amphora
7f960f0e Correct format of release note
c30aa9df Validate L7Rule value and cookie name
d668e1f2 Fix new pylint issues
04207f60 Reject invalid whitespace in HM url_path value
acd30aeb Remove unneeded sudo in lvs-masquerade.sh
5997304d Fix compile_amphora_details when using UDP listeners
df1ecbda Deny the creation of L7Policies for HTTPS/TCP/UDP listeners
7adeb21c Fix AttributeError in exception handler
edcd6931 Save the HAProxy state outside of its systemd unit
220f13f5 Restart rsyslog from cloud-init
4e1cc209 Pass timeout_dict to _get_haproxy_versions
5ade96be Fix Amphora RSyslog configuration for TCP failover
903b9a76 Use centos amphora image in the FIPS jobs
6958254a Fix ipv6 interface configuration
887ffeae Fix unplugging member ports
3a744f0c Fix amphora-agent elements for RHEL9
2f9fe00e Optimize DB object to provider dict conversions
9718e036 Fix detection of member operating status DRAIN
28220096 Fix amphora build in CentOS Stream 9
24ebc652 Preserve haproxy server states during reloads
f78d7b0d Add fips jobs
58eac3d9 Update scripts to use fips allowed algorithms
987d6a34 Add keepalive for redis-based taskflow boards
b47d5dfc Fix nft command line with negative priority values


Diffstat (except docs and test files)
-------------------------------------

.gitignore                                         |   2 +
.pylintrc                                          |   4 -
bin/create_dual_intermediate_CA.sh                 |  10 +-
bin/create_single_CA_intermediate_CA.sh            |   6 +-
devstack/plugin.sh                                 |   5 +-
diskimage-create/README.rst                        |   2 +
diskimage-create/diskimage-create.sh               |  21 +-
elements/amphora-agent/pkg-map                     |  21 +
.../static/usr/local/bin/lvs-masquerade.sh         |  12 +-
elements/amphora-fips/README.rst                   |   7 +
elements/amphora-fips/element-deps                 |   4 +
elements/amphora-fips/environment.d/95-enable-fips |  28 +
elements/amphora-fips/package-installs.yaml        |   2 +
elements/amphora-fips/pkg-map                      |  10 +
.../amphora-fips/post-install.d/10-enable-fips     |  22 +
elements/amphora-selinux/README.rst                |   3 +
elements/amphora-selinux/element-deps              |   2 +
elements/amphora-selinux/package-installs.json     |   4 +
elements/amphora-selinux/pkg-map                   |  12 +
.../post-install.d/50-selinux-policies             |  19 +
.../post-install.d/20-haproxy-tune-kernel          |   9 +-
.../haproxy-octavia/pre-install.d/01-repositories  |   2 +-
etc/policy/README.rst                              |  12 +
etc/policy/keystone_default_roles-policy.yaml      |   6 +-
.../keystone_default_roles_scoped-policy.yaml      |  37 +
.../backends/agent/api_server/amphora_info.py      |   8 +-
.../backends/agent/api_server/loadbalancer.py      |  14 +
.../amphorae/backends/agent/api_server/osutils.py  |  26 +-
octavia/amphorae/backends/agent/api_server/plug.py |  78 +-
.../amphorae/backends/agent/api_server/server.py   |   3 +-
.../api_server/templates/amphora-netns.systemd.j2  |   3 +
octavia/amphorae/backends/agent/api_server/util.py |   4 +
octavia/amphorae/backends/utils/haproxy_query.py   |  40 +-
octavia/amphorae/backends/utils/interface.py       | 247 +++++-
octavia/amphorae/backends/utils/interface_file.py  | 106 ++-
octavia/amphorae/backends/utils/network_utils.py   |   3 +-
octavia/amphorae/drivers/driver_base.py            |   9 +-
.../amphorae/drivers/haproxy/rest_api_driver.py    | 104 ++-
.../drivers/keepalived/vrrp_rest_driver.py         |   3 +-
octavia/amphorae/drivers/noop_driver/driver.py     |  11 +-
octavia/api/drivers/amphora_driver/v1/driver.py    |  13 +
octavia/api/drivers/amphora_driver/v2/driver.py    |  13 +
.../api/drivers/driver_agent/driver_listener.py    |  99 +--
octavia/api/drivers/driver_agent/driver_updater.py |  14 +-
octavia/api/drivers/utils.py                       |  12 +-
octavia/api/v2/controllers/l7rule.py               |   2 +-
octavia/api/v2/controllers/listener.py             |  14 +-
octavia/api/v2/controllers/member.py               |  40 +-
octavia/api/v2/controllers/pool.py                 |   2 +-
octavia/api/v2/types/l7rule.py                     |  13 +-
octavia/api/v2/types/pool.py                       |  14 +-
octavia/certificates/common/auth/barbican_acl.py   |  27 +-
octavia/common/base_taskflow.py                    |  19 +-
octavia/common/constants.py                        |  14 +-
.../jinja/haproxy/combined_listeners/jinja_cfg.py  |   6 +
.../haproxy/combined_listeners/templates/base.j2   |   3 +
.../combined_listeners/templates/haproxy.cfg.j2    |   2 +-
.../haproxy/combined_listeners/templates/macros.j2 |   5 +-
.../logging/templates/10-rsyslog.conf.template     |  36 +-
.../templates/user_data_config_drive.template      |   5 +
octavia/common/validate.py                         |   1 +
octavia/controller/worker/v1/controller_worker.py  |   3 +
.../controller/worker/v1/flows/amphora_flows.py    |   7 +-
.../worker/v1/flows/load_balancer_flows.py         |   4 +-
octavia/controller/worker/v1/flows/member_flows.py |  31 +-
.../worker/v1/tasks/amphora_driver_tasks.py        |  14 +-
.../controller/worker/v1/tasks/compute_tasks.py    |   6 +-
.../controller/worker/v1/tasks/lifecycle_tasks.py  |   2 +-
.../controller/worker/v1/tasks/network_tasks.py    | 257 +++++--
octavia/controller/worker/v2/controller_worker.py  | 168 +++--
.../controller/worker/v2/flows/amphora_flows.py    |   7 +-
.../worker/v2/flows/load_balancer_flows.py         |   4 +-
octavia/controller/worker/v2/flows/member_flows.py |  31 +-
.../worker/v2/tasks/amphora_driver_tasks.py        |  16 +-
.../controller/worker/v2/tasks/compute_tasks.py    |   6 +-
.../controller/worker/v2/tasks/lifecycle_tasks.py  |   2 +-
.../controller/worker/v2/tasks/network_tasks.py    | 258 +++++--
octavia/db/base_models.py                          |   7 +
octavia/db/models.py                               |  63 ++
octavia/network/base.py                            |  31 +-
octavia/network/data_models.py                     |   5 +-
.../drivers/neutron/allowed_address_pairs.py       |  27 +-
octavia/network/drivers/neutron/base.py            |  32 +
octavia/network/drivers/noop_driver/driver.py      |  76 +-
.../backend/agent/api_server/test_server.py        |  55 +-
.../api/drivers/driver_agent/test_driver_agent.py  |  37 +
.../backends/agent/api_server/test_amphora_info.py |  13 +-
.../backends/agent/api_server/test_loadbalancer.py |   5 +-
.../backends/agent/api_server/test_osutils.py      |  16 +
.../backends/agent/api_server/test_plug.py         | 152 +++-
.../amphorae/backends/utils/test_haproxy_query.py  |  34 +-
.../unit/amphorae/backends/utils/test_interface.py | 332 +++++++-
.../amphorae/backends/utils/test_interface_file.py | 160 +++-
.../drivers/haproxy/test_rest_api_driver_0_5.py    |  60 +-
.../drivers/haproxy/test_rest_api_driver_1_0.py    |  90 ++-
.../amphorae/drivers/noop_driver/test_driver.py    |   4 +-
.../api/drivers/amphora_driver/v1/test_driver.py   |  20 +-
.../api/drivers/amphora_driver/v2/test_driver.py   |  20 +-
.../drivers/driver_agent/test_driver_listener.py   | 173 ++++-
.../certificates/common/auth/test_barbican_acl.py  |   3 +-
.../haproxy/combined_listeners/test_jinja_cfg.py   |  15 +-
.../common/jinja/logging/test_logging_jinja_cfg.py |  53 +-
.../unit/common/jinja/test_user_data_jinja_cfg.py  |  18 +-
.../sample_configs/sample_configs_combined.py      |  21 +-
.../common/sample_configs/sample_configs_split.py  |  20 +-
.../worker/v1/flows/test_amphora_flows.py          |   8 +-
.../worker/v1/flows/test_load_balancer_flows.py    |  20 +-
.../worker/v1/flows/test_member_flows.py           |  25 +-
.../worker/v1/tasks/test_amphora_driver_tasks.py   |  18 +-
.../worker/v1/tasks/test_compute_tasks.py          |  41 +-
.../worker/v1/tasks/test_lifecycle_tasks.py        |   2 +-
.../worker/v1/tasks/test_network_tasks.py          | 835 ++++++++++++++++++---
.../controller/worker/v1/test_controller_worker.py |   5 +
.../worker/v2/flows/test_amphora_flows.py          |   4 +-
.../worker/v2/flows/test_load_balancer_flows.py    |  17 +-
.../worker/v2/flows/test_member_flows.py           |  21 +-
.../worker/v2/tasks/test_amphora_driver_tasks.py   |  26 +-
.../worker/v2/tasks/test_compute_tasks.py          |  41 +-
.../worker/v2/tasks/test_lifecycle_tasks.py        |   2 +-
.../worker/v2/tasks/test_network_tasks.py          | 803 ++++++++++++++++----
.../controller/worker/v2/test_controller_worker.py | 273 ++++++-
.../drivers/neutron/test_allowed_address_pairs.py  |  77 +-
.../unit/network/drivers/neutron/test_base.py      | 141 ++++
.../network/drivers/noop_driver/test_driver.py     |  67 +-
playbooks/enable-fips.yaml                         |   3 +
...mtu-black-hole-detection.-0640432a7202400f.yaml |   5 +
...-support-rsyslog-failover-f8bf00e0bf0fc27e.yaml |   5 +
.../notes/catch_validation-27ffe48ca187c46f.yaml   |   8 +
...syslog-config-is-reloaded-b4a25a98b661d0f1.yaml |   6 +
...syslog-reloaded-amphorav1-a4ec5127a459f3bf.yaml |   7 +
...o-support-centos-stream-9-e4c8599ae152d396.yaml |   4 +
...amphora-to-support-rhel-9-b10091e81b48533a.yaml |   4 +
...tial-tokens-with-barbican-3b7d13283206c124.yaml |   5 +
...ix-drain-status-detection-b9395fa4fe8c936f.yaml |   5 +
...r-agent-graceful-shutdown-daff9ffaccb09a9e.yaml |   7 +
...nforced-selinux-on-centos-27842ca6afbb500c.yaml |   4 +
...y-about-server-state-file-df70e5ac859417e2.yaml |   7 +
...itor-to-error-revert-task-feb38ba7641a4892.yaml |   6 +
...6-interface-configuration-61b1bd7d2c962cea.yaml |   5 +
...asks-with-redis-keepalive-af18211334c14f54.yaml |   8 +
...perating-status-on-reload-fe3688603bae8726.yaml |   5 +
...onntrack-max-value-in-amp-0e16eb50b42e7b58.yaml |  15 +
...x-plugging-member-subnets-8560cd9403ff79a7.yaml |   8 +
...-qos-apply-after-failover-561abbd153ab88ee.yaml |   6 +
...race-condiction-on-update-b5330c8fcf1800cd.yaml |   7 +
...ue-with-lvs-masquerade.sh-ebbb89886148c70f.yaml |   6 +
...ut-dict-in-failover-tasks-537456e0fe1d7cb8.yaml |   9 +
...x-unplugging-member-ports-262b35426e570edd.yaml |   7 +
...-update-listener-ca-error-167464debc06cba2.yaml |   5 +
...rules-and-session-cookies-cb88f3f1b90171f9.yaml |   9 +
...UPDATE-on-provider-errors-40a03adc8ef82a54.yaml |   5 +
...epresentation-of-db-model-1c4fe799186b4dea.yaml |   7 +
.../ping-healthcheck-selinux-e3b7d360c8503527.yaml |   6 +
...-protocols-for-l7policies-83d678171f13136a.yaml |   7 +
...rl_path-value-in-requests-3eb3adedcd696433.yaml |   7 +
zuul.d/jobs.yaml                                   |  31 +
zuul.d/projects.yaml                               |   8 +-
170 files changed, 5473 insertions(+), 1101 deletions(-)