We eagerly announce the release of: kolla-ansible 14.9.0: Ansible Deployment of Kolla containers This release is part of the yoga stable release series. The source is available from: https://opendev.org/openstack/kolla-ansible Download the package from: https://tarballs.openstack.org/kolla-ansible/ Please report issues through: https://bugs.launchpad.net/kolla-ansible/+bugs For more details, please see below. 14.9.0 ^^^^^^ New Features ************ * Since CVE-2022-29404 is fixed the default value for the LimitRequestBody directive in the Apache HTTP Server has been changed from 0 (unlimited) to 1073741824 (1 GiB). This limits the size of images (for example) uploaded in Horizon. Now this limit can be configured via "horizon_httpd_limitrequestbody". LP#2012588 * Adds support for deploying OpenSearch and OpenSearch dashboards. These services directly replace ElasticSearch and Kibana which are now end-of-life. Support for sending logs to a remote ElasticSearch (or OpenSearch) cluster is maintained. * Adds support for migrating from Elasticsearch to OpenSearch by running "kolla-ansible opensearch-migration" command. * etcd is now exposed internally via HAProxy on "etcd_client_port". * Added two new flags to alter behaviour in RabbitMQ: * *rabbitmq_message_ttl_ms*, which lets you set a TTL on messages. * *rabbitmq_queue_expiry_ms*, which lets you set an expiry time on queues. See https://www.rabbitmq.com/ttl.html for more information on both. * The config option *rabbitmq_ha_replica_count* is added, to allow for changing the replication factor of mirrored queues in RabbitMQ. While the flag is unset, the queues are mirrored across all nodes using "ha-mode":"all". Note that this only has an effect if the flag ` om_enable_rabbitmq_high_availability` is set to *True*, as otherwise queues are not mirrored. * The config option *rabbitmq_ha_promote_on_shutdown* has been added, which allows changing the RabbitMQ definition *ha-promote-on- shutdown*. By default *ha-promote-on-shutdown* is "when-synced". We recommend changing this to be "always". This basically means we don't mind losing some messages, instead we give priority to rabbitmq availability. This is most relevant when restarting rabbitmq, such as when upgrading. Note that setting the value of this flag, even to the default value of "when-synced", will cause RabbitMQ to be restarted on the next deploy. For more details please see: https://www.rabbitmq.com/ha.html#cluster-shutdown * Services using etcd3gw via tooz now use etcd via haproxy. This removes a single point of failure, where we hardcoded the first etcd host for backend_url. Upgrade Notes ************* * Default tags of "neutron_tls_proxy" and "glance_tls_proxy" have been changed to "haproxy_tag", as both services are using "haproxy" container image. Any custom tag overrides for those services should be altered before upgrade. Security Issues *************** * The kolla-genpwd, kolla-mergepwd, kolla-readpwd and kolla-writepwd commands now creates or updates passwords.yml with correct permissions. Also they display warning message about incorrect permissions. Bug Fixes ********* * The precheck for RabbitMQ failed incorrectly when "kolla_externally_managed_cert" was set to "true". LP#1999081 * Fixes create sasl account before config file is ready. LP#2015589 * The flags "--db-nb-pid" and "--db-sb-pid" have been corected to be " --db-nb-pidfile" and "--db-sb-pidfile" respectively. See here for reference: https://github.com/ovn- org/ovn/blob/6c6a7ad1c64a21923dc9b5bea7069fd88bcdd6a8/utilities/ovn- ctl#L1045 LP#2018436 * Configuration of service user tokens for all Nova and Cinder services is now done automatically, to ensure security of block- storage volume data. See LP#[2004555] for more details. * Adds configuration necessary for application credential access rules to properly function. LP#1965111 * Fixes deployment when using Ansible check mode. LP#2002661 * Fixes the incorrect endpoint URLs and service type information for the Cyborg service in the Keystone. LP#2020080 * Set the etcd internal hostname and cacert for tls internal enabled deployments. This allows services to work with etcd when coordination is enabled for TLS interal deployments. Without this fix, the coordination backend fails to connect to etcd and the service itself crashes. * fix missing [taskflow] section in masakari.conf.j2 LP#1966536 * Fixes opensearch migration process. Including case when elasticsearch is located in regular folder instead of docker volume. Furthermore it now has checks if there is data to migrate. * When upgrading or deploying RabbitMQ, the policy *ha-all* is cleared if *om_enable_rabbitmq_high_availability* is set to *false*. Changes in kolla-ansible 14.8.0..14.9.0 --------------------------------------- e885a17d3 Add keystone_authtoken.service_type 09a8f69ce Fix the Cyborg service 64c77ef01 Fix passwords.yml permissions f72ff9bc6 docs: Remove redundant section about vagrant-vbguest 7e13d5756 Fix Bash variable expansion issues in openrc file cb105dc29 always add service_user section to nova.conf 10a6388d0 Correct ovn-ctl --db-nb-pidfile usage in templates ed0d288c1 Designate: provide certificates file to use for neutron client requests ca25ca18e Fix faulty precheck for RabbitMQ ff1bd03ff Add precheck to fail if RabbitMQ HA needs configuring 3e14223d9 cli: fix find globals.d 458dcbfdb Remove RabbitMQ ha-all policy when not required e8f608d36 Fix create sasl account before config file is ready 486367ec6 Fix maximum width of the DIB Multiline-YAML 0b9e5cc88 opensearch: default dashboards tag to opensearch_tag c8f11df80 Fix merge action plugins verbose output 3930e70ae Add note about removing leading tabs in ceph.conf files 8affe4ed3 Add flags for RabbitMQ message TTL & queue expiry 7aae29b4d Fix deploy/genconfig in check mode 7d78a6cc3 Fix elasticsearch to opensearch migration b74d331aa nova: Fix live migration on RHEL9 derivatives 17fb40506 Add LimitRequestBody configuration for Horizon eca595c18 Add flag to change RabbitMQ ha-mode definition f01896ffd RabbitMQ: Support setting ha-promote-on-shutdown 6fab166fd Pin zun jobs to Docker 20 9b2a23824 Add OpenSearch ca0233589 CI: Drop monasca job 3d7eb1e20 ironic: fix dev mode for inspector f7364d96c Use haproxy-config instead of loadbalancer-config 9479ba016 Set the etcd internal hostname and cacert for tls internal enabled deployments b47c013cf hacluster: Use nodename to align with nova service names e21caa492 rocky: add to distro_python_version_map and update CI jobs ae8031154 CI: Always build images for centos jobs 928ef9cc5 Add [taskflow] section for masakari.conf.j2 d25e160cf Default neutron_tls_proxy and glance_tls_proxy to haproxy_tag edd0d07e3 Use loadbalancer to connect to etcd 5388aaced Add CentOS Stream 9 / Rocky Linux 9 host support 6373d5419 Fix docker version precheck (bad backport) fb0271e39 Put etcd behind HTTP loadbalancer 29926808a docs: fix information about libvirt SASL auth 87899a2f1 rocky - add to multiple if base_distro statements Diffstat (except docs and test files) ------------------------------------- README.rst | 4 +- ansible/action_plugins/merge_configs.py | 10 +- ansible/action_plugins/merge_yaml.py | 27 ++- ansible/group_vars/all.yml | 47 +++- ansible/inventory/all-in-one | 8 + ansible/inventory/multinode | 8 + ansible/opensearch-migration.yml | 168 ++++++++++++++ ansible/roles/aodh/templates/aodh.conf.j2 | 1 + ansible/roles/barbican/templates/barbican.conf.j2 | 1 + ansible/roles/bifrost/tasks/config.yml | 1 + ansible/roles/blazar/templates/blazar.conf.j2 | 1 + ansible/roles/cinder/templates/cinder-wsgi.conf.j2 | 2 +- ansible/roles/cinder/templates/cinder.conf.j2 | 9 +- ansible/roles/cloudkitty/defaults/main.yml | 2 +- .../roles/cloudkitty/templates/cloudkitty.conf.j2 | 1 + ansible/roles/common/defaults/main.yml | 9 + ansible/roles/common/tasks/config.yml | 6 + ansible/roles/common/templates/admin-openrc.sh.j2 | 30 +-- .../common/templates/conf/output/00-local.conf.j2 | 31 +++ .../common/templates/conf/output/01-es.conf.j2 | 2 +- .../templates/conf/output/03-opensearch.conf.j2 | 33 +++ .../templates/cron-logrotate-opensearch.conf.j2 | 3 + ansible/roles/cyborg/defaults/main.yml | 8 +- ansible/roles/cyborg/templates/cyborg.conf.j2 | 1 + .../roles/designate/templates/designate.conf.j2 | 2 + ansible/roles/destroy/tasks/cleanup_host.yml | 1 + ansible/roles/etcd/defaults/main.yml | 8 +- ansible/roles/etcd/tasks/loadbalancer.yml | 7 + ansible/roles/freezer/defaults/main.yml | 4 +- ansible/roles/freezer/templates/freezer.conf.j2 | 1 + ansible/roles/glance/defaults/main.yml | 3 +- ansible/roles/glance/templates/glance-api.conf.j2 | 1 + ansible/roles/gnocchi/templates/gnocchi.conf.j2 | 1 + ansible/roles/grafana/defaults/main.yml | 12 + .../roles/hacluster/tasks/bootstrap_service.yml | 8 +- .../hacluster/templates/hacluster_corosync.conf.j2 | 2 +- ansible/roles/heat/templates/heat.conf.j2 | 1 + .../roles/heat/templates/wsgi-heat-api-cfn.conf.j2 | 2 +- ansible/roles/heat/templates/wsgi-heat-api.conf.j2 | 2 +- ansible/roles/horizon/templates/horizon.conf.j2 | 5 +- ansible/roles/ironic/defaults/main.yml | 4 +- ansible/roles/ironic/tasks/clone.yml | 3 +- .../roles/ironic/templates/ironic-api-wsgi.conf.j2 | 2 +- .../ironic/templates/ironic-inspector.conf.j2 | 1 + ansible/roles/ironic/templates/ironic-tftp.json.j2 | 11 +- ansible/roles/ironic/templates/ironic.conf.j2 | 1 + ansible/roles/keystone/tasks/config.yml | 1 + .../roles/keystone/templates/wsgi-keystone.conf.j2 | 2 +- ansible/roles/loadbalancer/tasks/precheck.yml | 26 +++ ansible/roles/magnum/templates/magnum.conf.j2 | 1 + ansible/roles/manila/templates/manila.conf.j2 | 1 + ansible/roles/mariadb/tasks/lookup_cluster.yml | 1 + ansible/roles/masakari/templates/masakari.conf.j2 | 4 + ansible/roles/mistral/templates/mistral.conf.j2 | 1 + .../monasca/templates/monasca-api/api.conf.j2 | 1 + ansible/roles/murano/templates/murano.conf.j2 | 1 + ansible/roles/neutron/defaults/main.yml | 3 +- ansible/roles/neutron/templates/neutron.conf.j2 | 1 + ansible/roles/nova-cell/handlers/main.yml | 14 +- .../roles/nova-cell/tasks/get_cell_settings.yml | 1 + .../nova-cell/tasks/wait_discover_computes.yml | 1 + ansible/roles/nova-cell/templates/nova.conf.j2 | 13 ++ ansible/roles/nova-cell/templates/sshd_config.j2 | 3 + ansible/roles/nova/templates/nova-api-wsgi.conf.j2 | 2 +- ansible/roles/nova/templates/nova.conf.j2 | 14 ++ .../roles/octavia/templates/octavia-openrc.sh.j2 | 16 +- .../roles/octavia/templates/octavia-wsgi.conf.j2 | 2 +- ansible/roles/octavia/templates/octavia.conf.j2 | 1 + ansible/roles/opensearch/defaults/main.yml | 132 +++++++++++ ansible/roles/opensearch/handlers/main.yml | 33 +++ .../roles/opensearch/tasks/check-containers.yml | 18 ++ ansible/roles/opensearch/tasks/check.yml | 1 + ansible/roles/opensearch/tasks/config-host.yml | 17 ++ ansible/roles/opensearch/tasks/config.yml | 63 +++++ ansible/roles/opensearch/tasks/config_validate.yml | 1 + ansible/roles/opensearch/tasks/copy-certs.yml | 6 + .../roles/opensearch/tasks/deploy-containers.yml | 2 + ansible/roles/opensearch/tasks/deploy.yml | 12 + ansible/roles/opensearch/tasks/loadbalancer.yml | 7 + ansible/roles/opensearch/tasks/main.yml | 2 + ansible/roles/opensearch/tasks/precheck.yml | 25 ++ ansible/roles/opensearch/tasks/pull.yml | 3 + ansible/roles/opensearch/tasks/reconfigure.yml | 2 + ansible/roles/opensearch/tasks/register.yml | 7 + ansible/roles/opensearch/tasks/stop.yml | 6 + ansible/roles/opensearch/tasks/upgrade.yml | 48 ++++ .../templates/opensearch-dashboards.json.j2 | 23 ++ .../roles/opensearch/templates/opensearch.json.j2 | 23 ++ .../roles/opensearch/templates/opensearch.yml.j2 | 21 ++ .../templates/opensearch_dashboards.yml.j2 | 12 + ansible/roles/opensearch/vars/main.yml | 2 + ansible/roles/openvswitch/defaults/main.yml | 2 +- ansible/roles/ovn-db/templates/ovn-nb-db.json.j2 | 2 +- ansible/roles/ovn-db/templates/ovn-sb-db.json.j2 | 2 +- .../placement/templates/placement-api-wsgi.conf.j2 | 2 +- .../roles/placement/templates/placement.conf.j2 | 1 + ansible/roles/prechecks/tasks/package_checks.yml | 1 + ansible/roles/prechecks/vars/main.yml | 3 + .../prometheus-elasticsearch-exporter.json.j2 | 2 +- ansible/roles/rabbitmq/defaults/main.yml | 15 ++ ansible/roles/rabbitmq/tasks/deploy.yml | 4 + ansible/roles/rabbitmq/tasks/precheck.yml | 28 +++ .../roles/rabbitmq/tasks/remove-ha-all-policy.yml | 28 +++ ansible/roles/rabbitmq/tasks/upgrade.yml | 23 +- .../roles/rabbitmq/templates/definitions.json.j2 | 4 +- ansible/roles/sahara/templates/sahara.conf.j2 | 1 + ansible/roles/senlin/templates/senlin.conf.j2 | 1 + ansible/roles/solum/templates/solum.conf.j2 | 1 + ansible/roles/tacker/templates/tacker.conf.j2 | 1 + ansible/roles/telegraf/defaults/main.yml | 1 + ansible/roles/telegraf/templates/telegraf.conf.j2 | 5 + ansible/roles/trove/templates/trove.conf.j2 | 1 + ansible/roles/vitrage/templates/vitrage.conf.j2 | 1 + ansible/roles/watcher/templates/watcher.conf.j2 | 1 + ansible/roles/zun/templates/zun.conf.j2 | 1 + ansible/site.yml | 23 ++ ...rst => central-logging-guide-elasticsearch.rst} | 10 +- .../central-logging-guide-opensearch.rst | 258 +++++++++++++++++++++ .../reference/logging-and-monitoring/index.rst | 3 +- .../logging-and-monitoring/monasca-guide.rst | 4 + .../reference/storage/external-ceph-guide.rst | 8 + etc/kolla/globals.yml | 3 + etc/kolla/passwords.yml | 5 + kolla_ansible/cmd/genpwd.py | 19 +- kolla_ansible/cmd/mergepwd.py | 24 +- kolla_ansible/cmd/readpwd.py | 19 +- kolla_ansible/cmd/writepwd.py | 10 + ...-horizon-limitrequestbody-4f79433fa2cf1f6d.yaml | 9 + .../notes/add-opensearch-53ef174195acce45.yaml | 10 + .../notes/bug-1999081-769f1012263a48fd.yaml | 6 + .../notes/bug-2015589-94427c14cd857c98.yaml | 5 + ...-ctl-pid-flag-in-template-d915fe4b71548da0.yaml | 8 + .../notes/cve-2023-2088-51e7e050be2139bf.yaml | 9 + .../etcd-tcp-loadbalancer-08d9332ee3be9a8b.yaml | 4 + ...fix-app-cred-access-rules-14b5dcfcd5a5669a.yaml | 6 + .../notes/fix-check-mode-1f37ab5507c98954.yaml | 5 + .../notes/fix-cyborg-service-5bc5bf8748daf504.yaml | 6 + ...-etcd-coordination-config-b1c9f900ef13be13.yaml | 8 + ...-missing-taskflow-section-31b6654e29bec35d.yaml | 5 + .../fix-opensearch-migration-114061f943f60b71.yaml | 6 + ...passwords-yml-permissions-ca73638b71aeadf4.yaml | 7 + ...sage-ttl-and-queue-expiry-c163a370708f5b20.yaml | 7 + ...change-replication-factor-321c2f9e08e7d179.yaml | 9 + ...mq-ha-promote-on-shutdown-9099c6643f2d0cce.yaml | 13 ++ ...-policy-when-not-required-81dcf64542c4805f.yaml | 5 + ...s-proxies-use-haproxy-tag-aa030b5e5df6fbf0.yaml | 8 + ...r-for-etcdgw-coordination-6704a8b1389bbabe.yaml | 6 + roles/cephadm/defaults/main.yml | 1 + roles/cephadm/tasks/pkg_debian.yml | 1 + roles/cephadm/tasks/pkg_redhat.yml | 7 + tools/cleanup-host | 5 + tools/kolla-ansible | 8 +- tools/validate-all-file.py | 4 + zuul.d/base.yaml | 14 +- zuul.d/jobs.yaml | 212 ++++++++++++++++- zuul.d/nodesets.yaml | 52 +++-- zuul.d/project.yaml | 25 +- 170 files changed, 2262 insertions(+), 157 deletions(-)