- openstack-discuss - lists.openstack.org

[OSSA-2024-003] OpenStack Ironic: Unvalidated image data passed to qemu-img (CVE-2024-44082)
by Brian Rosmaita 04 Sep '24

04 Sep '24

======================================================== OSSA-2024-003: Unvalidated image data passed to qemu-img ======================================================== :Date: September 04, 2024 :CVE: CVE-2024-44082 Affects ~~~~~~~ - Ironic: <21.4.3, >=22.0.0 <23.0.2, >=23.1.0 <24.1.2, >=25.0.0 <26.0.1 - Ironic-python-agent: <9.4.2, >=9.5.0 <9.7.1, >=9.8.0 <9.11.1, >=9.12.0 <9.13.1 Description ~~~~~~~~~~~ Dan Smith and Julia Kreger of Red Hat and Jay Faulkner of G-Research noticed a vulnerability in image processing for Ironic, in which a specially crafted image could be used by an authenticated user to exploit undesired behaviors in qemu-img, including possible unauthorized access to potentially sensitive data. Mitigation ---------- The attached patches contain code allowing Ironic and the Ironic-Python-Agent (IPA) to pre-screen images before passing them to qemu-img. You should patch both of them because in some popular deployment configurations, it is possible for images to bypass the Ironic conductor for more efficient image downloads. In situations where it is not possible to patch IPA, there is a new configuration option, ``[conductor]conductor_always_validates_images``. When this option is True, all image downloads are forced to occur through the Ironic conductor, where they are validated. However, this may incur a significant performance hit; hence our recommendation to patch both Ironic and IPA. Patches are available for both Ironic and IPA for all maintained branches, namely, the Dalmatian development branch (current master) through Antelope. For these branches, ``conductor_always_validates_images`` defaults to False. For the unmaintained branches (Zed through Victoria), patches are not available for IPA due to a significant risk of regressions. For these branches, the Ironic patches set ``conductor_always_validates_images`` to True, as this is the only safe way to run Ironic with an un-patched IPA. Cached images ------------- Images in the Ironic image cache should be purged. To purge the cache, stop the Ironic conductor and remove the files in the ``[pxe]/instance_master_path`` directory on that conductor node. Supported image formats ----------------------- A new configuration option, ``[conductor]permitted_image_formats`` controls what image formats will be accepted by Ironic. By default, the allowed formats are 'raw' and 'qcow2', which are the only formats that Ironic is tested with. In previous releases, it was possible to use images of other, unsupported formats with Ironic, but this is now blocked by default due to security issues arising from the image conversion process. It is possible, though not recommended, to expand the list of permitted formats. Alternatively, users requiring the use of an image in an unsupported format can convert the image offline into a supported format before uploading the image to Glance or otherwise making the image available to Ironic. Consult the Ironic documentation for details. Warning ------- As is well-documented, the OpenStack Ironic project does not support the use of ironic-lib for non-Ironic use cases. In its normal supported context, ironic-lib assumes that images have been pre-screened before ironic-lib comes in contact with them. Thus, unsupported use of ironic-lib leaves you vulnerable to the exploit outlined in this OSSA. The Ironic project intends eventually to explicitly remove the vulnerable methods in ironic-lib, but reminds the reader that independent use of ironic-lib is not supported. Bugfix branches --------------- Patches are available for maintained bugfix branches. These patches will be merged to the maintained branches, but no new releases will be triggered from those branches. Patches ~~~~~~~ - https://review.opendev.org/927972 (2023.1/antelope(ironic)) - https://review.opendev.org/927979 (2023.1/antelope(ironic-python-agent)) - https://review.opendev.org/927970 (2023.2/bobcat(ironic)) - https://review.opendev.org/927978 (2023.2/bobcat(ironic-python-agent)) - https://review.opendev.org/927968 (2024.1/caracal(ironic)) - https://review.opendev.org/927976 (2024.1/caracal(ironic-python-agent)) - https://review.opendev.org/927965 (2024.2/dalmatian(ironic)) - https://review.opendev.org/927974 (2024.2/dalmatian(ironic-python-agent)) - https://review.opendev.org/927969 (Bugfix/24.0 (ironic)) - https://review.opendev.org/927967 (Bugfix/25.0 (ironic)) - https://review.opendev.org/927966 (Bugfix/26.0 (ironic)) - https://review.opendev.org/927983 (Bugfix/9.12 (ironic-python-agent)) - https://review.opendev.org/927981 (Bugfix/9.13 (ironic-python-agent)) - https://review.opendev.org/927984 (Bugfix/9.9 (ironic-python-agent)) - https://review.opendev.org/927982 (Unmaintained/victoria(ironic)) - https://review.opendev.org/927980 (Unmaintained/wallaby(ironic)) - https://review.opendev.org/927977 (Unmaintained/xena(ironic)) - https://review.opendev.org/927975 (Unmaintained/yoga(ironic)) - https://review.opendev.org/927973 (Unmaintained/zed(ironic)) Credits ~~~~~~~ - Dan Smith from Red Hat (CVE-2024-44082) - Jay Faulkner from G-Research (CVE-2024-44082) - Julia Kreger from Red Hat (CVE-2024-44082) References ~~~~~~~~~~ - https://launchpad.net/bugs/2071740 - http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-44082 Notes ~~~~~ - For more information about the unadvisability of calling 'qemu-img info' on untrusted images, see CVE-2024-04467. - The ironic unmaintained/* branches will receive no point releases, but patches for them are provided as a courtesy. - The ironic-python-agent unmaintained/* branches will receive no point releases, nor are patches for them available. - The ironic and ironic-python-agent bugfix/* branches will receive no point releases, but patches for them are provided as a courtesy. -- Brian Rosmaita OpenStack Vulnerability Management Team https://security.openstack.org/vmt.html

1 0

[keystoneauth] OpenID Connect support for Onelogin broke Okta
by Pavlo Shchelokovskyy 04 Sep '24

04 Sep '24

Hi all, I'd like to bring your attention and discuss the following bug https://bugs.launchpad.net/keystoneauth/+bug/2078437 When onboarding Caracal release internally, we've noticed a regression in keystoneauth, namely that openstack CLI using v3OidcPassword auth no longer works with Okta provider. Okta returns the following error: HTTP 401 {"error":"invalid_request","error_description":"Cannot supply multiple client credentials. Use one of the following: credentials in the Authorization header, credentials in the post body, or a client_assertion in the post body." It looks like this is the aftermath of the following patch https://review.opendev.org/c/openstack/keystoneauth/+/881969 that presumably aimed to fix the Onelogin provider. I've prepared a patch that should fix both my Okta use case and, to my understanding, also address the original Onelogin concern, but I do not obviously have the use case and Onelogin access the original author of the patch had, and also it seems the author has changed jobs and is no longer active in OpenStack. Maybe somebody still has such an access to try and validate the original Onelogin use case using my patch? Or at least validate my conclusions I've outlined in the bug comments on why this should work and satisfy both cases: https://review.opendev.org/c/openstack/keystoneauth/+/927581 CC Graeme Moss who has ported that original Onelogin patch to Antelope and Zed and thus might have some additional perspective. Best regards, -- Dr. Pavlo Shchelokovskyy Principal Software Engineer Mirantis Inc www.mirantis.com

1 0

[largescale-sig] Next meeting: August 21, 9utc
by Thierry Carrez 04 Sep '24

04 Sep '24

Hi everyone, The Large Scale SIG will be meeting this Wednesday in #openstack-operators on OFTC IRC, at 9UTC. You can doublecheck how that UTC time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20240821T09 Feel free to add topics to the agenda: https://etherpad.opendev.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez

3 2

Subject: Best Deployment Method for Large-Scale Production OpenStack
by Thamanna Farhath 04 Sep '24

04 Sep '24

Hello OpenStack Community, We are preparing for a large-scale OpenStack deployment in production and currently use Kolla Ansible. Before proceeding, we want to know if Kolla Ansible is the best choice for production or if there are better alternatives for large-scale deployments. Any recommendations or insights on the best deployment methods for production environments would be greatly appreciated. Thank you! Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

4 3

[neutron] "eventlet" deprecation status
by Rodolfo Alonso Hernandez 03 Sep '24

03 Sep '24

Hello Neutrinos: During the last week we have started the "journey" to deprecate "eventlet" in Neutron and the stadium projects. We have two fronts to deal with: the Neutron API and the Neutron agents. The Neutron agents are spawned using "oslo.service" ``ServiceLauncher``, based on "eventlet". It is first needed the deprecation of "eventlet" in this project. The Neutron API was still using, by default, the "eventlet" server. The migration to the WSGI module, to the detriment of the "eventlet" implementation, was required. Most of the main project APIs have already migrated. The Neutron API using ML2/OVS was already migrated to WSGI but it was not fully functional. The periodic workers were not running, at least when spawned with devstack: * https://review.opendev.org/c/openstack/devstack/+/922125 It was also noticeable the instability of some ML2/OVS tempest CI jobs. That was caused by the RPC event handler implementation in the Neutron API: * https://review.opendev.org/c/openstack/neutron/+/926922 The Neutron API using ML2/OVN was not working at all. It was first needed to refactor the way to initialize the OVN plugin: * https://review.opendev.org/c/openstack/neutron/+/922470 ML2/OVN was also missing the maintenance worker, that was not running when using WSGI: * https://review.opendev.org/c/openstack/devstack/+/922611 * https://review.opendev.org/c/openstack/neutron/+/922074 With ML2/OVN, that usually doesn't need RPC communication, was a problem with the RPC worker (my bad for so many iterations...): * https://review.opendev.org/c/openstack/devstack/+/924473 * https://review.opendev.org/c/openstack/devstack/+/924693 * https://review.opendev.org/c/openstack/devstack/+/925412 The TLS CI job running with ML2/OVN was failing too: * https://review.opendev.org/c/openstack/neutron/+/925376/ The next steps: * To test the Neutron API using ML2/OVN (that doesn't use oslo.messaging for RPC) without any "eventlet" call (at least not calling the monkey patch method). That will be a POC patch only, but enough for testing. * Once we have new versions of the oslo libraries, we'll start with the deprecation in the agents. Regards.

1 0

[all][rbac][tacker][horizon][neutron][octavia] oslo.policy 4.4.0 enable the RBAC new defaults by default
by Ghanshyam Mann 03 Sep '24

03 Sep '24

Hi All, You might have seen some fixes and testing changes on your project to test the oslo.policy 4.4.0. oslo.policy 4.4.0 enables the RBAC new defaults by default, which means those will be enabled for all the OpenStack services unless they have disabled them by overriding the default value. It enables both enforce_scope as well enforce_new_defaults. The requirement change[1] adds the oslo.policy 4.4.0 in upper-constraints which will enable the latest oslo.policy in upstream CI. Along with jobs failing on requirement change, I tested more integration jobs and fixing the failures[2]. Most of the fixes have been merged, and a few of them are up for review. Below is the list of fixes up for review, please merge them before the requirement change is merged. If there is something else I missed to test and failed, let me know on IRC or here. In-progress fixes: - Horizon: - https://review.opendev.org/c/openstack/horizon/+/927571 - Tacker: - https://review.opendev.org/c/openstack/tacker/+/926089/5 - Neutron + Designates job (making it non-voting until we figure out the root cause): - https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/927648 - Octavia (Michael is working on more modifications on this ): - https://review.opendev.org/c/openstack/octavia-tempest-plugin/+/926867 [1] https://review.opendev.org/c/openstack/requirements/+/925464 [2] https://review.opendev.org/q/topic:%22secure-rbac%22+owner:gmann@ghanshyamm… -gmann

2 4

[ops][kolla][kolla-ansible] Lets Encrypt Deployment on 2024.1/v18
by Forrest Fuqua 03 Sep '24

03 Sep '24

The documentation seems a little incomplete when it comes to deploying a stack with let's encrypt The attached config is after trying a few settings, but the lets encrypt container errors out every time during deployment causing haproxy to not get a proper ssl certificate the services.d entries are showing its ignoring kolla_internal_fqdn_cert and still using kolla_internal.pem in most services causing the following deployment error TASK [service-ks-register : keystone | Creating services] **************************************************************************************************************************************************** FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (5 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (4 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (3 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (2 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (1 retries left). failed: [100.70.0.1] (item=keystone (identity)) => {"action": "os_keystone_service", "ansible_loop_var": "item", "attempts": 5, "changed": false, "item": {"description": "Openstack Identity Service", "endpoints": [{"interface": "internal", "url": "https://openstack.cyberrange.rit.edu:5000"}, {"interface": "public", "url": "https://openstack.cyberrange.rit.edu:5000"}], "name": "keystone", "type": "identity"}, "module_stderr": "Failed to discover available identity versions when contacting https://openstack.cyberrange.rit.edu:5000. Attempting to parse version from URL.\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 715, in urlopen\n httplib_response = self._make_request(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 404, in _make_request\n self._validate_conn(conn)\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 1058, in _validate_conn\n conn.connect()\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connection.py\", line 419, in connect\n self.sock = ssl_wrap_socket(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/util/ssl_.py\", line 449, in ssl_wrap_socket\n ssl_sock = _ssl_wrap_socket_impl(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/util/ssl_.py\", line 493, in _ssl_wrap_socket_impl\n return ssl_context.wrap_socket(sock, server_hostname=server_hostname)\n File \"/usr/lib/python3.10/ssl.py\", line 513, in wrap_socket\n return self.sslsocket_class._create(\n File \"/usr/lib/python3.10/ssl.py\", line 1100, in _create\n self.do_handshake()\n File \"/usr/lib/python3.10/ssl.py\", line 1371, in do_handshake\n self._sslobj.do_handshake()\nssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n resp = conn.urlopen(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 799, in urlopen\n retries = retries.increment(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/util/retry.py\", line 592, in increment\n raise MaxRetryError(_pool, url, error or ResponseError(cause))\nurllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1021, in _send_request\n resp = self.session.request(method, url, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n resp = self.send(prep, **send_kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n r = adapter.send(request, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/requests/adapters.py\", line 517, in send\n raise SSLError(e, request=request)\nrequests.exceptions.SSLError: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/generic/base.py\", line 133, in _do_create_plugin\n disc = self.get_discovery(session,\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 605, in get_discovery\n return discover.get_discovery(session=session, url=url,\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/discover.py\", line 1459, in get_discovery\n disc = Discover(session, url, authenticated=authenticated)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/discover.py\", line 539, in __init__\n self._data = get_version_data(session, url,\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/discover.py\", line 106, in get_version_data\n resp = session.get(url, headers=headers, authenticated=authenticated)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1154, in get\n return self.request(url, 'GET', **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 930, in request\n resp = send(**kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1025, in _send_request\n raise exceptions.SSLError(msg)\nkeystoneauth1.exceptions.connection.SSLError: SSL exception connecting to https://openstack.cyberrange.rit.edu:5000: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/tmp/ansible-tmp-1720019606.1431425-10481-181523778594420/AnsiballZ_catalog_service.py\", line 107, in <module>\n _ansiballz_main()\n File \"/tmp/ansible-tmp-1720019606.1431425-10481-181523778594420/AnsiballZ_catalog_service.py\", line 99, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/tmp/ansible-tmp-1720019606.1431425-10481-181523778594420/AnsiballZ_catalog_service.py\", line 47, in invoke_module\n runpy.run_module(mod_name='ansible_collections.openstack.cloud.plugins.modules.catalog_service', init_globals=dict(_module_fqn='ansible_collections.openstack.cloud.plugins.modules.catalog_service', _modlib_path=modlib_path),\n File \"/usr/lib/python3.10/runpy.py\", line 224, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib/python3.10/runpy.py\", line 96, in _run_module_code\n _run_code(code, mod_globals, init_globals,\n File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 211, in <module>\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 207, in main\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/module_utils/openstack.py\", line 415, in __call__\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 113, in run\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 175, in _find\n File \"/opt/ansible/lib/python3.10/site-packages/openstack/service_description.py\", line 89, in __get__\n proxy = self._make_proxy(instance)\n File \"/opt/ansible/lib/python3.10/site-packages/openstack/service_description.py\", line 289, in _make_proxy\n found_version = temp_adapter.get_api_major_version()\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/adapter.py\", line 352, in get_api_major_version\n return self.session.get_api_major_version(auth or self.auth, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1289, in get_api_major_version\n return auth.get_api_major_version(self, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 497, in get_api_major_version\n data = get_endpoint_data(discover_versions=discover_versions)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 268, in get_endpoint_data\n service_catalog = self.get_access(session).service_catalog\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 131, in get_access\n self.auth_ref = self.get_auth_ref(session)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/generic/base.py\", line 203, in get_auth_ref\n self._plugin = self._do_create_plugin(session)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/generic/base.py\", line 155, in _do_create_plugin\n raise exceptions.DiscoveryFailure(\nkeystoneauth1.exceptions.discovery.DiscoveryFailure: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. SSL exception connecting to https://openstack.cyberrange.rit.edu:5000: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1} #config excerpts from globals.yml kolla_base_distro: "ubuntu" kolla_internal_vip_address: "100.70.0.20" # /etc/hosts updated on all hosts to have openstack.cyberrange.rit.edu point to the internal IP kolla_internal_fqdn: "openstack.cyberrange.rit.edu" kolla_external_vip_address: "129.21.246.130" kolla_external_fqdn: "openstack.cyberrange.rit.edu" kolla_enable_tls_internal: "yes" kolla_enable_tls_external: "yes" kolla_certificates_dir: "/etc/kolla/certificates" kolla_external_fqdn_cert: "{{ kolla_certificates_dir }}/haproxy.pem" kolla_internal_fqdn_cert: "{{ kolla_certificates_dir }}/haproxy.pem" #kolla_admin_openrc_cacert: "" kolla_copy_ca_into_containers: "yes" haproxy_backend_cacert: "{{ 'ca-certificates.crt' if kolla_base_distro in ['debian', 'ubuntu'] else 'ca-bundle.trust.crt' }}" haproxy_backend_cacert_dir: "/etc/ssl/certs" letsencrypt_email: "fffics(a)rit.edu" enable_letsencrypt: yes letsencrypt_cert_server: "https://acme-v02.api.letsencrypt.org/directory" # attempt to renew Let's Encrypt certificate every 12 hours letsencrypt_cron_renew_schedule: "0 */12 * * *"

3 2

Best Deployment Method for Large-Scale Production OpenStack
by Thamanna Farhath 03 Sep '24

03 Sep '24

Hello OpenStack Community, We are preparing for a large-scale OpenStack deployment in production and currently use Kolla Ansible. Before proceeding, we want to know if Kolla Ansible is the best choice for production or if there are better alternatives for large-scale deployments. Any recommendations or insights on the best deployment methods for production environments would be greatly appreciated. Thank you! Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

2 1

[release][PTLs] 2024.2 / Dalmatian Cycle Highlights
by Elõd Illés 03 Sep '24

03 Sep '24

Hi, This is a reminder, that *this week* is the week for Cycle highlights [1][2]! They need to be added to deliverable yamls so that they can be included in release marketing preparations. (See the details about how to add them at the project team guide [3]) [1] https://releases.openstack.org/dalmatian/schedule.html [2] https://releases.openstack.org/dalmatian/schedule.html#d-cycle-highlights [3] https://docs.openstack.org/project-team-guide/release-management.html#cycle… Thanks, Előd Illés irc: elodilles @ #openstack-release

1 0

[openstack-ansible] ussuri to victoria major upgrade
by Ram Ram 03 Sep '24

03 Sep '24

Hi Team I am planning openstack-ansible major upgrade from current version Ussuri to next sequential version Victoria. I am checking below link for the same. https://docs.openstack.org/openstack-ansible/victoria/admin/upgrades/major-… And I need support regarding below points: 1. What are the pre-requisite backups to be taken before the upgrade? 2. What is the impact of the upgrade , are all openstack services upgraded parallel or one by one. Example: there are 3 control nodes having each openstack control service lxc container (neutron, nova-api, cinder) running, so at a time one service of openstack is upgraded on each control node example nova (control-1)--->nova(control02)--->nova(control03) or all together. 3. What is the downtime expected in this upgrade? 4. How are openstack services upgraded on the compute nodes? What is the impact on VM running on the compute node that is being upgraded? 5. Any suggestion or experience which can be helpful for the upgrade. 6. What should be the rollback plan in case some issue occurs? Regards Rambo

2 1

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2024.2/R-4)
by Goutham Pacha Ravi 02 Sep '24

02 Sep '24

Hello Stackers, Last week was the feature freeze [1] within OpenStack projects that follow a "cycle-with-rc" release model [2]. These project teams are currently closing out a few feature freeze exceptions and working on bug fixes and documentation so they can ship their 2024.2 "Dalmatian" release candidates next week. When this happens, these projects can begin merging feature changes to the trunk branches again as they target the next release, 2025.1 "Epoxy." The release schedule for Epoxy has been published [3]. Members of the OpenInfra community are gathering in Suwon, South Korea, for the OpenInfra Summit on the 3rd and 4th of September 2024 [4]. This turned out to be a "sold-out" event; we're excited to hear about this event from folks participating in person. It's election season in the OpenStack community. We're electing four members to the Technical Committee [5]. The polling window officially opens on 2024-09-04. If you've been an individual member of the OpenInfra Foundation and have contributed to any OpenStack project within the past year, you should have received a ballot via email. If you have not, please send the election officials an email [6], or hop onto the #openstack-election channel on OFTC and let them know. During the past week, the TC merged an update to the tested runtimes for the 2025.1 release, which is due on 2025-04-04 [3]. === Weekly Meeting === The OpenStack Technical Committee meeting last week was held over IRC, and it was well attended [7]. We began brainstorming our agenda for various upcoming events, including the OpenInfra Summit in Suwon [4], the OpenInfra Days in North America (2024-10-15 to 2024-10-16), and the OpenInfra Virtual Project Teams Gathering (2024-10-21 to 2024-10-25). There will be some current TC members and candidates at these in-person events; Amy Marrich (spotz) will be hosting a meet-and-greet. Please stop by and say hello. The team also discussed the health of the OpenStack Zuul Gate jobs. Ghanshyam Mann (gmann) brought up the requirements bump to oslo.policy 4.4.0. Most projects have addressed incompatibilities; however, there is still some ongoing work [8]. One of the two donated ARM64 clouds was lost recently, affecting Gate jobs running on ARM64. The remaining cloud (osuosl) is slower than expected because of this consolidation. Projects like Kolla had to stop running image build jobs for ARM64. There was discussion on the importance and relevance of ARM support, with some members noting continued interest in ARM usage, especially in AI/ML. Jeremy Stanley (fungi) initiated a discussion [9] on improving communication patterns around code review and contributions, highlighting the need for more engagement from TC members. We also discussed that project teams, including the Technical Committee, must vet the questions in the User Survey and weed out questions or answer choices that are no longer relevant. The next meeting of the OpenStack Technical Committee is on 2024-09-03. This meeting will be held simultaneously over video and IRC. Please find the agenda for the meeting and the details to join within the TC Meeting Wiki [10]. I'm looking forward to your participation. === Governance Proposals === ==== Merged ==== Define testing runtime for 2025.1 release | https://review.opendev.org/c/openstack/governance/+/926150 ==== Open for review ==== charms: retire unmaintained charms | https://review.opendev.org/c/openstack/governance/+/926793 Retire all single charm repositories | https://review.opendev.org/c/openstack/governance/+/903490 Retire kuryr-kubernetes and kuryr-tempest-plugin | https://review.opendev.org/c/openstack/governance/+/922507 === Upcoming Events === 2024-09-03: OpenInfra Summit Asia, Suwon 2024-09-06: OpenInfra Days, Tokyo 2024-09-10: OpenInfra Board Meeting: https://board.openinfra.dev/ 2024-09-18: OpenStack 2025.1 Election Polling Ends 2024-10-15: OpenInfra Days NA, Indianapolis: https://ittraining.iu.edu/explore-topics/titles/oid-iu/ Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] Release Schedule: https://releases.openstack.org/dalmatian/schedule.html [2] Cycle-with-RC release model: https://releases.openstack.org/reference/release_models.html#cycle-with-rc [3] 2025.1 Release Schedule: https://releases.openstack.org/epoxy/schedule.html [4] OpenInfra Summit 2024, Suwon: https://2024.openinfraasia.org/a/schedule [5] 2025.1 OpenStack Technical Committee Election: https://governance.openstack.org/election/#tc-candidates [6] 2025.1 OpenStack Election Officials: https://governance.openstack.org/election/#election-officials [7] TC Meeting IRC Log, 2024-08-27: https://meetings.opendev.org/meetings/tc/2024/tc.2024-08-27-18.00.log.html [8] oslo.policy 4.4.0 enables the RBAC new defaults by default https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [9] Bridging the gap between the community and contributing orgs: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [10] TC Meeting Agenda, 2024-09-03: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Agenda

1 0

Can not work on Dashboard(throwing 504-Timed out) on Openestack-on-ansible environment
by sudeb ghosh 02 Sep '24

02 Sep '24

Hi Team , We have an 'Openstack on Ansible' Private cloud environment with 3 controllers & 35 Compute host nodes. Age of this environment is around 4 yrs. DISTRIB_RELEASE: 21.2.1 DISTRIB_CODENAME: Ussuri DISTRIB_DESCRIPTION: Openstack-Ansible Top of this Linux containers are there where the services are running inside such as neutron, galera, horizon, keystone, etcd, nova-api, utility etc for all 3 controllers. Current Set Up: We find in haproxy config file only controller2 & controller3 are configured. So the galera container along with Mariadb Services are running fine within controller2(Primary) & controller3. But Galera container in controller1 is stopped & mariadb service not running here. Issue Started with: All of suddent we faced, user not able to log in dashboard 5 days back. The dashboard page is not getting loaded. Action Taken: We find Maria-DB in controller2 is non-primary & DB on controller-3 is down. Made Maria-DB in controller2 primary & DB on controller-3 up which resulted to load the page & log in. Current Issue: 1) But the Dashboard page is too slow to work. Sometime throwing 504-Timed out error 2) VM Console(VNC) not working for any VM 3) NOt able to create any VM(Showing 'Scheduling' continuously) 4) 25/35 Hypervisons are showing down in Dashboard within 'Hypervisor list' tab but those are up as per CLI 5) Not able to create any volume Request you to go through the Problem description & Can anybody help me out providing the solution for the same? Pl do intimate me if any more info is required. Regards, Sudeb Ghosh sudeb_ece(a)yahoo.co.in BR// Sudeb Ghosh 7044064878 9332034788

2 3

[neutron] Bug deputy report for week of August 25th
by Miguel Lavalle 01 Sep '24

01 Sep '24

I was the Neutron bugs deputy last week. Following is a brief summary of the reported bugs: Critical bugs ----------------- None High bugs -------------- https://bugs.launchpad.net/neutron/+bug/2078382 [OVN] User defined router flavor with no LSP associated to router interfaces. Assignee Miguel Lavalle. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/917800/ https://bugs.launchpad.net/neutron/+bug/2077942 [OVN][FT] ``test_agent_delete`` is randomly failing. Assignee Rodolfo Alonso. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/927236 https://bugs.launchpad.net/neutron/+bug/2078518 neutron designate scenario job failing with new RBAC. Needs owner Under investigation ------------------------- https://bugs.launchpad.net/neutron/+bug/2077879 [OVN] Router gateway does not have a chassis defined Edit. Reporter is testing solution given by ihrachys based on https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/926940. Waiting from reporter to indicate whether the solution is satisfactory or not.

1 0

[nova][ci] openstack-tox-docs job failing, hold your rechecks
by melanie witt 01 Sep '24

01 Sep '24

Hey all, Just a heads up... The last several openstack-tox-docs job runs on the master branch have failed, both gate and check queues: https://zuul.opendev.org/t/openstack/builds?job_name=openstack-tox-docs&pro… because of AFAICT the following warning which is treated as an error: 2024-08-28 19:24:04.604443 | ubuntu-jammy | /home/zuul/src/opendev.org/openstack/nova/.tox/docs/lib/python3.10/site-packages/openstackdocstheme/page_context.py:70: RemovedInSphinx90Warning: Sphinx 9 will drop support for representing paths as strings. Use "pathlib.Path" or "os.fspath" instead. 2024-08-28 19:24:04.649996 | ubuntu-jammy | src_file = full_src_file[len(str(app.builder.env.srcdir)):].lstrip('/') The error looks like it's coming from Sphinx and I did find an upper-constraints bump of Sphinx happened earlier today: https://review.opendev.org/c/openstack/requirements/+/927358 The version was bumped from 7.4.6 to 8.0.2 -- which may possibly be related to the failures. I will look more into the failure but if anyone else knows how to fix this, please feel free to propose something. -melwitt

2 3

[manila] Feature freeze exceptions
by Carlos Silva 30 Aug '24

30 Aug '24

Hello, Zorillas! This is feature freeze week and we managed to get a lot done, but during this week's IRC meeting we talked about the features we are working on, and would likely need a couple more days to be merged. They are: [1] NetApp SnapLock support This feature will allow users to create WORM shares and OpenStack operators to configure retention policy for the manila shares while using the NetApp ONTAP driver. This is a feature targeting a single driver. [2] [NetApp] Custom storage efficiency policy support for Manila share This feature is also constrained to the NetApp ONTAP driver. It will allow configuring custom efficiency policies using Manila. The change is not very numerous in terms of lines of code and has received some reviews. The author needs some more time to hash out some implementation details. [3] Export Location Metadata This is a feature that introduces the possibility of adding metadata to export locations. The generic metadata mechanism is being used, and it is thoroughly tested in manila-tempest-plugin. This change has received reviews and the author has been responsive. The author needs some time to figure out some unit test failures. [4] Ensure shares API An API change that will allow OpenStack operators to start the process that recalculates the export location of their shares, which is currently only done during the service restart. Reviews have been provided, but we would also like some more time for some more testing and more reviews. [5] Add json and yaml formatting to manila-manage This is an RFE. Doesn't introduce any APIs, and will allow OpenStack operators to specify the output format in the `manila-manage service list` command, which is useful for automations consuming this feature. I would like to grant a feature freeze exception for the three features mentioned and they should be merged by next Thursday (September 5th). [1] https://review.opendev.org/c/openstack/manila/+/926049 [2] https://review.opendev.org/c/openstack/manila/+/925079 [3] https://review.opendev.org/q/topic:%22bp/metadata-for-share-resources%22+st… [4] https://review.opendev.org/c/openstack/manila/+/924279 [5] https://review.opendev.org/c/openstack/manila/+/899295 Thank you, carloss

1 0

[Ironic] [Kolla Ansible] [Yoga] Redfish Drive
by Ali Hussain 30 Aug '24

30 Aug '24

Hello Everyone, We are having issue deploying Ironic baremental image to compute using Ironic(Yoga). The Baremetal machine is enrolled using Redfish driver. It goes to Available state as well. Once we attempt to deploy it the following exception is thrown. Error: Detected change in error condition on node “BMNode1”. Deploy step deploy.deploy failed: Redfish exception occurred. Error: Setting boot mode to uefi failed for node <UUID>. Error: HTTP PATCH http://IP/redfish/v1/Systems/1 returned code 400. Base.v1_10_3.GeneralError: The service detected a malformed request body that it was unable to interpret. Extended information: [{'MessageId': 'Base.1.10.*UnrecognizedRequestBody*', 'Severity': 'Warning', 'Resolution': 'Correct the request body and resubmit the request if it failed.', 'Message': 'The service detected a malformed request body that it was unable to interpret.', 'MessageArgs': [''], 'RelatedProperties': ['']}]. This is issue with Ironic used with Kolla Ansible but if we try with Bifrost the image booting goes fine. Any help or suggestion would be helpful to us. Best regards, Ali Hussain

2 3

[elections][tc] Question for Candidates: Availability
by Jay Faulkner 30 Aug '24

30 Aug '24

Hello all, First of all, I'm incredibly happy that the community has stepped up and provided us the opportunity to avoid any issues at all around company affiliations this cycle. Thank you to all the new folks who have run for the TC and helped with this. My two questions are simple: How much of your workweek is spent focused on upstream OpenStack? How much of that time will be available to use for TC-related activities? Thanks, Jay Faulkner

5 4

[release] Release countdown for week R-4, Sep 02-06
by Elõd Illés 30 Aug '24

30 Aug '24

Development Focus ----------------- We just passed feature freeze! Until release branches are cut, you should stop accepting featureful changes to deliverables following the cycle-with-rc release model, or to libraries. Exceptions should be discussed on separate threads on the mailing-list, and feature freeze exceptions approved by the team's PTL. Focus should be on finding and fixing release-critical bugs, so that release candidates and final versions of the 2024.2 Dalmatian deliverables can be proposed, well ahead of the final 2024.2 Dalmatian release date. General Information ------------------- We are still finishing up processing a few release requests, but the 2024.2 Dalmatian release requirements are now frozen. If new library releases are needed to fix release-critical bugs in 2024.2 Dalmatian, you must request a Requirements Freeze Exception (RFE) from the requirements team before we can do a new release to avoid having something released in 2024.2 Dalmatian that is not actually usable. This is done by posting to the openstack-discuss mailing list with a subject line similar to: [$PROJECT][requirements] RFE requested for $PROJECT_LIB Include justification/reasoning for why a RFE is needed for this lib. If/when the requirements team OKs the post-freeze update, we can then process a new release. A soft String freeze is now in effect, in order to let the I18N team do the translation work in good conditions. In Horizon and the various dashboard plugins, you should stop accepting changes that modify user-visible strings. Exceptions should be discussed on the mailing-list. By September 26th, 2024 this will become a hard string freeze, with no changes in user-visible strings allowed. Actions ------- stable/2024.2 branches should be created soon for all not-already-branched libraries. You should expect 2-3 changes to be proposed for each: a .gitreview update, a reno update (skipped for projects not using reno), and a tox.ini constraints URL update. Please review those in priority so that the branch can be functional ASAP. The Prelude section of reno release notes is rendered as the top level overview for the release. Any important overall messaging for 2024.2 Dalmatian changes should be added there to make sure the consumers of your release notes see them. Upcoming Deadlines & Dates -------------------------- RC1 deadline: September 13th, 2024 (R-3 week) Final RC deadline: September 26th, 2024 (R-1 week) Final 2024.2 Dalmatian release: October 2nd, 2024 OpenInfra Summit Asia: September 3-4, 2024 Előd Illés irc: elodilles @ #openstack-release

1 0

[all][elections][ptl][tc] Opt in to CIVS voting system
by Sławek Kapłoński 30 Aug '24

30 Aug '24

Hi, The election uses the Condorcet Internet Voting Service (CIVS). Due to CIVS policy, to vote in private CIVS polls, you must opt in to email communication. To opt in, please enter your Gerrit email address in the following page before the start of the election Sep 04, 2024 23:45 UTC, and confirm with the code that will sent to you via email. https://civs1.civs.us/cgi-bin/opt_in.pl If you have any question, please contact the election officials. https://governance.openstack.org/election/#election-officials -- Slawek Kaplonski Principal Software Engineer Red Hat

3 2

kolla openstack and ceph cephadm compatible?
by William Muriithi 29 Aug '24

29 Aug '24

Hello, I am faced with an issue where I can't run kolla-ansible prechecks against one of the system and the only thing unique about this system is that its the one I used to bootstrap ceph, hence have cephadm installed. It makes me assume that the two might be incompatible. Ceph is using a system account called cephuser and kolla is using an system account called kolla, so I am lost why they are stumping on each other. Granted, both have full sudo, so that could be the route where this mess came up. The three systems are both execution machine and storage machine. When I initially set it up, I made assumptions that cinder was part of the core service. I then struggled to bring up the virtual machine and its while debugging this issue that I realized I need cinder. Fine, I enabled cinder on /etc/kolla/globals.d/cinder.yml as documented, but now I can't run prechecks. This was failing only on cinder. When I run deploy, now its failing on glance too. Have anyone noticed something similar? Can it be fixed without reinstalling all the systems? This is the error I am seeing:- The full traceback is: Traceback (most recent call last): File "/var/lib/kolla/.ansible/tmp/ansible-tmp-1724965733.8092885-882811-135535224643810/AnsiballZ_kolla_container_facts.py", line 107, in <module> _ansiballz_main() File "/var/lib/kolla/.ansible/tmp/ansible-tmp-1724965733.8092885-882811-135535224643810/AnsiballZ_kolla_container_facts.py", line 99, in _ansiballz_main invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS) File "/var/lib/kolla/.ansible/tmp/ansible-tmp-1724965733.8092885-882811-135535224643810/AnsiballZ_kolla_container_facts.py", line 47, in invoke_module runpy.run_module(mod_name='ansible.modules.kolla_container_facts', init_globals=dict(_module_fqn='ansible.modules.kolla_container_facts', _modlib_path=modlib_path), File "/usr/lib64/python3.9/runpy.py", line 225, in run_module return _run_module_code(code, init_globals, run_name, mod_spec) File "/usr/lib64/python3.9/runpy.py", line 97, in _run_module_code _run_code(code, mod_globals, init_globals, File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/tmp/ansible_kolla_container_facts_payload_msvskycx/ansible_kolla_container_facts_payload.zip/ansible/modules/kolla_container_facts.py", line 132, in <module> File "/tmp/ansible_kolla_container_facts_payload_msvskycx/ansible_kolla_container_facts_payload.zip/ansible/modules/kolla_container_facts.py", line 124, in main File "/tmp/ansible_kolla_container_facts_payload_msvskycx/ansible_kolla_container_facts_payload.zip/ansible/modules/kolla_container_facts.py", line 105, in use_podman File "/usr/local/lib/python3.9/site-packages/podman/domain/manager.py", line 68, in reload latest = self.manager.get(self.id) File "/usr/local/lib/python3.9/site-packages/podman/domain/containers_manager.py", line 44, in get response.raise_for_status() File "/usr/local/lib/python3.9/site-packages/podman/api/client.py", line 75, in raise_for_status raise APIError(cause, response=self._response, explanation=message) podman.errors.exceptions.APIError: 500 Server Error: Internal Server Error (getting container from store "1127b70a453e15c7b5e2ba8da307f8b17791e6a23fb2b2e94a4d5dd5fce59907": container not known) fatal: [ostack-49.openstack.example.com] FAILED! => { "changed": false, "module_stderr": "OpenSSH_8.7p1, OpenSSL 3.0.7 1 Nov 2022\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host ostack-49.openstack.example.com originally ostack-49.openstack.example.com\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: not matched 'final'\r\ndebug2: match not found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1 (parse only)\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug1: configuration requests final Match pass\r\ndebug1: re-parsing configuration\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug3: /etc/ssh/ssh_config line 55: Including file /etc/ssh/ssh_config.d/50-redhat.conf depth 0\r\ndebug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf\r\ndebug2: checking match for 'final all' host ostack-49.openstack.example.com originally ostack-49.openstack.example.com\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 3: matched 'final'\r\ndebug2: match found\r\ndebug3: /etc/ssh/ssh_config.d/50-redhat.conf line 5: Including file /etc/crypto-policies/back-ends/openssh.config depth 1\r\ndebug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config\r\ndebug3: gss kex names ok: [gss-curve25519-sha256-,gss-nistp256-sha256-,gss-group14-sha256-,gss-group16-sha512-]\r\ndebug3: kex names ok: [curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512]\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/var/lib/kolla/.ssh/known_hosts'\r\ndebug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/var/lib/kolla/.ssh/known_hosts2'\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 881450\r\ndebug3: mux_client_request_session: session request sent\r\ndebug1: mux_client_request_session: master session id: 2\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 1\r\nShared connection to ostack-49.openstack.example.com closed.\r\n", "module_stdout": "Traceback (most recent call last):\r\n File \"/var/lib/kolla/.ansible/tmp/ansible-tmp-1724965733.8092885-882811-135535224643810/AnsiballZ_kolla_container_facts.py\", line 107, in <module>\r\n _ansiballz_main()\r\n File \"/var/lib/kolla/.ansible/tmp/ansible-tmp-1724965733.8092885-882811-135535224643810/AnsiballZ_kolla_container_facts.py\", line 99, in _ansiballz_main\r\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n File \"/var/lib/kolla/.ansible/tmp/ansible-tmp-1724965733.8092885-882811-135535224643810/AnsiballZ_kolla_container_facts.py\", line 47, in invoke_module\r\n runpy.run_module(mod_name='ansible.modules.kolla_container_facts', init_globals=dict(_module_fqn='ansible.modules.kolla_container_facts', _modlib_path=modlib_path),\r\n File \"/usr/lib64/python3.9/runpy.py\", line 225, in run_module\r\n return _run_module_code(code, init_globals, run_name, mod_spec)\r\n File \"/usr/lib64/python3.9/runpy.py\", line 97, in _run_module_code\r\n _run_code(code, mod_globals, init_globals,\r\n File \"/usr/lib64/python3.9/runpy.py\", line 87, in _run_code\r\n exec(code, run_globals)\r\n File \"/tmp/ansible_kolla_container_facts_payload_msvskycx/ansible_kolla_container_facts_payload.zip/ansible/modules/kolla_container_facts.py\", line 132, in <module>\r\n File \"/tmp/ansible_kolla_container_facts_payload_msvskycx/ansible_kolla_container_facts_payload.zip/ansible/modules/kolla_container_facts.py\", line 124, in main\r\n File \"/tmp/ansible_kolla_container_facts_payload_msvskycx/ansible_kolla_container_facts_payload.zip/ansible/modules/kolla_container_facts.py\", line 105, in use_podman\r\n File \"/usr/local/lib/python3.9/site-packages/podman/domain/manager.py\", line 68, in reload\r\n latest = self.manager.get(self.id)\r\n File \"/usr/local/lib/python3.9/site-packages/podman/domain/containers_manager.py\", line 44, in get\r\n response.raise_for_status()\r\n File \"/usr/local/lib/python3.9/site-packages/podman/api/client.py\", line 75, in raise_for_status\r\n raise APIError(cause, response=self._response, explanation=message)\r\npodman.errors.exceptions.APIError: 500 Server Error: Internal Server Error (getting container from store \"1127b70a453e15c7b5e2ba8da307f8b17791e6a23fb2b2e94a4d5dd5fce59907\": container not known)\r\n", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1 } Regards, William

1 0

PTG October 2024 Team Signup
by Kendall Nelson 29 Aug '24

29 Aug '24

Hello Everyone, Hopefully you have seen by now, the next OpenInfra PTG[1] which will take place October 21-25, 2024 virtually! Registration is now open[2]! To sign up your team, you must complete the survey[3] by August 30th at 7:00 UTC. We NEED accurate contact information (email and IRC nickname ideally) for the moderator of your team’s sessions. This is because the survey information will be used to organize the schedule signups which will be done via the PTGBot. If you are not on IRC, please get setup[4] on the OFTC network and join #openinfra-events. You are also encouraged to familiarize yourself with the PTGBot documentation[5] as well. If you have any questions, please reach out to ptg(a)openinfra.dev! Information about signing up for time slots will be sent to moderators after the team signup deadline. Continue to visit openinfra.dev/ptg for updates. -Kendall (diablo_rojo) [1] PTG Site: https://openinfra.dev/ptg/ [2] PTG Registration: http://ptg.openinfra.dev/ [3] Team Survey: https://openinfrafoundation.formstack.com/forms/oct2024_ptg_survey [4] Setup IRC: https://docs.openstack.org/contributors/common/irc.html [5] PTGBot README: https://opendev.org/openstack/ptgbot/src/branch/master/README.rst

1 2

[all][elections][tc] TC Election Campaigning Kickoff
by Ian Y. Choi 29 Aug '24

29 Aug '24

Developers, The TC Election Campaigning Period has now started[1]. During the next couple days, you are all encouraged to ask the candidates questions about their platforms[2], opinions on OpenStack, community governance, and anything else that will help you to better determine how you will vote. This is the time to raise any issues you wish the future TC to consider, and to evaluate the opinions of the nominees prior to their election. Candidates, Each of you has posted a platform[2], and announced your nomination to the developers. From this point, you are encouraged to ask each other questions about the posted platforms, and begin discussion of any points that you feel are particularly important during the next cycle. While you are not yet TC members, your voices and opinions about the issues raised in your platforms and questions raised by the wider community will help ensure that the future TC has the widest possible input on the matters of community concern, and the electorate has the best information to determine the ideal TC composition to address these and other issues that may arise. [1] https://governance.openstack.org/election/ [2] https://opendev.org/openstack/election/src/branch/master/candidates/2025.1/…

1 0

[publiccloud-sig] Cancelling next weeks meeting - Sept 4th
by Felix Kronlage-Dammers 29 Aug '24

29 Aug '24

Hi, a lot of the frequent participants of the publiccloud-sig are going to be travelling next wednesday. As such, Tobias and I decided that we will skip the call next week. Next regular call will then be on September 18th. felix -- Felix Kronlage-Dammers Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[swift] Sync rings automatically
by Mark Kirkwood 28 Aug '24

28 Aug '24

The familiar process for ring modification is: 1/ Create or amend corresponding builder file 2/ Rebalance it 3/ (R)sync the newly modified ring to all hosts in the cluster Recently I was wondering about removing the need for 3/. A very simple minded approach would be to make a cron job on each host to periodically rsync from one of the proxies (designated as the 'master' where we modify rings).. However, I then decided that it would be nicer to make a new Swift daemon and add it to the code. I have coded this up (I'm looking at adding a patch to Swift's Gerrit but thought I'd mention it here first). the basic design is: - New daemon swift-ring-syncer - Backend code in common/ring_syncer.py - Conf file ring-syncer.conf One host is designated the authoritative source (config 'ringbuilder'). All hosts can run this daemon (including the ringbuilder), Rings will be rsync'd at a configurable interval (e.g 300s).. This is safe to do even when the rings are being modified as the current code does a temp file write plus atomic rename (If it is desired to sync the builder files too then their 'save' function would need to use this sane logic instead of writing to them directly). Does this sound like something that would be good in the main Swift code? Let me know and I'll setup Gerrit if so! Regards Mark

1 0

[all][elections][ptl][tc] Combined PTL/TC 2025.1 cycle Election Nominations End
by Ian Y. Choi 28 Aug '24

28 Aug '24

The PTL and TC Nomination period is now over. The official candidate lists for PTLs [0] and TC seats [1] are available on the election website. -- PTL Election Details -- There are 4 projects without candidates, so according to this resolution[2], the TC will have to decide how the following projects will proceed: Kuryr, Mistral, Swift, Watcher There is 1 project that will have elections: Cinder. -- TC Election Details -- The official candidate list is available on the election website[1]. Now begins the campaigning period where candidates and the electorate may debate their statements. Polling will start Sep 04, 2024 23:45 UTC. Thank you, [0] https://governance.openstack.org/election/#ptl-candidates [1] https://governance.openstack.org/election/#tc-candidates [2] https://governance.openstack.org/resolutions/20141128-elections-process-for…

1 0