July 2024 - openstack-discuss - lists.openstack.org

[openstack][kolla-ansible]about group node config overwrite
by Nguyễn Hữu Khôi 29 Oct '25

29 Oct '25

Hello Koller. I want to ask if we can overwrite config for a group of nodes. For example, compute01 >> compute10 will have the same nova configuration compute11 >> compute20 will have the same nova configuration. It will be very nice if we support this way. Thank you. Regards Nguyen Huu Khoi

4 7

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

How to config DPDK compute in OVN
by dangerzone ar 22 Apr '25

22 Apr '25

Hi....as far I understand ovs does support dpdk compute to be deployed. I'm referring to the guideline ' *https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html <https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html>* ' ... It's too brief.. May I know how to config dpdk compute in OVN networking openstack? Hope someone could help what is the config to be done.... Currently compute is run without dpdk over openstack train ovn based. Please need some info on how I can proceed. Thank you

2 1

[oslo] python-consul2 is unmaintained
by Takashi Kajinami 21 Apr '25

21 Apr '25

Hello, I recently noticed that python-consul2 library[1], which is used by the consul coordination driver in tooz, hasn't been updated for 3+ years. Looking at issues in the project, there is an indication that the library is not compatible with ACL API in consul >= 1.11[2]. There is another fork created[3] and this includes fixes to support new API endpoints. Although the driver in tooz does not directly use the feature, I think it's better to discuss our plan to mitigate risks caused by the unmaintained library. As far as I can think of, there are three options now 1. Deprecate consul driver and remove it 2. Keep using python-consul2 now. At least basic functionality is proven to work by our functional tests (Note that the tests do not cover all configuration patterns) 3. Replace python-consul2 by py-consul Before we start the overall discussions I'd like to ask a few questions to the wider audience, as our inputs. - Is anyone using the consul coordination driver now ? - Does anyone have experience with python-consul2 and/or py-consul ? Note that this discussion affects the http messaging driver[4] currently proposed, because it also requires client library to communicate with Consul. Actually I noticed the situation when I noticed that the requirement check job is failing because the library is not part of global requirements list... Thank you, Takashi Kajinami [1] https://github.com/poppyred/python-consul2 [2] https://github.com/poppyred/python-consul2/issues/42 [3] https://github.com/criteo-forks/py-consul [4] https://review.opendev.org/c/openstack/oslo.messaging/+/912499 -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

2 3

[octavia] Purging ERROR'ed amphora and load balancers stuck in transitional states
by Paul Browne 11 Apr '25

11 Apr '25

Hello Octavia users, For a while I've been trying to find a systematic way to remove Octavia LBs+Amphora that have become stuck in ERROR or transitional PENDING_* states ; $ openstack loadbalancer amphora list -f value | grep ERROR a2323f14-3aaa-418a-8249-111aaa9c21fe 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR MASTER 10.8.0.242 192.168.3.247 e5b236ba-e7ee-4ed7-9f58-57ce7a408489 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR BACKUP 10.8.0.190 192.168.3.247 6b556f28-93c9-49dd-b6ee-4379288e7957 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR MASTER 10.8.0.39 192.168.3.126 c669db5d-8686-4d5c-9e95-e02030b34301 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR BACKUP 10.8.0.174 192.168.3.126 $ openstack loadbalancer list -f value | grep -vi active 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe k8s-clusterapi-cluster-default-ci-6386871107-kube-upgrade-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.247 PENDING_UPDATE amphora d5e402fe-2c4b-49af-a700-532cb408cee5 k8s-clusterapi-cluster-default-ci-6386871107-latest-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.126 PENDING_DELETE amphora These resources are marked immutable and so cannot be failed over or deleted; $ openstack loadbalancer amphora failover a2323f14-3aaa-418a-8249-111aaa9c21fe Load Balancer 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe is immutable and cannot be updated. (HTTP 409) (Request-ID: req-6e66c4e8-c3d3-4549-a03c-367017c8c8b3) $ openstack loadbalancer failover 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe Invalid state PENDING_UPDATE of loadbalancer resource 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe (HTTP 409) (Request-ID: req-84b44212-e7a8-4101-a16f-18c774c0577e) The backing Nova instances for these Amphora do seem to exist and be in good working order. Is there any API way to purge these out of Octavia's service state, or would (very careful) DB hackery be required here? Many thanks, Paul Browne ******************* Paul Browne Research Computing Platforms University Information Services Roger Needham Building JJ Thompson Avenue University of Cambridge Cambridge United Kingdom E-Mail: pfb29(a)cam.ac.uk<mailto:pfb29@cam.ac.uk> Tel: 0044-1223-746548 *******************

4 6

[tc][all][monasca ] Retiring the Monasca project
by Ghanshyam Mann 16 Feb '25

16 Feb '25

Hello everyone, As you might know, the Monasca project is not active and was marked 'inactive' since 2024.1 cycle[1]. In this cycle also, this project is leaderless[2] and no maintainer stepped up to maintain it. As discussed in the Leaderless project etherpad[2], the next step is to retire the project. I'm sending this email to propose the Monasca retirement and based on the response, I will proceed next. [1] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [2] https://etherpad.opendev.org/p/2024.2-leaderless#L72 -gmann

6 17

[ironic]About the OS supported by Diskimage-builder
by Junya Noguchi (Fujitsu) 03 Feb '25

03 Feb '25

To whom it may concern. My name is Junya Noguchi. I have some questions about the supported OS for Diskimage-builder operation. Please respond. References https://docs.openstack.org/ironic-python-agent-builder/latest/admin/dib.html Does it mean that the OS that says "work but receive only limited testing and support" works but is not supported? References https://docs.openstack.org/diskimage-builder/latest/ This document describes RHEL and other operating systems that are not supported in the previous question. Are these operating systems supported? Also, are there any plans to support more operating systems in the future? Best regards, Junya Noguchi

5 7

[glance][ops] proposal to remove the glance-scrubber
by Brian Rosmaita 19 Jan '25

19 Jan '25

Hello Glance users, The Glance project team wants to determine whether anyone is using the glance-scrubber utility [0]. If not, we are thinking of deprecating it this cycle to be removed in the appropriate future cycle. If you have not previously heard of the glance-scrubber, that's most likely because it's only useful for particular deployment configurations of which yours is not one. See the doc [0] for details if you are curious. Our suspicion is that the glance-scrubber may have outlived its usefulness for modern OpenStack configurations. (The last time I remember it being discussed at a PTG was during the snowstorm in Dublin, which I believe was the Rocky development cycle). On the other hand, the team also has some ideas for improving the scrubber, but there's no point doing that if it's not actually being used in the first place. So if removal of the glance-scrubber would negatively affect you, please respond to this email so we can get some data points. Thanks! [0] https://docs.openstack.org/glance/latest/cli/glancescrubber.html

2 2

[cinder][kubernetes] pvc stuck in reserved state
by Eugen Block 10 Jan '25

10 Jan '25

Hi, we have a couple of k8s clusters in our openstack environment (Victoria) and are facing some problems when pods get moved to different worker nodes. The PVCs get stuck in "reserved" state, and they only get attached after resetting the volumes to "available" state. First I thought it was due to the missing multiattach volume type which seems to work fine in our customer's cluster. I also found the "duplicate connector" error messages in the nova-compute.log: > Unable to update attachment.(Invalid volume: duplicate connectors > detected on volume But after adding the multiattach volume type and retyping the affected volumes, the PVCs still get stuck in "reserved" state while I don't see any error messages on the hypervisors anymore. So it seems that something has changed/improved, but now I don't have a trace where to look next. Could this be a cinder-csi driver issue or something? Our cinder backend is ceph (Pacific). Not sure which information could help so please ask for anything I can provide. Any pointers are highly appreciated! Thanks! Eugen

1 2

ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version
by Swogat Pradhan 23 Dec '24

23 Dec '24

Hi, I am currently trying to deploy openstack wallaby using tripleo arch. I created the network jinja templates, ran the following commands also: #openstack overcloud network provision --stack overcloud --output networks-deployed-environment.yaml custom_network_data.yaml # openstack overcloud network vip provision --stack overcloud --output vip-deployed-environment.yaml custom_vip_data.yaml # openstack overcloud node provision --stack overcloud --overcloud-ssh-key /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml and used the environment files in the openstack overcloud deploy command: (undercloud) [stack@hkg2director ~]$ cat deploy.sh #!/bin/bash THT=/usr/share/openstack-tripleo-heat-templates/ CNF=/home/stack/ openstack overcloud deploy --templates $THT \ -r $CNF/templates/roles_data.yaml \ -n $CNF/workplace/custom_network_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e $CNF/templates/node-info.yaml \ -e $CNF/templates/scheduler-hints.yaml \ -e $CNF/workplace/networks-deployed-environment.yaml \ -e $CNF/workplace/vip-deployed-environment.yaml \ -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ -e $CNF/workplace/custom-net-bond-with-vlans.yaml Now when i run the ./deploy.sh script i encounter an error stating: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command.: tripleoclient.exceptions.InvalidConfiguration: Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command. 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END return value: 1 Can someone tell me where the mistake is? With regards, Swogat Pradhan

4 4

[kolla] [openeuler] CI jobs status
by Michał Nasiadka 26 Nov '24

26 Nov '24

Hello Koalas, OpenEuler jobs in master (to be Antelope) are failing for a long time. If no one is interested in maintaining OpenEuler support in Kolla-Ansible - I will propose dropping it’s support on Wednesday Kolla meeting. Best regards, Michal

4 3

[horizon][ceph] object storage container storage policy missing
by Satish Patel 24 Nov '24

24 Nov '24

Folks, Openstack Release: Zed Recently I have integrated ceph RGW service with openstack/keystone and testing with command line tool "openstack container create foo1" and it works but when I go to Horizon UI I am seeing it doesn't have storage policy dropdown list and because of that I am not able to create container or upload file from Horizon GUI https://ibb.co/KhYW6Rd It works with commands like but not horizon. $ openstack container create foo1 +---------------------------------------+-----------+------------------------------------------------+ | account | container | x-trans-id | +---------------------------------------+-----------+------------------------------------------------+ | AUTH_d4d82849885e4126aef402c408ad2c5c | foo1 | tx00000c057454841b13497-0065b558b4-42ff2eb-ahm | +---------------------------------------+-----------+------------------------------------------------+ $ openstack container list +------+ | Name | +------+ | foo1 | +------+

2 2

[ironic] Having issue with deploying ISO image to node
by James Leong 17 Nov '24

17 Nov '24

Hi all, I am trying to play around with ironic, and I am having an issue with deploying an Ubuntu 20.04 ISO image onto my bare metal node. Since I am quite new to bare metal, I am not entirely sure what went wrong as I couldn't find any error messages shown on my log file (ironic-conductor, ironic-api, ironic_neutron_agent, and nova-compute-ironic). This might be my guess; the ironic inspection is hitting timeout, which is causing the deployment to timeout. However, I have no idea why the inspection is failing at timeout. Any help would be appreciated. Best, James

6 7

[largescale-sig]scaling story
by Alex Song (宋文平) 25 Oct '24

25 Oct '24

Hi everyone, we want to share our scaling story. Please refer the link: https://etherpad.opendev.org/p/large-scale-inspur Thanks, Alex Song

6 8

[nova][libvirtd] internal error: unable to execute QEMU command 'query-named-block-nodes'
by Satish Patel 22 Oct '24

22 Oct '24

Folks, I have a ceph cluster and recently I configured rbd-mirror to replicate all data to remove ceph cluster for disaster recovery. Yesterday for POC we did a hard cutover on ceph and point openstack to the new cluster. All other vms came back up fine but 2 vms stuck in this error state in libvirt logs 2024-01-31 22:44:37.591+0000: 474597: error : qemuMonitorJSONCheckErrorFull:399 : internal error: unable to execute QEMU command 'query-named-block-nodes': cannot read image start for probe: Permission denied If it's a wider issue than it should impact all the vms but why only two VM stuck and not starting and libvirt giving me this error in logs nova-compute logs also showing same error 2024-01-31 22:44:40.925 7 INFO nova.compute.manager [None req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00 ca5c652478c7429e964257990800e9cb - - default default] [instance: 2de0f880-77c7-4d2c-9e01-898c57ad3693] Successfully reverted task state from powering-on on failure for instance. 2024-01-31 22:44:40.944 7 ERROR oslo_messaging.rpc.server [None req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00 ca5c652478c7429e964257990800e9cb - - default default] Exception during message handling: libvirt.libvirtError: internal error: unable to execute QEMU command 'query-named-block-nodes': cannot read image start for probe: Permission denied

3 8

[kuryr][tc] Final call for Kuryr PTL/DPL liaisons
by Ghanshyam Mann 15 Oct '24

15 Oct '24

Hello Everyone, Kuryr project ends up being a leaderless project for the 2024.2 Dalmatian cycle[1]. There is no volunteer to serve as PTL or DPL liaisons yet. As the next step, I have proposed to mark this project Inactive: - https://review.opendev.org/c/openstack/governance/+/916945 If you would like to serve as a leader and help maintain this project, this is the right time to step up by replying to this email or commenting in gerrit change 916945. [1] https://etherpad.opendev.org/p/2024.2-leaderless#L99 Ref: https://governance.openstack.org/tc/reference/emerging-technology-and-inact… -gmann

4 10

Kolla Ansible on Ubuntu 20.04 - cloud-init & other network issues
by Tobias McNulty 10 Oct '24

10 Oct '24

Hi, I'm attempting to use Kolla Ansible 14.6.0 to deploy OpenStack Yoga on a small 3-node Ubuntu 20.04 cluster. The nodes have 128 GB RAM each, dual Xeon processors, and dual 10G Intel NICs. The NICs are connected to access ports on a 10G switch with separate VLANs for the local and external networks. All the playbooks run cleanly, but cloud-init is failing in the Ubuntu 20.04 and 22.04 VMs I attempt to boot. The VM images are unmodified from https://cloud-images.ubuntu.com/, and cloud-init works fine if I mount a second volume with user-data. The error is a timeout attempting to reach 169.254.169.254. This occurs both when booting a VM in an internal routed network and directly in an external network. I tried various neutron plugin agents (ovn, linuxbridge, and openvswitch both with and without firewall_driver = openvswitch <https://docs.openstack.org/kolla-ansible/latest/reference/networking/neutro…>) first with a clean install of the entire OS each time, all with the same result. Running tcpdump looking for 169.254.169.254 shows nothing. As a possible clue, the virtual NICs are unable to pass any traffic (e.g., to reach an external DHCP server) unless I completely disable port security on the interface (even if the associated security group is wide open). But disabling port security does not fix cloud-init (not to mention I don't really want to disable port security). Are there any additional requirements related to deploying OpenStack with Kolla on Ubuntu 20.04? This is a fairly vanilla configuration using the multinode inventory as a starting point. I tried to follow the Quick Start <https://docs.openstack.org/kolla-ansible/yoga/user/quickstart.html> as closely as possible; the only material difference I see is that I'm using the same 3 nodes for control + compute. I am using MAAS so it's easy to get a clean OS install on all three nodes ahead of each attempt. I plan to try again with the standard (non-HWE) kernel just in case, but otherwise I am running out of ideas. In case of any additional clues, here are my globals.yml and inventory file, along with the playbook I'm using to configure the network, images, VMs, etc., after bootstrapping the cluster: https://gist.github.com/tobiasmcnulty/7dbbdbc67abc08cbb013bf5983852ed6 Thank you in advance for any advice! Cheers, Tobias

5 9

[all][tc][ptls] Eventlet: Python 3.13 - Attach your seatbelt
by Herve Beraud 06 Sep '24

06 Sep '24

Hi all, As Python 3.13 is close to become a reality, new concerning incompatibilities between this new CPython version and Eventlet also become a reality: - https://github.com/eventlet/eventlet/issues/964 Fedora Python maintainers proposed a patch, but for now the problem remains unsolved. Also, many people reported bugs related to incompatibility between python3.12/dnspython/eventlet/pip-system-cert. There is an example who summarize well the problem https://github.com/eventlet/eventlet/issues/957 Do you have some visibility concerning the adoption of Python 3.13 into Openstack? Turbulences ahead... -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

4 5

[Kolla] [LetsEncrypt] Internal/External/Backend Certificates
by jjjamesg＠proton.me 05 Sep '24

05 Sep '24

Hi all, another one here that can't seem to get kolla running with letsencrypt certs, just curious as to what settings are required for kolla to pull internal/external/backend certs from letsencrypt? I've looked at the docs and have tried a few settings but keep getting errors, sure i had this working last week before i tore it down ready to deploy for production and now i just cant get it to complete the deployment because of certs, only thing i can think of is i need to run kolla-ansible -i multinode certificates but i thought that was just for self made certs? Also, what are best practices for backend tls certs, self signed or using the internal cert etc?? Is it possible to use letsencrypt at all for backend? Cheers all

3 6

[ops][kolla][kolla-ansible] Lets Encrypt Deployment on 2024.1/v18
by Forrest Fuqua 03 Sep '24

03 Sep '24

The documentation seems a little incomplete when it comes to deploying a stack with let's encrypt The attached config is after trying a few settings, but the lets encrypt container errors out every time during deployment causing haproxy to not get a proper ssl certificate the services.d entries are showing its ignoring kolla_internal_fqdn_cert and still using kolla_internal.pem in most services causing the following deployment error TASK [service-ks-register : keystone | Creating services] **************************************************************************************************************************************************** FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (5 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (4 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (3 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (2 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (1 retries left). failed: [100.70.0.1] (item=keystone (identity)) => {"action": "os_keystone_service", "ansible_loop_var": "item", "attempts": 5, "changed": false, "item": {"description": "Openstack Identity Service", "endpoints": [{"interface": "internal", "url": "https://openstack.cyberrange.rit.edu:5000"}, {"interface": "public", "url": "https://openstack.cyberrange.rit.edu:5000"}], "name": "keystone", "type": "identity"}, "module_stderr": "Failed to discover available identity versions when contacting https://openstack.cyberrange.rit.edu:5000. Attempting to parse version from URL.\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 715, in urlopen\n httplib_response = self._make_request(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 404, in _make_request\n self._validate_conn(conn)\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 1058, in _validate_conn\n conn.connect()\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connection.py\", line 419, in connect\n self.sock = ssl_wrap_socket(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/util/ssl_.py\", line 449, in ssl_wrap_socket\n ssl_sock = _ssl_wrap_socket_impl(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/util/ssl_.py\", line 493, in _ssl_wrap_socket_impl\n return ssl_context.wrap_socket(sock, server_hostname=server_hostname)\n File \"/usr/lib/python3.10/ssl.py\", line 513, in wrap_socket\n return self.sslsocket_class._create(\n File \"/usr/lib/python3.10/ssl.py\", line 1100, in _create\n self.do_handshake()\n File \"/usr/lib/python3.10/ssl.py\", line 1371, in do_handshake\n self._sslobj.do_handshake()\nssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n resp = conn.urlopen(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 799, in urlopen\n retries = retries.increment(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/util/retry.py\", line 592, in increment\n raise MaxRetryError(_pool, url, error or ResponseError(cause))\nurllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1021, in _send_request\n resp = self.session.request(method, url, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n resp = self.send(prep, **send_kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n r = adapter.send(request, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/requests/adapters.py\", line 517, in send\n raise SSLError(e, request=request)\nrequests.exceptions.SSLError: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/generic/base.py\", line 133, in _do_create_plugin\n disc = self.get_discovery(session,\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 605, in get_discovery\n return discover.get_discovery(session=session, url=url,\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/discover.py\", line 1459, in get_discovery\n disc = Discover(session, url, authenticated=authenticated)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/discover.py\", line 539, in __init__\n self._data = get_version_data(session, url,\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/discover.py\", line 106, in get_version_data\n resp = session.get(url, headers=headers, authenticated=authenticated)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1154, in get\n return self.request(url, 'GET', **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 930, in request\n resp = send(**kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1025, in _send_request\n raise exceptions.SSLError(msg)\nkeystoneauth1.exceptions.connection.SSLError: SSL exception connecting to https://openstack.cyberrange.rit.edu:5000: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/tmp/ansible-tmp-1720019606.1431425-10481-181523778594420/AnsiballZ_catalog_service.py\", line 107, in <module>\n _ansiballz_main()\n File \"/tmp/ansible-tmp-1720019606.1431425-10481-181523778594420/AnsiballZ_catalog_service.py\", line 99, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/tmp/ansible-tmp-1720019606.1431425-10481-181523778594420/AnsiballZ_catalog_service.py\", line 47, in invoke_module\n runpy.run_module(mod_name='ansible_collections.openstack.cloud.plugins.modules.catalog_service', init_globals=dict(_module_fqn='ansible_collections.openstack.cloud.plugins.modules.catalog_service', _modlib_path=modlib_path),\n File \"/usr/lib/python3.10/runpy.py\", line 224, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib/python3.10/runpy.py\", line 96, in _run_module_code\n _run_code(code, mod_globals, init_globals,\n File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 211, in <module>\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 207, in main\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/module_utils/openstack.py\", line 415, in __call__\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 113, in run\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 175, in _find\n File \"/opt/ansible/lib/python3.10/site-packages/openstack/service_description.py\", line 89, in __get__\n proxy = self._make_proxy(instance)\n File \"/opt/ansible/lib/python3.10/site-packages/openstack/service_description.py\", line 289, in _make_proxy\n found_version = temp_adapter.get_api_major_version()\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/adapter.py\", line 352, in get_api_major_version\n return self.session.get_api_major_version(auth or self.auth, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1289, in get_api_major_version\n return auth.get_api_major_version(self, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 497, in get_api_major_version\n data = get_endpoint_data(discover_versions=discover_versions)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 268, in get_endpoint_data\n service_catalog = self.get_access(session).service_catalog\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 131, in get_access\n self.auth_ref = self.get_auth_ref(session)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/generic/base.py\", line 203, in get_auth_ref\n self._plugin = self._do_create_plugin(session)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/generic/base.py\", line 155, in _do_create_plugin\n raise exceptions.DiscoveryFailure(\nkeystoneauth1.exceptions.discovery.DiscoveryFailure: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. SSL exception connecting to https://openstack.cyberrange.rit.edu:5000: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1} #config excerpts from globals.yml kolla_base_distro: "ubuntu" kolla_internal_vip_address: "100.70.0.20" # /etc/hosts updated on all hosts to have openstack.cyberrange.rit.edu point to the internal IP kolla_internal_fqdn: "openstack.cyberrange.rit.edu" kolla_external_vip_address: "129.21.246.130" kolla_external_fqdn: "openstack.cyberrange.rit.edu" kolla_enable_tls_internal: "yes" kolla_enable_tls_external: "yes" kolla_certificates_dir: "/etc/kolla/certificates" kolla_external_fqdn_cert: "{{ kolla_certificates_dir }}/haproxy.pem" kolla_internal_fqdn_cert: "{{ kolla_certificates_dir }}/haproxy.pem" #kolla_admin_openrc_cacert: "" kolla_copy_ca_into_containers: "yes" haproxy_backend_cacert: "{{ 'ca-certificates.crt' if kolla_base_distro in ['debian', 'ubuntu'] else 'ca-bundle.trust.crt' }}" haproxy_backend_cacert_dir: "/etc/ssl/certs" letsencrypt_email: "fffics(a)rit.edu" enable_letsencrypt: yes letsencrypt_cert_server: "https://acme-v02.api.letsencrypt.org/directory" # attempt to renew Let's Encrypt certificate every 12 hours letsencrypt_cron_renew_schedule: "0 */12 * * *"

3 2

PTG October 2024 Team Signup
by Kendall Nelson 29 Aug '24

29 Aug '24

Hello Everyone, Hopefully you have seen by now, the next OpenInfra PTG[1] which will take place October 21-25, 2024 virtually! Registration is now open[2]! To sign up your team, you must complete the survey[3] by August 30th at 7:00 UTC. We NEED accurate contact information (email and IRC nickname ideally) for the moderator of your team’s sessions. This is because the survey information will be used to organize the schedule signups which will be done via the PTGBot. If you are not on IRC, please get setup[4] on the OFTC network and join #openinfra-events. You are also encouraged to familiarize yourself with the PTGBot documentation[5] as well. If you have any questions, please reach out to ptg(a)openinfra.dev! Information about signing up for time slots will be sent to moderators after the team signup deadline. Continue to visit openinfra.dev/ptg for updates. -Kendall (diablo_rojo) [1] PTG Site: https://openinfra.dev/ptg/ [2] PTG Registration: http://ptg.openinfra.dev/ [3] Team Survey: https://openinfrafoundation.formstack.com/forms/oct2024_ptg_survey [4] Setup IRC: https://docs.openstack.org/contributors/common/irc.html [5] PTGBot README: https://opendev.org/openstack/ptgbot/src/branch/master/README.rst

1 2

[MAGNUM] Fedora Core OS include NVIDIA GPU drivers
by Oliver Weinmann 19 Aug '24

19 Aug '24

Hi all, I'm running OpenStack 2023.2 (Bobcat) deployed with Kolla-Ansible. Currently I'm struggling to get the NVIDIA drivers to work in a FCOS 35 image. I know FCOS 35 is EOL and outdated, but for a K8s 1.21.11 cluster it is the only version that still works with Magnum. So far I managed to get the GPU Passthrough working for two NVIDIA Tesla T4 GPUs. I have attached the GPUs to an Ubuntu instance and ran geekbench without any issues. Then I tried to install the latest NVIDIA drivers in FCOS 35 by running: sudo rpm-ostree install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$ (rpm -E %fedora).noarch.rpm https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$ (rpm -E %fedora).noarch.rpm systemctl reboot rpm-ostree install --apply-live akmod-nvidia akmods pciutils rpmdevtools xorg-x11-drv-nvidia-cuda It fails on: rpm-ostree install --apply-live akmod-nvidia akmods pciutils rpmdevtools xorg-x11-drv-nvidia-cuda error: Cannot download rpm-build-libs-4.17.0-4.fc35.x86_64.rpm: All mirrors were tried; Last error: Status code: 403 for https://fedoraproject-updates-archive.fedoraproject.org/fedora/35/x86_64/rp… I guess the binaries are no longer hosted? Running the same on FCOS 36 and latest FCOS 40 works fine. Next I have taken new images from the FCOS instances with the NVIDIA driver. openstack server image create --name fedora_coreos_36_nvidia_cuda fedora_coreos_35_latest_no_update --wait This takes ages but anyway I can successfully deploy new instances from the images with my gpu flavor and the GPU is working fine. Again tested with geekbench. But now when adding new K8s worker nodes in a fresh nodegroup on an existing K8s 1.21.11 cluster the nodes are not joining the cluster: openstack coe nodegroup create \ --node-count 1 \ --role worker-gpu \ --flavor gpuflavor \ --image fedora_coreos_36_nvidia_cuda \ $CLUSTER_ID worker-gpu I have tested the latest available images of FCOS36 and they work fine, so something must be wrong with how I created the new image. Is there some magic that needs to be done? I'm also not sure if this is the best way of doing this. Maybe there is already a ready FCOS image with nvidia drivers? I can find lots of outdated information on the internet. Generally I tend to use Vexxhost CAPI from now on, which works great. This is really just a one off attempt to have a working NVIDIA T4 GPU in a K8s 1.21.11 cluster. Best Regards, Oliver

5 16

networking-onos plugin
by Ganesan, Sathya 14 Aug '24

14 Aug '24

Good Day, Is there an alternative integration available for ONOS and Openstack since networking-onos is archived? Sathya Ganesan Geico Tech | Data Center Core Network Engineering - SDN | sganesan(a)geico.com<mailto:sganesan@geico.com> | 240.478.2121 | M-F 9a-5:30p EST Sensitivity: Company-Internal ==================== This email/fax message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution of this email/fax is prohibited. If you are not the intended recipient, please destroy all paper and electronic copies of the original message.

2 2

[rabbitmq] Maximum Number of Sessions (8192) Reached
by Eugen Block 14 Aug '24

14 Aug '24

Hi *, I can't seem to find much about rabbitmq and logind, so I wanted to ask the list if anyone has encountered the same and if so, how they dealt with it. We're supporting a Victoria cluster (installed with our own deployment method) mostly controlled by pacemaker. And on the rabbit master node I see this warning constantly: ---snip--- 2024-07-29T14:09:23.552576+02:00 control01 su: pam_unix(su:session): session opened for user rabbitmq by (uid=0) 2024-07-29T14:09:24.450657+02:00 control01 su: pam_unix(su:session): session closed for user rabbitmq 2024-07-29T14:09:24.500356+02:00 control01 su: (to rabbitmq) root on none 2024-07-29T14:09:24.502370+02:00 control01 su: pam_systemd(su:session): Failed to create session: Maximum number of sessions (8192) reached, refusing further sessions. 2024-07-29T14:09:24.502681+02:00 control01 su: pam_unix(su:session): session opened for user rabbitmq by (uid=0) 2024-07-29T14:09:25.565203+02:00 control01 su: pam_unix(su:session): session closed for user rabbitmq 2024-07-29T14:09:25.609613+02:00 control01 su: (to rabbitmq) root on none ---snip--- Looking into loginctl list-sessions, almost all of them belong to rabbitmq and they have a very old timestamp (2023). I'm aware of older systemd versions which can't handle closing sessions correctly [0], but we can't upgrade at this time. Would enabling "linger" fix this (loginctl enable-linger rabbitmq) after a reboot? During the next maintenance window I would reboot control01 and watch how the other control nodes behave wrt rabbit login-sessions. But I'm wondering if somebody already has dealt with this? Thanks for any pointers! Eugen [0] https://www.suse.com/support/kb/doc/?id=000020549

1 2

Firewall as a Service - kolla ansible 2023.1
by Thamanna Farhath 06 Aug '24

06 Aug '24

Hello OpenStack Community, I hope this message finds you well. My name is Thamanna , and I am currently working with the OpenStack Kolla-Ansible deployment version 2023.1. I am interested in deploying Firewall as a Service (FWaaS) within our OpenStack environment. However, I have noticed that FWaaS is not available in the Kolla-Ansible 2023.1 version. Could anyone please provide guidance on the following points? 1. Is there an official or recommended way to integrate FWaaS with Kolla-Ansible 2023.1? 2. Are there any specific configurations or additional components required to enable FWaaS? 3. Has anyone in the community successfully deployed FWaaS with this version, and if so, could you share your experience and any potential challenges faced? 4. Are there alternative approaches or tools that you would recommend for deploying FWaaS in an OpenStack environment? Any help or pointers to relevant documentation would be greatly appreciated. Thank you in advance for your assistance! Best regards, Thamanna mailto:thamanna.f@zybisys.com Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

3 5

Live migration fails
by Szabo, Istvan (Agoda) 06 Aug '24

06 Aug '24

Hi, I have couple of compute nodes where the live migration fails with existing vms. When I quickly spawn a vm and try live migration it works so I assume shouldn't be a big problem with the compute node. However I have many existing vms where it fails with a servername not found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 ERROR nova.conductor.tasks.migrate [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Unable to find record for source node servername on servername: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Failed to compute_task_migrate_server: Compute host servername could not be found.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Setting instance to ACTIVE state.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.672 227612 ERROR oslo_messaging.rpc.server [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Exception during message handling: ComputeHostNotFound: Compute host am-osfecn-4025 Tried with this command: nova live-migration --block-migrate id. Any idea? Thank you. ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

5 16

Assistance needed - Application Credential
by Naveen Anbarasu 05 Aug '24

05 Aug '24

Hi team, I have created an application credential for the compute service and successfully obtained a token. Access rule: "service": "compute", "method": "GET", "path": "/v2.1/servers" However, when using the token to retrieve the server list, I receive a Error 403 Forbidden error with the message: 'Policy doesn't allow os_compute_api:servers to be performed. But i have a necessary permission within the respective project (member role) How can I solve this problem? Thanks in advance

7 12

[openstack][kolla-ansible]about upgrading openstack
by Nguyễn Hữu Khôi 04 Aug '24

04 Aug '24

Hello. I read this doc about upgrading openstack https://docs.openstack.org/kolla-ansible/latest/user/operating-kolla.html But I would like to know if We install a new kolla-ansible(version to upgrade) on the new deployment host, apply changes with password.yml, global.yml and multinode and keep old /etc/kolla/config. Finally, we use kolla-ansible deploy instead of kolla-ansible upgrade. Are these steps ok? Pls correct me if I am wrong. Thank you. Regards Nguyen Huu Khoi

3 8

[election] [tc] Notification of non-candidacy; how it impacts TC diversity
by Jay Faulkner 01 Aug '24

01 Aug '24

Hi all, The nomination period for TC members / PTLs is coming up soon. I wanted to inform the community that I do not intend on standing for election for another year on the TC. Now is a good time, especially if you work somewhere other than Red Hat, to consider serving as a TC member to ensure our leadership team contains representation for all perspectives of OpenStack use. I'm happy to speak with anyone privately who is interested in running about what it's like to be on the TC, and to share some strategies I exercised to ensure the value of my TC membership was being communicated to leadership at my job. Thanks, Jay Faulkner Technical Committee Member

6 5

oslo.service
by engineer2024 01 Aug '24

01 Aug '24

Can anyone guide me how to configure and use oslo.service for nova-api service for tracing and debugging ? Thanks elinux

3 2

[cinder] Stepping down from Cinder core
by Pete Zaitcev 01 Aug '24

01 Aug '24

Hello: Because of a job change, my time available is becoming quite limited, so I'm not going to be able to contribute at the appropriate level. So, I contacted our PTL to get me removed from the core team. Thanks everyone for the help and the journey in Cinder. Greetings, Pete

3 2

nova deploy issue
by Parsa Aminian 30 Jul '24

30 Jul '24

hello when I try to reconfigure nova with kolla ansible this error will be shown : TASK [nova-cell : Waiting for nova-compute services to register themselves] ******************************************************************************************* task path: /usr/local/share/kolla-ansible/ansible/roles/nova-cell/tasks/wait_discover_computes.yml:8 fatal: [control01 -> {{ groups[nova_cell_conductor_group][0] }}]: FAILED! => { "msg": "The conditional check 'inventory_hostname == all_computes_in_batch[0]' failed. The error was: error while evaluating conditional (inventory_hostname == all_computes_in_batch[0]): 'all_computes_in_batch' is undefined\n\nThe error appears to be in '/usr/local/share/kolla-ansible/ansible/roles/nova-cell/tasks/wait_discover_computes.yml': line 8, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n- block:\n - name: Waiting for nova-compute services to register themselves\n ^ here\n" Does anyone have any idea what I should do ?

1 0

Assistance Needed for Trove Instance Creation
by Thamanna Farhath 30 Jul '24

30 Jul '24

Hello community, I am currently deploying Trove using Kolla Ansible 2023.1, following the guide provided here: https://satishdotpatel.github.io/trove-dbaas-openstack/ . Despite cleaving to the instructions, I am facing an issue where the Trove instance ends up in an error state. trove-guestagent.conf [DEFAULT] transport_url = rabbit://openstack:USI0SmaT7lPF7MJCxHPZ4alm8wiyxoihssD1VlnA@192.168.1.101:5672,openstack:USI0SmaT7lPF7MJCxHPZ4alm8wiyxoihssD1VlnA@192.168.1.102:5672,openstack:USI0SmaT7lPF7MJCxHPZ4alm8wiyxoihssD1VlnA@192.168.1.103:5672// control_exchange = trove root_grant = ALL root_grant_option = True debug = False Docker Status Check: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f698ee14d928 mysql:5.7.29 "docker-entrypoint.s…" 18 minutes ago Up 18 minutes 0.0.0.0:3306->3306/tcp, :::3306->3306/tcp, 33060/tcp database Telnet Connection: telnet 192.168.50.78 3306 Trying 192.168.50.78... Connected to 192.168.50.78. Escape character is '^]'. J 5.7.29p+C5 }0^k[zH3mysql_native_password Syslog(SSH into the trove instance) ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)\n": Exception: Running command error: b"mysql: [Warning] Using a password on the command line interface can be insecure.ERROR trove.guestagent.datastore.service [None req-a03ff3dd-5a8f-4fd4-80d5-76e31fa16dd0 - - - - - -] Timeout while waiting for database status to change. Expected state healthy, current state is running When looking for the missing directory, I discovered it in /var/lib/mysql/mysqld.sock and /var/lib/mysql/mysqld.pid paths. How can I get an active instance?Kindly help me resolve this. Thank you in advance Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

3 2

[horizon] How to enable MFA using TOTP on Horizon (ZED)
by Shubham Kumar Yadav 30 Jul '24

30 Jul '24

Hi all, I was working on a MFA in openstack (ZED). Is it possible to enable MFA using TOTP on Horizon? I'm using a ZED Openstack with Ubuntu 22.04 , and I know that from the CLI point of view, you can have Multi Factor Authentication, but I don't see that implemented on Horizon. i added MFA using - https://docs.openstack.org/keystone/pike/advanced-topics/auth-totp.html but now the problem is i can use the token issued to run openstack cli but not able to login into Horizon with user and password/totp. I see The Totp support was added to Horizon in the Bobcat (2023.2) release So i would like to ask how it is possible to backport the feature in Horizon in the zed release. I would appreciate any help! Thanks!

2 1

[ironic] ironic-inspector bugfix branches
by Riccardo Pittau 30 Jul '24

30 Jul '24

Hello ironicers! With the deprecation and the migration of the ironic-inspector functionalities to the new agent interface in ironic we have decided to also stop cutting bugfix branches for the project. The last bugfix branch for ironic-inspector is bugfix/12.2 supported until December 2024. We'll continue with the normal release schedule until full retirement of the project. Thanks Ciao! Riccardo

1 0

[neutron] CI Meeting cancelled for today
by Yatin Karel 30 Jul '24

30 Jul '24

Hello Neutron Team !! I will not be able to chair the CI meeting today as I have to attend some internal meetings. Sorry for the late notice. We have some known issues in CI and all are under track with patches in gate or already merged. There are some issues in the periodic line specific to ovs-master jobs and 9-stream so will report issues for those. -- Thanks and Regards Yatin Karel

1 0

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2024.2/R-9)
by Goutham Pacha Ravi 29 Jul '24

29 Jul '24

Hello Stackers, We're counting down 9 weeks to the 2024.2 (Dalmatian) release [1]. OpenStack projects that follow the cycle-with-rc release model [2] will have their code freeze in 4 weeks. Between now and the release, the release team will communicate with us weekly to coordinate deliverables across the community. The elections for OpenStack PTLs and Technical Committee members are around the corner as well, so now is a good time to ensure that you are eligible to vote in these elections! Please navigate to: https://civs1.civs.us/cgi-bin/opt_in.pl Enter your preferred email address on Gerrit and request activation. It only takes a minute! :) The following are updates from the OpenStack Technical Committee during the past week. === Weekly Meeting === The TC had a productive meeting via IRC on OFTC's #openstack-tc channel last week [3]. We discussed follow-ups for the eventlet removal goal proposal [4] and recent updates to inactive and emerging OpenStack projects. We spent some time assessing gate health concerns, including an issue with a change to `setuptools` that requires us to bump the upper constraint on the "packaging" library in stable/2023.1 [5]. This was important to get some CVE fixes landed in various OpenStack code repositories. Finally, we refreshed our understanding of the TC's house rules. To summarize: - Changes to the TC charter require at least 2/3 of the TC voting in favor, while governance motions require at least 1/3 of the TC to vote in favor. - TC members can vote positively, negatively, or abstain (using the “RollCall-Vote” in Gerrit). - Decisions need more positive votes than negative votes (ties mean the proposal is rejected). - Any motion or charter-change is deliberately open at least a week after it was last proposed/updated, allowing the TC and the wider community to participate fully. - Significant governance changes take a while to merge because of the importance of ensuring full TC and community engagement in votes. Our next weekly meeting is tomorrow (2024-07-30) at 1800 UTC via IRC on OFTC's #openstack-tc channel. You can find the agenda on the OpenStack TC wiki page [6]. Please consider joining us and remember, you can edit the wiki and propose new topics! === Governance Proposals === ==== Merged ==== - Add `os-test-images` under glance | https://review.opendev.org/c/openstack/governance/+/925044 ==== Open for review/discussion ==== - [Follow-up] Fix typos in the eventlet proposal | https://review.opendev.org/c/openstack/governance/+/924944 - Update criteria for inactive projects to become active again | https://review.opendev.org/c/openstack/governance/+/921500 - Follow up on the affiliation diversity handling | https://review.opendev.org/c/openstack/governance/+/923876 - Inactive state extensions: Monasca | https://review.opendev.org/c/openstack/governance/+/923919 === Upcoming Events === - 2024-08-12: Call for Proposals due for the OpenInfra Days NA [7] - 2024-08-14: Nominations open for 2025.1 OpenStack PTL+TC elections - 2024-08-24: OpenInfra Days, Vietnam: https://2024.vietopeninfra.org/ - 2024-08-29: Dalmatian-3 milestone - 2024-09-03: OpenInfra Summit Asia, Suwon The OpenStack Technical Committee serves you! We're always willing to chat and would love feedback: === How to contact the TC === You can reach out to the TC in multiple ways: 1. Email: Send an email with the tag [tc] on this email list. 2. Ping us using the 'tc-members' keyword on the #openstack-tc IRC channel. 3. Join us at our weekly meeting: The Technical Committee meets every week on Tuesdays at 1800 UTC [7]. Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] Release Schedule: https://releases.openstack.org/dalmatian/schedule.html [2] OpenStack cycle-with-rc model: https://releases.openstack.org/reference/release_models.html#cycle-with-rc [3] TC Meeting IRC Logs, 2024-07-23: https://meetings.opendev.org/meetings/tc/2024/tc.2024-07-23-18.00.log.html [4] Remove Eventlet From Openstack: https://review.opendev.org/c/openstack/governance/+/902585 [5] Update packaging constraint in stable/2023.1: https://review.opendev.org/c/openstack/requirements/+/924764 [6] TC Meeting, 2024-07-30: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [7] Call for Proposals, OpenInfra Days NA: https://docs.google.com/forms/d/e/1FAIpQLSeFQBL1qGNegDWDHCVTLtu_wLf06w7JJkc…

1 0

Cinder Bug Report 2024-07-29
by Jon Bernard 29 Jul '24

29 Jul '24

Hello Argonauts, Cinder Bug Meeting Etherpad <https://etherpad.opendev.org/p/cinder-bug-squad-meeting> Undecided - When restoring volume using the imported backup, it is always scheduled to the same host. <https://bugs.launchpad.net/cinder/+bug/2074058> - Status: In Progress (assigned) - The cinder-backup swift driver connects to swift in a different region. <https://bugs.launchpad.net/cinder/+bug/2074126> - Status: In Progress (assigned) - allow_qcow2_backing_file flag cannot be passed into some image_utils operations <https://bugs.launchpad.net/cinder/+bug/2074377> - Status: New (unassigned) Thanks, -- Jon

1 0

[nova][ops] Deprecate AMI/direct kernel boot support
by Dan Smith 29 Jul '24

29 Jul '24

Hi all, tl;dr: We really need ops feedback about any direct need to retain this feature. Since the early days, Nova has supported the notion of "direct linux kernel boot" (with libvirt/kvm at least). This means each bootable image in Glance is actually three parts: a disk image, a ramdisk image, and a kernel. Booting the instance involves putting the kernel and ramdisk into memory in a special way that the kernel can immediately start booting instead of doing things like running the bootloader like a normal system does. This is present in Nova because of long-ago Amazon compatibility, as this is how Xen paravirt guests used to work for them. I think that in the early days of Nova, compatibility with the _actual_ images used on Amazon was important, but I think that is no longer something we need to care about today, just as we have deprecated EC2 API compatibility. It should also be noted that some of it is broken, and the recent CVE fixes have broken it like three times in the last 30 days. Continuing to support it makes the code more complex and fragile, and arguably less secure. So, I'm asking for operator feedback on this question: Do you need this feature, and if so, why? Speak now or expect to see it deprecated for removal very soon. Thanks! --Dan

3 5

[neutron] Bug deputy report (week starting on July 22)
by Bernard Cafarelli 29 Jul '24

29 Jul '24

Hey neutrinos, here is the list of bugs reported during 2024 week 30 Most are in progress (or already fixed), except these: https://bugs.launchpad.net/bugs/2074207 (endless loop in DHCP agent), https://bugs.launchpad.net/bugs/2073872 (L3 HA routes issue in Yoga) and https://bugs.launchpad.net/neutron/+bug/2074056 (security groups doc/API) Critical * Couple of jobs broken as rpc-server not enabled since patch 924473 - https://bugs.launchpad.net/bugs/2073844 Mentioning for visibility, this was fixed in devstack with https://review.opendev.org/c/openstack/devstack/+/924693 * compute tests involving shelve/snapshots/rescue fails with Image is unacceptable: Image content does not match disk_format - https://bugs.launchpad.net/bugs/2073944 Also for visibility, fixed in nova with https://review.opendev.org/c/openstack/nova/+/924866 High * Switch from distributed to centralized Floating IPs breaks connectivity to the existing FIPs - https://bugs.launchpad.net/neutron/+bug/2073987 Fix in progress by slaweq - https://review.opendev.org/c/openstack/neutron/+/925007 * DHCP agent makes endless attempts to configure a network with MTU < 1280 - https://bugs.launchpad.net/bugs/2074207 Related/follow-up to similar https://bugs.launchpad.net/neutron/+bug/1988069, unassigned Medium * ``ProcessMonitor._check_child_processes`` cannot release the GIL inside a locked method - https://bugs.launchpad.net/bugs/2073743 Patch by Rodolfo - https://review.opendev.org/c/openstack/neutron/+/924581 * [eventlet-deprecation] Reduce the ``IpConntrackManager`` process pool to a single thread - https://bugs.launchpad.net/neutron/+bug/2073745 Patch by Rodolfo - https://review.opendev.org/c/openstack/neutron/+/924582 * "Tagging" extension does not initialize the policy enforcer - https://bugs.launchpad.net/bugs/2073782 Patch by Rodolfo - https://review.opendev.org/c/openstack/neutron/+/924656 * "Tagging" extension cannot add tags with charater "/" - https://bugs.launchpad.net/bugs/2073836 Patches by Rodolfo - https://review.opendev.org/c/openstack/neutron-lib/+/924700 and https://review.opendev.org/c/openstack/neutron/+/924724 * Neutron L3 agent not create ECMP in HA router but in single mode is ok - https://bugs.launchpad.net/bugs/2073872 L3-HA bug reported on Yoga, needs some eyes * IPv6 dns nameservers described with their scope on the IP are not supported - https://bugs.launchpad.net/bugs/2073894 Elvira working on it (fix in oslo.utils) * Subnet pool attribute updates fail when using address scopes - https://bugs.launchpad.net/neutron/+bug/2074092 Possible regression from https://review.opendev.org/c/openstack/neutron/+/667511, fix discussed https://review.opendev.org/c/openstack/neutron/+/924984 * [OVN] Trunk subports should have the same VIF type as the parent port - https://bugs.launchpad.net/neutron/+bug/2074185 Assigned to Rodolfo * OVN maintenance tasks may be delayed 10 minutes in the podified deployment - https://bugs.launchpad.net/bugs/2074209 Assigned to Slaweq Low * Invalid documented security group rule protocol "any" - https://bugs.launchpad.net/neutron/+bug/2074056 Sounds like low-hanging fruit on documentation and security groups * DB semantic violation seen in neutron-tempest-plugin-openvswitch logs deleting router - https://bugs.launchpad.net/bugs/2074312 Assigned to Brian Wishlist * [RFE] Extend "port list" command to show the trunk subports - https://bugs.launchpad.net/bugs/2074187 -- Bernard Cafarelli

1 0

[release][ptl] 2025.1 Epoxy release schedule - review request from teams
by Elõd Illés 29 Jul '24

29 Jul '24

Hi, Let me share the proposed 2025.1 Epoxy release schedule [1] (see the generated page [2] - for Patch Set 2 !!!). PTLs, teams, please review it and let us know your opinions! [1] https://review.opendev.org/c/openstack/releases/+/924671 [2] https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/… Cheers, Előd Illés irc: elodilles @ #openstack-release

1 0

how upgrade OpenStack version 5.8.0 to 6.1.0
by kjme001＠gmail.com 28 Jul '24

28 Jul '24

Hello I have OpenStack version 5.8.0, how do I upgrade it to a newer version, e.g. to version 6.1.0? thanks for your help regards

5 6

[nova] rescheduler
by Alex Song (宋文平) 26 Jul '24

26 Jul '24

Hi all, Is the reschedule flow only for building instance? Why evacuate and migration hasn’t? Thanks in advance Alex song

3 2

[qa] Core lists cleanup
by Martin Kopec 26 Jul '24

26 Jul '24

Hi everyone, we're planning to clean up the QA core lists, which have grown over time, to ensure they contain only active contributors. This will help anyone seeking a core reviewer for their patch know whom to contact. A core reviewer's activity will be assessed based on a 2-year period of making reviews. In other words, we will remove all users who haven't made any reviews in the respective repositories in the past 2 years. The cleanup will focus on these 3 core lists: * tempest-core * devstack-core * qa-release We're going to assess the activity and make the changes not sooner than August 12. If this cleanup affects you and you still wish to be included in one of the above core lists and work in a core reviewer capacity, please let me know. Thanks, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 0

[neutron] Drivers meeting 26.07.2024 cancelled
by Sławek Kapłoński 26 Jul '24

26 Jul '24

Hi, There is no agenda for today's meeting so I am cancelling it. Have a great Friday and weekend. See You on the meeting next week :) -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

keystone credentials
by engineer2024 25 Jul '24

25 Jul '24

Hi all, We are receiving this message while executing keystone playbook to upgrade from yoga to zed through openstack ansible: ---- FAILED! => {"changed": true, "cmd": ["/openstack/venvs/keystone-26.3.0/bin/keystone-manage", "credential_migrate", "--keystone-user", "keystone", "--keystone-group", "keystone"], "msg": "non-zero return code", "rc": 1,} --- Then I tried credential_setup to setup new fernet keys which succeeded, but when trying to do credential_migrate, it is still failing with the above message. The debug log shows something like "unable to decrypt; use the same primary key used for encryption" I even copied the newly setup keys to the other two containers. Stil it does not work. As a last resort , I have destroyed the three keystone containers and recreated and ran the playbook. Then it worked. But in our scenario, this caused downtime for the apps using this keystone setup which attracted an RCA requirement. So, I want to know how to avoid downtime as in a normal upgrade , when creating/destroying containers and the reason for the above error and the workaround... Appreciate your time. Thanks elinux

1 1

Upcoming Student Opportunities
by Kendall Nelson 25 Jul '24

25 Jul '24

Hello Everyone, I have a number of universities looking for projects and mentors for students in the coming semester. Three universities in particular - DIckinson University, Boston University, and the University of Jos in Nigeria. Dickinson University's course takes place over the whole academic year (two semesters) starting the start of September. Students will work in groups of 2-4 with the actual contribution period being October through May. Most students will be in their last year of a Bachelor's degree in CS. Boston University's course takes place across one semester. Groups of students will be 3-5 people and students will also be in their last year of a Bachelor's degree in CS. Students will make their project selections early September and begin contributing soon after. It's likely this course will also take place next semester. *For both universities, I need project proposals and volunteers (if interested of course), by the 23 of August. * The University of Jos partnership is more internship based and has a more flexible timeline. I am still gathering more details on what this will look like, but I wanted to give you all a heads up. If you have questions about any of these opportunities, please let me know!! Likewise, if you're interested, please let me know and I can support you in getting a project proposal together. -- Kendall Nelson *Senior Upstream Developer Advocate* The OpenInfra Foundation <https://openinfra.dev/>

1 0

update ubuntu and nova does not work
by kjme001＠gmail.com 25 Jul '24

25 Jul '24

update ubuntu and nova does not work During the update of ubuntu I selected option Y, as below and broke one node, the new service starts but does not work from the cluster side, what can I do now to fix it? Configuration file '/etc/nova/nova.conf' ==> Modified (by you or by a script) since installation. ==> Package distributor has shipped an updated version. What would you like to do about it ? Your options are: Y or I : install the package maintainer's version N or O : keep your currently-installed version D : show the differences between the versions Z : start a shell to examine the situation The default action is to keep your current version. *** nova.conf (Y/I/N/O/D/Z) [default=N] ?

2 1

[manila] Proposing Kiran Pawar to manila core
by Carlos Silva 25 Jul '24

25 Jul '24

Hello everyone, I would like to add Kiran Pawar to the OpenStack Manila core team. Kiran has been contributing with reviews [1], new features, bug fixes [2] to the OpenStack Manila components for several cycles and has clearly acquired knowledge on many areas of Manila. He has designed and worked in the implementation of several features in the core of Manila, such as share backups, deferred deletion for shares and snapshots, and most recently has been working on shares encryption and integrating Manila with Barbican. He's clearly demonstrated understanding not only on the manila core, but also in the integrations with share drivers. The participation on PTGs and other events evolving Manila has also been strong, and very impactful. Kiran also brings knowledge on large cloud environments, having worked closely on large deployments and finding solutions for their problems. This addition also means more diversity among the Manila core group, and maintaining a healthy community with different companies sharing the responsibility of maintaining an open source project. Kiran brings a lot of experience with cloud environments to the table and has been helpful on reviews and contributing with a lot of code to Manila, making him a good candidate to join the core team. In case there are no objections, I will add him to the core team in 1 week from now. [1] https://bit.ly/manila-contribution-60-days [2] https://bit.ly/kpdev-code-contributions-to-manila Thank you! Regards, carloss

4 4

[manila] Proposing Ashley Rodrigues to python-manilaclient and manila-ui core
by Carlos Silva 25 Jul '24

25 Jul '24

Hello everyone, I would like to nominate Ashley Rodriguez as python-manilaclient and manila-ui core. Ashley started contributing to OpenStack Manila more than three years ago, and managed to work on a couple of features such as the integration of Manila with OpenStack SDK, metadata for shares and share snapshots, several enhancements to the Ceph drivers testing on CI and several other bug fixes [1], which helped her in acquiring precious knowledge on the code bases, testing and providing feedback. Ashley has also been working as a mentor for many mentorship initiatives we have been promoting in Manila (Outreachy, NDSU, GHC and CMU), where we focused on different repositories, but she always demonstrated the ability to guide the interns providing insightful reviews, debugging and demonstrating good knowledge on python-manilaclient and manila-ui. This addition to the core team is one more step into maintaining a healthy community, as well as a very good way to recognize all the work and dedication Ashley has been putting on these repositories in different ways. In case there are no objections, I will add Ashley to the python-manilaclient and manila-ui core sub groups in 1 week from now. [1] https://review.opendev.org/q/owner:ashrod98@redhat.com [2] https://bit.ly/manila-contribution-60-days Thank you! Regards, carloss

4 4

Updating Instance Flavor Properties in OpenStack Cluster Without Downtime
by Mia smith 25 Jul '24

25 Jul '24

Hello OpenStack Community, In our OpenStack cluster, we recently updated some flavor properties. As a result, we need to apply these changes to all currently active instances that were created with the old flavor properties. We are looking for a solution that allows us to update these instance properties without causing any downtime or needing to resize the instances. Could anyone provide suggestions on how to achieve this? Is there a recommended method or ad-hoc command that can be used to safely update the properties directly in the database? Thank you for your assistance!

3 8

[Openstack-ansible][train] multi node deployment with CEPH-ansible
by Danish Khan 25 Jul '24

25 Jul '24

Dear Team, I am trying to deploy an openstack-ansible train with Ceph-ansible. I am stuck in a loophole due to python dependencies. As suggested, I applied a fix for the rabbitmq version and I am able to deploy openstack-ansible train successfully without ceph. But when I am deploying it with ceph then it is getting stuck at below: TASK [ceph_client : Install ceph packages] ********************************************************************************************************************************* FAILED - RETRYING: Install ceph packages (5 retries left). FAILED - RETRYING: Install ceph packages (4 retries left). FAILED - RETRYING: Install ceph packages (3 retries left). FAILED - RETRYING: Install ceph packages (2 retries left). FAILED - RETRYING: Install ceph packages (1 retries left). fatal: [sp-uat-infra-c1]: FAILED! => {"attempts": 5, "cache_update_time": 1721053914, "cache_updated": false, "changed": false, "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\" install 'ceph-common' 'python-ceph'' failed: E: Unable to correct problems, you have held broken packages.\n", "rc": 100, "stderr": "E: Unable to correct problems, you have held broken packages.\n", "stderr_lines": ["E: Unable to correct problems, you have held broken packages."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSome packages could not be installed. This may mean that you have\nrequested an impossible situation or if you are using the unstable\ndistribution that some required packages have not yet been created\nor been moved out of Incoming.\nThe following information may help to resolve the situation:\n\nThe following packages have unmet dependencies:\n python-ceph : Depends: python-cephfs (>= 12.2.13-0ubuntu0.18.04.11) but it is not going to be installed\n Depends: python-rados (>= 12.2.13-0ubuntu0.18.04.11) but it is not going to be installed\n Depends: python-rbd (>= 12.2.13-0ubuntu0.18.04.11) but it is not going to be installed\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Some packages could not be installed. This may mean that you have", "requested an impossible situation or if you are using the unstable", "distribution that some required packages have not yet been created", "or been moved out of Incoming.", "The following information may help to resolve the situation:", "", "The following packages have unmet dependencies:", " python-ceph : Depends: python-cephfs (>= 12.2.13-0ubuntu0.18.04.11) but it is not going to be installed", " Depends: python-rados (>= 12.2.13-0ubuntu0.18.04.11) but it is not going to be installed", " Depends: python-rbd (>= 12.2.13-0ubuntu0.18.04.11) but it is not going to be installed"]} python-ceph is not getting install due to dependencies: #vi /etc/ansible/roles/ceph_client/vars/debian.yml. python_ceph_packages: - python-ceph - python3-rados - python3-rbd I tried to update *python-ceph* with *python3-ceph* and the deployment was successful but the cinder-volume service is not coming up due to python error: #journalctl -u cinder-volume.service -f 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service [req-53d87e63-50ab-4976-b531-2b8ffdb82900 - - - - -] Error starting thread.: TypeError: expected bytes, str found 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service Traceback (most recent call last): 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/oslo_service/service.py", line 810, in run_service 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service service.start() 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/cinder/service.py", line 229, in start 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service service_id=Service.service_id) 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/cinder/volume/manager.py", line 444, in init_host 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service self._init_host(added_to_cluster, **kwargs) 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/cinder/volume/manager.py", line 478, in _init_host 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service self.driver.init_capabilities() 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/cinder/volume/driver.py", line 760, in init_capabilities 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service stats = self.get_volume_stats(True) 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/cinder/volume/drivers/rbd.py", line 643, in get_volume_stats 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service self._update_volume_stats() 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/cinder/volume/drivers/rbd.py", line 620, in _update_volume_stats 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service free_capacity, total_capacity = self._get_pool_stats() 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "/openstack/venvs/cinder-20.2.6/lib/python3.6/site-packages/cinder/volume/drivers/rbd.py", line 549, in _get_pool_stats 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service '{"prefix":"df", "format":"json"}', '') 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service File "rados.pyx", line 1067, in rados.Rados.mon_command 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service TypeError: expected bytes, str found 2024-07-15 21:18:43.753 1277912 ERROR oslo_service.service Regards, Danish Khan

2 5

[ops] [ec2-api] DB duplicate entry with ec2 list-images
by Massimo Sgaravatto 25 Jul '24

25 Jul '24

Dear all On a Yoga installation I see a problem with the ec2-api: [sgaravat@lxsgaravat ~]$ euca-describe-images -I ${EC2_ACCESS_KEY} -S ${EC2_SECRET_KEY} euca-describe-images: error (DBDuplicateEntry): Unknown error occurred. [sgaravat@lxsgaravat ~]$ In the log I see a duplicate entry errore [*] It's not fully clear to me what the problem is and why this query should add an entry in that table ... Any help is appreciated Thanks, Massimo PS: yes, I am aware that the ec2-api project is "dead" and not part of recent OpenStack distributions [*] 2024-07-25 14:49:11.052 37118 ERROR ec2api.api oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '4ae7a803-d7cd-4725-8b2e-65797a64ef94' for key 'items.items_os_id_idx'") 2024-07-25 14:49:11.052 37118 ERROR ec2api.api [SQL: INSERT INTO items (id, project_id, vpc_id, os_id, data) VALUES (%(id)s, %(project_id)s, %(vpc_id)s, %(os_id)s, %(data)s)] 2024-07-25 14:49:11.052 37118 ERROR ec2api.api [parameters: {'id': 'ami-e561c280', 'project_id': '56c3f5c047e74a78a71438c4412e6e13', 'vpc_id': None, 'os_id': '4ae7a803-d7cd-4725-8b2e-65797a64ef94', 'data': '{"is_public": true}'}] 2024-07-25 14:49:11.052 37118 ERROR ec2api.api (Background on this error at: https://sqlalche.me/e/14/gkpj) 2024-07-25 14:49:11.052 37118 ERROR ec2api.api 2024-07-25 14:49:11.055 37118 ERROR ec2api.api [req-cda08c1a-009c-4c7d-b019-c30bce32baa4 9bd03f63fa9d4beb8de31e6c2f2c8d12 56c3f5c047e74a78a71438c4412e6e13 - - -] Environment: {"wsgi.url_scheme": "http", "REQUEST_METHOD": "POST", "SCRIPT_NAME": "", "RAW_PATH_INFO": "/services/Cloud", "PATH_INFO": "/services/Cloud", "CONTENT_TYPE": "application/x-www-form-urlencoded", "CONTENT_LENGTH": "53", "SERVER_PROTOCOL": "HTTP/1.0", "SERVER_NAME": "192.168.60.42", "SERVER_PORT": "8788", "REMOTE_ADDR": "192.168.60.229", "REMOTE_PORT": "33840", "GATEWAY_INTERFACE": "CGI/1.1", "HTTP_ACCEPT_ENCODING": "gzip, deflate", "HTTP_ACCEPT": "*/*", "HTTP_USER_AGENT": "euca2ools/3.4.1 (CPython 2.7.5; Linux 3.10.0-1160.92.1.el7.x86_64; x86_64) requestbuilder/0.7.1 requests/2.19.1", "HTTP_HOST": " cloud-areapd-test.pd.infn.it:8788", "HTTP_X_AMZ_CONTENT_SHA256": "afe617936fea90d9c62d9f87dd3e9d18e1fc5178d08aeebfb8737f32042a0b6d", "HTTP_X_AMZ_DATE": "20240725T124910Z", "HTTP_AUTHORIZATION": "AWS4-HMAC-SHA256 Credential=220cd4ffe5aa435cb56adfd5a9a3b234/20240725/undefined-15030/ec2/aws4_request, SignedHeaders=accept;accept-encoding;content-length;content-type;host;x-amz-content-sha256;x-amz-date, Signature=c0cc81d64a4477db4dc8be0d478b8746bb013f7273b6460f411d95a5d6efe946", "HTTP_X_FORWARDED_PROTO": "https", "HTTP_X_FORWARDED_FOR": "193.205.157.174", "HTTP_CONNECTION": "close"}: oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '4ae7a803-d7cd-4725-8b2e-65797a64ef94' for key 'items.items_os_id_idx'") 2024-07-25 14:49:11.055 37118 INFO ec2api.api [req-cda08c1a-009c-4c7d-b019-c30bce32baa4 9bd03f63fa9d4beb8de31e6c2f2c8d12 56c3f5c047e74a78a71438c4412e6e13 - - -] 0.811724s 192.168.60.229 POST /services/Cloud DescribeImages 500 [euca2ools/3.4.1 (CPython 2.7.5; Linux 3.10.0-1160.92.1.el7.x86_64; x86_64) requestbuilder/0.7.1 requests/2.19.1] application/x-www-form-urlencoded text/xml 2024-07-25 14:49:11.056 37118 INFO ec2api.wsgi.server [req-cda08c1a-009c-4c7d-b019-c30bce32baa4 9bd03f63fa9d4beb8de31e6c2f2c8d12 56c3f5c047e74a78a71438c4412e6e13 - - -] 193.205.157.174,192.168.60.229 "POST /services/Cloud HTTP/1.1" status: 500 len: 348 time: 0.8139334 2024-07-25 14:49:14.391 37119 INFO ec2api.api [req-417ab595-646d-4468-9464-f19c7392c201 9bd03f63fa9d4beb8de31e6c2f2c8d12 56c3f5c047e74a78a71438c4412e6e13 - - -] 4.822310s 192.168.60.229 POST /services/Cloud DescribeInstances 200 [euca2ools/3.4.1 (CPython 2.7.5; Linux 3.10.0-1062.9.1.el7.x86_64; x86_64) requestbuilder/0.7.1 requests/2.19.1] application/x-www-form-urlencoded text/xml 2024-07-25 14:49:14.392 37119 INFO ec2api.wsgi.server [req-417ab595-646d-4468-9464-f19c7392c201 9bd03f63fa9d4beb8de31e6c2f2c8d12 56c3f5c047e74a78a71438c4412e6e13 - - -] 193.206.210.240,192.168.60.229 "POST /services/Cloud HTTP/1.1" status: 200 len: 11274 time: 4.8234410

1 0

[ironic] Call for maintainers: terraform-provider-ironic
by Dmitry Tantsur 25 Jul '24

25 Jul '24

Hi folks! In OpenShift, we used to use https://github.com/openshift-metal3/terraform-provider-ironic to drive Ironic and Inspector during the initial installation process before the control plane is up. Now that we have moved away from Terraform for this purpose, the project is on minimal support with the intention of completely abandoning it once no stable versions of OpenShift use it to provision the control plane. Are there any parties interested in maintaining terraform-provider-ironic after that point? It supports several features like inspection, BIOS settings, RAID. It does not, to the best of my knowledge, support Keystone authentication and has not been migrated away from Inspector. Dmitry

1 0

[cinder] No meeting today (7/24)
by Jon Bernard 24 Jul '24

24 Jul '24

Hello all, I failed in my effort to secure adequate childcare today, so I need to cancel today's meeting. Hope to see you in the next one. -- Jon

1 0

[horizon][puppet-openstack] Proper Python snippets placement
by Francesco Di Nucci 24 Jul '24

24 Jul '24

Hello, some time ago I found that on EL9 Horizon searches for extra Python snippets in "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.d", while puppet-horizon places dashboards snippets in "/etc/openstack-dashboard/local_settings.d", so these are not loaded. At the same time, "/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.py" is a symlink to "/etc/openstack-dashboard/local_settings". Of the following, what should be adopted as proper approach on a machine? * Having symlinks to /etc/openstack-dashboard for both local_settings.py and local_settings.d * Editing files in /usr/share/openstack-dashboard without having symlinks * Using a symlink for local_settings.py but placing snippets under /usr/share/openstack-dashboard Best regards -- Francesco Di Nucci System Administrator Compute & Networking Service, INFN Naples Email:francesco.dinucci@na.infn.it

4 4

[kolla-ansible]Material choices for ephemerals
by Franck VEDEL 24 Jul '24

24 Jul '24

Hello. I would like your opinion regarding a hardware configuration and my Openstack cluster project. I have the possibility of purchasing 6 servers to build this cluster for students, my current Openstack being too small. I have a proposition from the manufacturer (DELL): 6 servers with characteristics per server: - BOSS-N1 controller card + with 2 M.2 960GB (RAID 1) - 5 x 3.84TB Data Center NVMe Read Intensive AG Drive U2 Gen4 with Carrier - 12 x 64GB RDIMM, 4800MT/s Dual Rank My problem is: BOSS is used to install the system (Ubuntu 22.04 in my case). But, various tests on a small cluster with SSD disks showed that the solution with ephemerals was really the most interesting for what I want to do. Problem: I don't believe you can have the system on the BOSS and the ephemerals on the NVMe disks. So my questions are: 1) am I wrong and ephemerals are possible on NVMe (I use kolla-ansible to deploy, I haven't seen how to do that)? 2) or the solution is to create a Ceph cluster on the NVME of the 6 servers 3) or a better solution exists Thank you in advance if you can help me make these choices. Franck

2 2

Unrestricted AC Access Rules: "You must upgrade keystonemiddleware on all services"
by openstack＠tr.id.au 23 Jul '24

23 Jul '24

Hey there, I'm interested in creating an unrestricted Application Credential (AC) which is still constrained by Access Rules. A use could be allowing admins to use automation to generate read-only ACs for various purposes. The AC for automatically generating other ACs would need to be unrestricted, but should still only have the minimal privileges required to create its children. However, when attempting this with multiple versions of keystone, including 2024.1, I get the following error in the keystone service log: ``` Attempted to use application credential access rules with a middleware that does not understand them. You must upgrade keystonemiddleware on all services that accept application credentials as an authentication method.: NoneType: None ``` A minimal way to reproduce this: 1. Install keystone according to https://docs.openstack.org/keystone/latest/install/keystone-install-ubuntu.…. (I've also tested with keystone running in a virtualenv and in a kolla container.) 2. Generate a "parent" AC using the bootstrap admin: `openstack application credential create --max-width 120 --role admin --access-rules '[ { "path": "/v3/**", "method": "POST", "service": "identity" }, { "path": "/v3/**", "method": "GET", "service": "identity" } ]' --expiration '2024-07-31T23:59:59' --unrestricted test_unrestricted` (This gives expected output, including an AC id and secret.) 3. In another terminal, load the parent AC and then try to generate the child: `openstack application credential create --max-width 120 --role admin --access-rules '[ { "path": "/v3/**", "method": "GET", "service": "identity" } ]' --expiration '2024-07-31T23:59:59' test_read_only`. (This returns `Invalid application credential: Cannot create an application credential with unassigned role` in the client, as well as the earlier error about upgrading middleware in keystone service log.) I've tested that the parent/unrestricted AC *can* generate the child successfully when the parent does *not* have Access Rules. I didn't see anything about these errors or constraints when going through the application credential documentation (eg https://docs.openstack.org/keystone/latest/user/application_credentials.html) or in a web search, or in a search of the openstack-discuss archives. I may raise a bug report/feature request, but before I do that, I just wondered if I have likely overlooked some possible extra configuration or documentation which could explain what I've done wrong? Alternatively, if this were raised as a feature request, would it likely be supported, or would the functionality I'm imagining be too difficult or impossible to implement? Cheers, Tim

1 0

Scheduler and filters
by Gk Gk 23 Jul '24

23 Jul '24

Hi, Can anyone tell me the difference between the schedulers and filters in nova. As per what I grasped, it seems we have the same type of weighers like schedulers. Then what's the difference ? Please explain.. Thanks Y.G

3 2

[OSSA-2024-002] OpenStack Nova: Incomplete file access fix and regression for QCOW2 backing files and VMDK flat descriptors (CVE-2024-40767)
by Jeremy Stanley 23 Jul '24

23 Jul '24

================================================================== OSSA-2024-002: Incomplete file access fix and regression for QCOW2 backing files and VMDK flat descriptors ================================================================== :Date: July 23, 2024 :CVE: CVE-2024-40767 Affects ~~~~~~~ - Nova: <27.4.1, >=28.0.0 <28.2.1, >=29.0.0 <29.1.1 Description ~~~~~~~~~~~ Arnaud Morin (OVH) reported a vulnerability in Nova. By supplying a raw format image which is actually a specially crafted QCOW2 image with a backing file path or VMDK flat image with a descriptor file path, an authenticated user may convince systems to return a copy of the referenced file’s contents from the server resulting in unauthorized access to potentially sensitive data. All Nova deployments are affected. Patches ~~~~~~~ - https://review.opendev.org/924734 (2023.1/antelope) - https://review.opendev.org/924733 (2023.2/bobcat) - https://review.opendev.org/924732 (2024.1/caracal) - https://review.opendev.org/924731 (2024.2/dalmatian) Credits ~~~~~~~ - Arnaud Morin from OVH (CVE-2024-40767) References ~~~~~~~~~~ - https://launchpad.net/bugs/2071734 - http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-40767 Notes ~~~~~ - The patches linked above should apply cleanly to the public state of their respective branches at time of disclosure, and depend on some commits which merged after the `OSSA-2024-001 <https://security.openstack.org/ossa/OSSA-2024-001.html>`_ fixes as well as the final states of the Nova changes linked from that advisory (those did see some minor adjustments before they merged). - The QCOW2 issue is due to an incomplete fix in OSSA-2024-001 affecting systems where the ``use_cow_images`` configuration option is disabled, while the VMDK issue is a regression of the earlier `OSSA-2023-002 <https://security.openstack.org/ossa/OSSA-2023-002.html>`_ vulnerability reintroduced by the new implementation in OSSA-2024-001. Both problems were identified in the final hours before OSSA-2024-001 publication but, due to time constraints, were redacted from that bug and moved to a separate report. - Neither the methods introduced in these patches nor the fixes for OSSA-2024-001 are capable of blocking malicious images which are already resident in Nova's cache. At this time we do not have useful operator guidance for identifying and removing such existing images from the cache but strongly caution, if you do attempt to use the qemu-img tool to find them, to make sure you're using a version of it patched for `QEMU CVE-2024-4467 <https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-4467>`_. -- Jeremy Stanley OpenStack Vulnerability Management Team

1 0

[MAGNUM] Fedora Core OS include GPU drivers
by Oliver Weinmann 23 Jul '24

23 Jul '24

Hi all, I'm running OpenStack 2023.2 (Bobcat) deployed with Kolla-Ansible. Currently I'm struggling to get the NVIDIA drivers to work in a FCOS 35 image. I know FCOS 35 is EOL and outdated, but for a K8s 1.21.11 cluster it is the only version that still works with Magnum. So far I managed to get the GPU Passthrough working for two NVIDIA Tesla T4 GPUs. I have attached the GPUs to a Ubuntu instance and ran geekbench without any issues. Then I tried to install the latest NVIDIA drivers in FCOS 35 by running: sudo rpm-ostree install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$ (rpm -E %fedora).noarch.rpm https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$ (rpm -E %fedora).noarch.rpm systemctl reboot rpm-ostree install --apply-live akmod-nvidia akmods pciutils rpmdevtools xorg-x11-drv-nvidia-cuda It fails on: rpm-ostree install --apply-live akmod-nvidia akmods pciutils rpmdevtools xorg-x11-drv-nvidia-cuda I guess the binaries are no longer hosted? Running the same on FCOS 36 and latest FCOS 40 works fine. Next I have taken new images from the FCOS instances with the NVIDIA driver. openstack server image create --name fedora_coreos_36_nvidia_cuda fedora_coreos_35_latest_no_update --wait This takes ages but anyway I can successfully deploy new instances from the images with my gpu flavor and the GPU is working fine. Again tested with geekbench. But now when adding new K8s worker nodes in a fresh nodegroup on an existing cluster 1.21.11 cluster the nodes are not joining the cluster: openstack coe nodegroup create \ --node-count 1 \ --role worker-gpu \ --flavor gpuflavor \ --image fedora_coreos_36_nvidia_cuda \ $CLUSTER_ID worker-gpu But I know from the past that FCOS 35 was the only version that was working fine with K8s 1.21.11. So I tried with FCOS 35 and this works fine, but unfortunately without the NVIDIA drivers. So maybe the problem is related to this and I need to get FCOS 35 working? Or a newer FCOS version? I'm also not sure if this is the best way of doing this. Maybe there is already a ready FCOS image with nvidia drivers? I can find lots of outdated information on the internet. Generally I tend to use Vexxhost CAPI from now on, which works great. This is really just a one off attempt to have a working NVIDIA T4 GPU in a K8s 1.21.11 cluster. Best Regards, Oliver

1 0

[Neutron] - Floating ip allocation in a specific subnet
by MIchele Delli Veneri 23 Jul '24

23 Jul '24

Dear all, My network setup includes a single external network (ext_vlan) with 2 subnets: 10.x.x.0/24 and 193.x.x.0/24. Is there a way to setup the allocation logic such as that given a project, all floating ips in the project are always allocated on one of the two subnets unless the subnet is explicited (as it may happen through the Openstack CLI or the admin horizon interface)? The ideal outcome would be, given the project test-project and test-project2 , that running openstack floating ip create —project test-project ext_vlan always allocate the floating ip on the 193.x.x subnet while openstack floating ip create —project test-project2 ext_vlan always allocates the floating ip on the 10.x.x. subnet. Best regards, Michele

4 4

Not able to find updated repos
by Ashok Thipe 22 Jul '24

22 Jul '24

Hi, We need to install latest community version of openstack but we are unable to get updated repositories. could anyone please help. With regards, Ashok Thipe

3 4

[cinder] How extend volume if cinder-volume host down
by Nguyễn Hữu Khôi 22 Jul '24

22 Jul '24

Hello guys. Every volume was managed by a specified host. When this host was down or removed. How Can I extend this volume which was managed by this host? I would like to know if we have a solution for this case? Nguyen Huu Khoi

2 4

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2024.2/R-10)
by Goutham Pacha Ravi 22 Jul '24

22 Jul '24

Hello Stackers, As we get closer to the code freeze dates for the upcoming 2024.2 ("Dalmatian") release cycle [1], OpenStack project teams are finalizing features and bug fixes. We're ten weeks away from the official release date, but the feature freeze is squarely at halfway to that milestone. I'd like to share the governance changes that the OpenStack TC has approved in the past week. TC members joined several reviewers to approve the proposal to drop Eventlet from OpenStack [2]. Hervé Beraud (hberaud) and others will update the proposal further based on the feedback that was received through the review process. You may also see updates regarding this across Oslo and OpenStack service projects; and this may become an official community goal if we are able to close out on the feedback and agree upon a structure/timeline. I'd like to congratulate the Skyline team for having graduated from its "Emerging Project" status under the OpenStack services umbrella. In a similar vein, the Watcher project has been bootstrapped with a few new core maintainers and they have decided to adopt a "Distributed Project Maintainers" leadership model. We approved a request from the Freezer team to remain "inactive" as it builds back the contributor momentum necessary to churn out releases of its deliverables again. With the next OpenStack TC and PTL elections around the corner (nominations open on 2024-08-14 [3]), we updated the TC charter to clarify that changes proposed to documentation and governance repositories also qualify towards becoming an "Active Contributor" [4]. === Weekly Meeting === The last meeting of the OpenStack TC was held on IRC on 2024-07-16 [5]. We discussed CI instability and pending patches in Masakari and Adjutant. The project PTLs and maintainers responded to TC requests and have now addressed many concerns. We brainstormed gate issues in the past weeks, and difficulties getting all the patches concerning CVE-2024-32498 merged [6]. A constant theme seems to be test VM SSH connections failing within tests. Although this has been addressed a lot over the years and many changes have made the situation better, intermittent failures still remain. If you're interested in helping the OpenStack QA team with this, please let us know! We also discussed the goal proposal to drop Eventlet [2]. The proposal will require further work to be targeted as a goal. The next meeting of the OpenStack Technical Committee is on 2024-07-23. The agenda is on the OpenStack TC's wiki [7]. Please attend and voice your opinions! === Governance Proposals === ==== Merged ==== Update to include docs and miscellaneous repos for AC status | https://review.opendev.org/c/openstack/governance/+/915021 Move Skyline out from the "Emerging Projects" list | https://review.opendev.org/c/openstack/governance/+/924109 Remove Eventlet from OpenStack | https://review.opendev.org/c/openstack/governance/+/902585 Transition Watcher project to DPL | https://review.opendev.org/c/openstack/governance/+/923583 Inactive state extensions: Freezer | https://review.opendev.org/c/openstack/governance/+/923441 ==== Open for review/discussion ==== Follow up on the affiliation diversity handling | https://review.opendev.org/c/openstack/governance/+/923876 Update criteria for the inactive projects to become active again | https://review.opendev.org/c/openstack/governance/+/921500 Inactive state extensions: Monasca | https://review.opendev.org/c/openstack/governance/+/923919 Retire kuryr-kubernetes and kuryr-tempest-plugin | https://review.opendev.org/c/openstack/governance/+/922507 === Upcoming Events === 2024-08-12: Call for Proposals due for the OpenInfra Days NA [8] 2024-08-14: Nominations open for 2025.1 OpenStack PTL+TC elections 2024-08-24: OpenInfra Days, Vietnam: https://2024.vietopeninfra.org/ 2024-08-29: Dalmatian-3 milestone 2024-09-03: OpenInfra Summit Asia, Suwon I'll leave you with a link to the upbeat OpenInfra Summit welcome video featuring some of our APAC contributors: https://www.youtube.com/watch?v=aGuB1cf3DbY. Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] Release Schedule: https://releases.openstack.org/dalmatian/schedule.html [2] Goal to remove Eventlet from OpenStack: https://governance.openstack.org/tc/goals/proposed/remove-eventlet.html [3] OpenStack Elections: https://governance.openstack.org/election/ [4] Active Contributors: https://governance.openstack.org/tc/reference/charter.html#voters-for-tc-se… [5] TC Meeting IRC Logs, 2024-07-16: https://meetings.opendev.org/meetings/tc/2024/tc.2024-07-16-18.00.log.html [6] Arbitrary file access through custom QCOW2 external data (CVE-2024-32498): https://www.openwall.com/lists/oss-security/2024/07/02/2 [7] TC Meeting, 2024-07-23: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [8] OID North America Call for Proposals: https://docs.google.com/forms/d/1U3TokM5QWTe9_MJvKbOiDc_cNNW_LlFpOL3t-Loib1…

1 0

[neutron] Bug Deputy Report July 15 - July 21
by Miro Tomaska 22 Jul '24

22 Jul '24

Hello All, Here is the bug deputy report for the week of July 15th *Critical:* - https://bugs.launchpad.net/neutron/+bug/2073251 - linuxbridge/ovs jobs fails randomly as Test vms not getting ip from dhcp agent Assigned: Yatin Karel *High:* - https://bugs.launchpad.net/neutron/+bug/2073464 - [DHCP] Race condition when creating a network Assigned: Rodolfo Alonso - https://bugs.launchpad.net/neutron/+bug/2073567 [master][ml2-ovn] Multiple Unexpected exception in notify_loop: neutron_lib.exceptions.PortNotFound Unassigned - https://bugs.launchpad.net/neutron/+bug/2073585 - ubuntu focal jobs fails at dbcounter installation with TypeError: canonicalize_version() Unassigned, but Rodolfo Alonso already proposed a patch to lock down setup tools to v70.2.0 *Medium:* - https://bugs.launchpad.net/neutron/+bug/2073490 - [DHCP] Improve the DHCP agent log Assigned: Rodolfo Alonso - https://bugs.launchpad.net/neutron/+bug/2073309 Neither fdb_removal_limit or mac_binding_removal_limit get set in NB_Global.options when they are configured Assigned: Terry Wilson *Undecided:* - https://bugs.launchpad.net/neutron/+bug/2073326 - ovn] OVSDB Error ' references nonexistent' occurs when updating one router. Most likely will get closed as an unsupported and modified branch is being used. Move to "won't fix"? - https://bugs.launchpad.net/neutron/+bug/2073509 - east and west traffic is not allowed/blocked with neutron fwaas v2 Involves Neutron fwaas, simple reproducing steps but need to confirm if this even a valid bug or is it by design. -- Miro Tomaska Senior Software Engineer

1 0

Kolla deploy
by William Muriithi 22 Jul '24

22 Jul '24

Hello, What could I be doing wrong here? When we attempt openstack deploy, this step fails with the error message below? kolla-ansible -i /etc/kolla/multinode deploy debug3: mux_client_request_alive: done pid = 91641 debug3: mux_client_request_session: session request sent debug1: mux_client_request_session: master session id: 2 debug3: mux_client_read_packet: read header failed: Broken pipe debug2: Received exit status from master 1 The full traceback is: File "/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/modules/kolla_container.py", line 425, in main File "/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_podman_worker.py", line 515, in recreate_or_restart_container self.start_container() File "/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_podman_worker.py", line 538, in start_container self.create_container() File "/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_podman_worker.py", line 501, in create_container container = self.pc.containers.create(**args) File "/usr/local/lib/python3.9/site-packages/podman/domain/containers_create.py", line 354, in create response.raise_for_status(not_found=ImageNotFound) File "/usr/local/lib/python3.9/site-packages/podman/api/client.py", line 75, in raise_for_status raise APIError(cause, response=self._response, explanation=message) fatal: [ostack-40.openstack.perasotech.com] FAILED! => { "changed": true, "invocation": { "module_args": { "action": "recreate_or_restart_container", "api_version": "auto", "auth_email": null, "auth_password": null, "auth_registry": "quay.io", "auth_username": null, "client_timeout": 120, "container_engine": "podman", "detach": true, "dimensions": { "ulimits": {} }, "environment": { "KOLLA_CONFIG_STRATEGY": "COPY_ALWAYS" }, "graceful_timeout": 60, "ignore_missing": false, "image": "quay.io/openstack.kolla/fluentd:master-rocky-9", "labels": {}, "name": "fluentd", "privileged": false, "remove_on_exit": true, "restart_policy": "unless-stopped", "restart_retries": 10, "security_opt": [], "state": "running", "tls_verify": false, "tty": false } }, "msg": "'Traceback (most recent call last):\\n File \"/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/modules/kolla_container.py\", line 425, in main\\n File \"/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_podman_worker.py\", line 515, in recreate_or_restart_container\\n self.start_container()\\n File \"/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_podman_worker.py\", line 538, in start_container\\n self.create_container()\\n File \"/tmp/ansible_kolla_container_payload_xai740dx/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_podman_worker.py\", line 501, in create_container\\n container = self.pc.containers.create(**args)\\n File \"/usr/local/lib/python3.9/site-packages/podman/domain/containers_create.py\", line 354, in create\\n response.raise_for_status(not_found=ImageNotFound)\\n File \"/usr/local/lib/python3.9/site-packages/podman/api/client.py\", line 75, in raise_for_status\\n raise APIError(cause, response=self._response, explanation=message)\\npodman.errors.exceptions.APIError: 500 Server Error: Internal Server Error (statfs /var/log/journal: no such file or directory)\\n'" } Have anyone see the issue before? Regards, William

2 3

[cinder][glance]Errors with patch bug/2059809
by Nguyễn Hữu Khôi 22 Jul '24

22 Jul '24

Hello, i encounter this error when apply patch about bug/2059809: raise exception.ImageConversionNotAllowed( 2024-07-20 03:35:28.067 736 ERROR oslo_messaging.rpc.server cinder.exception.ImageConversionNotAllowed: Image Conversion disallowed for image 7bacbdfb-2ea3-4ed8-90ad-3ec0e274a1d9: Image conversion is disabled. The volume type you have requested requires that the image it is being created from be in 'raw' format, but the image you are using has the disk_format property 'qcow2'. You must use an image with the disk_format property 'raw' to create a volume of this type. I am using Zed version. Could I have some suggestions? Thank you. Nguyen Huu Khoi

5 14

Kolla-Fluentd: not found: manifest unknown
by Ashok Thipe 22 Jul '24

22 Jul '24

Hi, I'm trying to deploy openstack using Kolla-ansible and encountering following error: TASK [common : Ensure fluentd image is present for label check] ****************** fatal: [localhost]: FAILED! => {"changed": true, "msg": "'Traceback (most recent call last):\\n File \"/usr/local/lib/python3.9/site-packages/docker/api/client.py\", line 268, in _raise_for_status\\n response.raise_for_status()\\n File \"/usr/local/lib/python3.9/site-packages/requests/models.py\", line 1021, in raise_for_status\\n raise HTTPError(http_error_msg, response=self)\\nrequests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.46/images/create?tag=master-centos-stream9&fromImage= quay.io%2Fopenstack.kolla%2Ffluentd\\n\\nThe above exception was the direct cause of the following exception:\\n\\nTraceback (most recent call last):\\n File \"/tmp/ansible_kolla_container_payload_7owwfp32/ansible_kolla_container_payload.zip/ansible/modules/kolla_container.py\", line 425, in main\\n File \"/tmp/ansible_kolla_container_payload_7owwfp32/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 522, in ensure_image\\n self.pull_image()\\n File \"/tmp/ansible_kolla_container_payload_7owwfp32/ansible_kolla_container_payload.zip/ansible/module_utils/kolla_docker_worker.py\", line 209, in pull_image\\n json.loads(line.strip().decode(\\'utf-8\\')) for line in self.dc.pull(\\n File \"/usr/local/lib/python3.9/site-packages/docker/api/image.py\", line 429, in pull\\n self._raise_for_status(response)\\n File \"/usr/local/lib/python3.9/site-packages/docker/api/client.py\", line 270, in _raise_for_status\\n raise create_api_error_from_http_exception(e) from e\\n File \"/usr/local/lib/python3.9/site-packages/docker/errors.py\", line 39, in create_api_error_from_http_exception\\n raise cls(e, response=response, explanation=explanation) from e\\ndocker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.46/images/create?tag=master-centos-stream9&fromImage= quay.io%2Fopenstack.kolla%2Ffluentd: Not Found (\"manifest for quay.io/openstack.kolla/fluentd:master-centos-stream9 not found: manifest unknown: manifest unknown\")\\n'"} ansible version used: ansible [core 2.15.12] Kolla-ansible version: kolla-ansible@master OS: CentOS 9 stream Please suggest workaround for the above issue. Thanks and regards, Ashok Thipe

2 1

encountering repos metadata issue during overcloud deployment
by Ashok Thipe 21 Jul '24

21 Jul '24

Hi, I'm trying to deploy openstack (zed version) using tripleo method and encountering "error while loading metadata for repos: centos 8" issue during running overcloud deployment. If there are any possible solutions this issue/bug, please care to share. With regards, Ashok Thipe

3 3

Assistance Needed for Trove Instance Creation
by Thamanna Farhath 20 Jul '24

20 Jul '24

Dear OpenStack Community, I am currently working with OpenStack Kolla Ansible 2023.1 1.setting up Trove instances as per the documentation https://docs.openstack.org/trove/2023.1/admin/building_guest_images.html Despite following the steps outlined, I am encountering some issues. Image Not Found: I followed the steps it downloads the required packages and it creates the image directory. However, the trove image is not present in the specified path $HOME/images. 2.I have also referred to https://satishdotpatel.github.io/trove-dbaas-openstack/ and followed each step, but below issue persists. Error with Messaging and Volume Configuration: While creating a Trove instance, I encountered the following errors: ERROR oslo.messaging._drivers.impl_rabbit [-] Failed to declare consumer for topic 'guestagent.01a5cdd0-df3c-4f61-817f-114b530a1bb3': Queue.declare: (406) PRECONDITION_FAILED - inequivalent arg 'x-queue-type' for queue 'guestagent.01a5cdd0-df3c-4f61-817f-114b530a1bb3' in vhost '/': received the value 'quorum' of type 'longstr' but current is none guest-agent[802]: 2024-07-01 09:24:13.831 802 ERROR trove.guestagent.volume [-] Volume '/dev/vdb' was not formatted.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. Steps Taken: Created and configured images as per the OpenStack documentation. Ensured that all Trove-related configurations are properly set up. Verified RabbitMQ configurations to avoid direct configurations and used Trove’s trove-guestagent configuration for managing RabbitMQ settings. /etc/kolla/config/trove.conf [DEFAULT] management_networks = ee2a2a81-178c-4d79-867b-8baf53ec236d nova_keypair = trove-ssh-key management_security_groups = 451e4b4a-58b4-4e9f-84a8-5bdb21e34cd5 /etc/kolla/config/trove-guestagent.conf [DEFAULT] transport_url = rabbit://openstack:<RABBITMQ_PASSWORD>@192.168.8.14:5672,openstack:<RABBITMQ_PASSWORD>@192.168.8.15:5672,openstack:<RABBITMQ_PASSWORD>@192.168.8.16:5672// control_exchange = trove root_grant = ALL root_grant_option = True debug = False [oslo_messaging_rabbit] heartbeat_in_pthread = false amqp_durable_queues = true rabbit_quorum_queue = true Request for Assistance: Are there any other recommended methods to build the Trove guest image if the documented steps do not work? Is there a known core issue related to the errors I am encountering with oslo.messaging and trove.guestagent.volume? What specific configurations or logs should I check to resolve these errors? Any additional steps or best practices for setting up Trove instances with Kolla Ansible? Thank you in advance for your help. Any insights or suggestions to resolve these issues would be greatly appreciated. Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

2 2

Kernel reported iSCSI connection [ID_SESSION]:0 error (1020 - ISCSI_ERR_TCP_CONN_CLOSE: TCP connection closed) state (1), conn 0 login rejected: initiator error - target not found (02/03)
by elgaplok12＠gmail.com 18 Jul '24

18 Jul '24

hi I have a problem with iscsid. so in the iscsiadm -m session there is still iqn-blablabla:.... when I want to delete or log out, there is an error Logout: iscsiadm --mode node --targetname iqn.2010-10.org.openstack:volume-blablabla --portal ip_address --logout Logging out of session [sid: 45, target: iqn.2010-10.org.openstack:volume-blablabla, portal: ip_address,port] iscsiadm: Could not logout of [sid: 45, target: iqn.2010-10.org.openstack:volume-blablabla, portal: ip_address,port]. iscsiadm: initiator reported error (32 - target likely not connected) iscsiadm: Could not logout of all requested sessions Delete: iscsiadm --mode node --targetname iqn.2010-10.org.openstack:volume-blablabla --portal ip_address --op delete iscsiadm: This command will remove the record [iface: default, target: iqn.2010-10.org.openstack:volume-blablabla, portal: ip_address,port], but a session is using it. Logout session then rerun command to remove record. iscsiadm: Could not execute operation on all records: session exists Sessions: tcp: [160] ip_address:3260,1 iqn.2010-10.org.openstack:volume-blablabla (non-flash) tcp: [161] ip_address:3260,1 iqn.2010-10.org.openstack:volume-blablabla (non-flash) tcp: [162] ip_address:3260,1 iqn.2010-10.org.openstack:volume-blablabla (non-flash) tcp: [163] ip_address:3260,1 iqn.2010-10.org.openstack:volume-blablabla (non-flash) tcp: [164] ip_address:3260,1 iqn.2010-10.org.openstack:volume-blablabla(non-flash) tcp: [44] ip_address:3260,1 iqn.2010-10.org.openstack:volume-blablabla (non-flash) tcp: [45] ip_address:3260,1 iqn.2010-10.org.openstack:volume-blablabla (non-flash) NB: blablabla is uuid volume

1 0

Re: Trove question
by Hirotaka Wakabayashi 18 Jul '24

18 Jul '24

Hello Ксения, Thanks for the contact! Trove team provides the following document for admins to upgrade your Trove service. https://docs.openstack.org/trove/latest/admin/upgrade.html Reading the docs above, you can easily try upgrading Trove for test by using `devstack`. https://docs.openstack.org/trove/latest/install/install-devstack.html If you have any questions, you can contact openstack-discuss@. https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… Best Regards,hirotaka On Wednesday, July 17, 2024 at 09:06:57 PM GMT+9, Ксения Петрова <xeniya.petrova(a)ps.kz> wrote: Hello! We're about to launch our DBaaS on trove in our cloud hosting platform. But we're uncertain about what would be the best way on applying a new version of a trove guest image to our customers databases. So what is the best practices for the process of upgrading guest image? Or nobody of cloud providers cares and upgrading is the completely client's concern? Real-world examples with companies who already sell DBaaS would be the best. Thank you in advance! Best regards, Xeniya Petrova PS Cloud Services

1 0

[Glance] Cancelling Weekly Meeting (July 18)
by Pranali Deore 18 Jul '24

18 Jul '24

Hello, I'm not feeling well, won't be able to chair today's meeting, so cancelling the same. See you next week! Thanks, Pranali Deore

1 0

[Diskimage-builder]Attempting to build opensuse failed.
by Junya Noguchi (Fujitsu) 18 Jul '24

18 Jul '24

To whom it may concern An attempt to build an OpenSUSE image using Diskimage-builder failed. It seems that dracut failed during initial image creation because /dev wasn't mounted. Is there a way to solve this problem? [xxxxxxx@xxxxxxxxxxx ~]$ cat /etc/os-release | grep PRETTY_NAME PRETTY_NAME="Rocky Linux 9.3 (Blue Onyx)" [xxxxxxx@xxxxxxxxxxx ~]$ [xxxxxxx@xxxxxxxxxxx ~]$ disk-image-create opensuse baremetal dhcp-all-interfaces -a amd64 grub2 -o opensuse_test 2024-07-05 02:06:02.796 | diskimage-builder version 3.32.0 2024-07-05 02:06:02.797 | Building elements: base opensuse baremetal dhcp-all-interfaces grub2 2024-07-05 02:06:02.948 | Expanded element dependencies to: package-installs base baremetal zypper cache-url install-bin sysprep dib-init-system install-types pkg-map dhcp-all-interfaces grub2 opensuse 2024-07-05 02:06:03.037 | Building in /tmp/dib_build.pTiDO08e 2024-07-05 02:06:03.041 | Searching elements for block-device[-amd64].yaml ... 2024-07-05 02:06:03.047 | ... done 2024-07-05 02:06:19.736 | Running hooks from /tmp/dib_build.pTiDO08e/hooks/extra-data.d 2024-07-05 02:07:17.361 | dracut: Stored kernel commandline: 2024-07-05 02:07:17.362 | dracut: No dracut internal kernel commandline stored in the initramfs 2024-07-05 02:07:17.363 | dracut: *** Creating image file '/boot/initrd-5.3.18-150300.59.106-default' *** 2024-07-05 02:07:24.019 | dracut: *** Creating initramfs image file '/boot/initrd-5.3.18-150300.59.106-default' done *** 2024-07-05 02:07:24.127 | update-bootloader: 2024-07-05 02:07:24 <3> update-bootloader-4241 run_command.294: '/usr/lib/bootloader/grub2-efi/config' failed with exit code 1, output: 2024-07-05 02:07:24.129 | <<<<<<<<<<<<<<<< 2024-07-05 02:07:24.129 | + /usr/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-05 02:07:24.129 | /usr/sbin/grub2-probe: error: cannot find a device for / (is /dev mounted?). 2024-07-05 02:07:24.129 | >>>>>>>>>>>>>>>> 2024-07-05 02:07:24.129 | Updating bootloader failed 2024-07-05 02:07:24.204 | bootloader_entry: 2024-07-05 02:07:24 <3> bootloader_entry-1651 run_command.294: '/usr/lib/bootloader/grub2-efi/config' failed with exit code 1, output: 2024-07-05 02:07:24.218 | <<<<<<<<<<<<<<<< 2024-07-05 02:07:24.218 | + /usr/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-05 02:07:24.218 | /usr/sbin/grub2-probe: error: cannot find a device for / (is /dev mounted?). 2024-07-05 02:07:24.218 | >>>>>>>>>>>>>>>> 2024-07-05 02:07:24.218 | Failed to get root password hash 2024-07-05 02:07:24.218 | Failed to import /etc/uefi/certs/40905999.crt 2024-07-05 02:07:24.220 | warning: %post(kernel-default-5.3.18-150300.59.106.1.x86_64) scriptlet failed, exit status 255 2024-07-05 02:07:24.221 | done] 2024-07-05 02:07:27.003 | Executing %posttrans script 'ca-certificates-mozilla-2.56-150200.24.1.noarch.rpm' [... 2024-07-05 02:07:27.003 | update-bootloader: 2024-07-05 02:07:27 <3> update-bootloader-1757 run_command.294: '/usr/lib/bootloader/grub2-efi/install' failed with exit code 1, output: 2024-07-05 02:07:27.003 | <<<<<<<<<<<<<<<< 2024-07-05 02:07:27.003 | target = x86_64-efi 2024-07-05 02:07:27.003 | + /usr/sbin/shim-install --config-file=/boot/grub2/grub.cfg --no-nvram --removable 2024-07-05 02:07:27.003 | /usr/sbin/grub2-probe: error: cannot find a device for /boot/efi (is /dev mounted?). 2024-07-05 02:07:27.003 | >>>>>>>>>>>>>>>> 2024-07-05 02:07:27.040 | update-bootloader: 2024-07-05 02:07:27 <3> update-bootloader-1757 run_command.294: '/usr/lib/bootloader/grub2-efi/config' failed with exit code 1, output: 2024-07-05 02:07:27.040 | <<<<<<<<<<<<<<<< 2024-07-05 02:07:27.040 | + /usr/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-05 02:07:27.040 | /usr/sbin/grub2-probe: error: cannot find a device for / (is /dev mounted?). 2024-07-05 02:07:27.040 | >>>>>>>>>>>>>>>> 2024-07-05 02:07:27.089 | . 2024-07-05 02:07:27.089 | Creating initrd: /boot/initrd-5.3.18-150300.59.106-default 2024-07-05 02:07:27.304 | dracut: Executing: /usr/bin/dracut --logfile /var/log/YaST2/mkinitrd.log --force /boot/initrd-5.3.18-150300.59.106-default 5.3.18-150300.59.106-default 2024-07-05 02:07:27.370 | dracut: Turning off host-only mode: '/run' is not mounted! 2024-07-05 02:07:54.459 | dracut: Stored kernel commandline: 2024-07-05 02:07:54.460 | dracut: No dracut internal kernel commandline stored in the initramfs 2024-07-05 02:07:54.461 | dracut: *** Creating image file '/boot/initrd-5.3.18-150300.59.138-default' *** 2024-07-05 02:08:01.205 | dracut: *** Creating initramfs image file '/boot/initrd-5.3.18-150300.59.138-default' done *** 2024-07-05 02:08:01.308 | update-bootloader: 2024-07-05 02:08:01 <3> update-bootloader-8343 run_command.294: '/usr/lib/bootloader/grub2-efi/config' failed with exit code 1, output: 2024-07-05 02:08:01.309 | <<<<<<<<<<<<<<<< 2024-07-05 02:08:01.309 | + /usr/sbin/grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-05 02:08:01.309 | /usr/sbin/grub2-probe: error: cannot find a device for / (is /dev mounted?). 2024-07-05 02:08:01.309 | >>>>>>>>>>>>>>>> 2024-07-05 02:08:01.310 | Updating bootloader failed 2024-07-05 02:08:01.311 | warning: %posttrans(device-mapper-2.03.05_1.02.163-150200.8.49.1.x86_64.rpm) scriptlet failed, exit status 1 2024-07-05 02:08:11.000 | .....done] 2024-07-05 02:08:12.494 | There are running programs which still use files and libraries deleted or updated by recent upgrades. They should be restarted to benefit from the latest updates. Run 'zypper ps -s' to list these programs. 2024-07-05 02:08:12.494 | 2024-07-05 02:08:12.494 | Since the last system boot core libraries or services have been updated. 2024-07-05 02:08:12.494 | Reboot is suggested to ensure that your system benefits from these updates. 2024-07-05 02:08:12.541 | Unmount /tmp/dib_build.pTiDO08e/mnt/var/cache/zypp 2024-07-05 02:08:12.582 | Unmount /tmp/dib_build.pTiDO08e/mnt/tmp/in_target.d 2024-07-05 02:08:12.603 | Unmount /tmp/dib_build.pTiDO08e/mnt/sys 2024-07-05 02:08:12.623 | Unmount /tmp/dib_build.pTiDO08e/mnt/proc 2024-07-05 02:08:12.667 | Unmount /tmp/dib_build.pTiDO08e/mnt/dev/pts 2024-07-05 02:08:12.688 | Unmount /tmp/dib_build.pTiDO08e/mnt/dev 2024-07-05 02:08:12.882 | INFO diskimage_builder.block_device.blockdevice [-] State already cleaned - no way to do anything here [xxxxxxx@xxxxxxxxxxx ~]$

2 2

Inconsistent server rebuild behavior
by Andrew Bogott 17 Jul '24

17 Jul '24

I'm puzzled by nova instance rebuilds. When I run 'openstack server rebuild --image <whatever>' on a new test VM, it does what I expect: - throws away the old server image - boots up a brand new server using the requested image with the old user/vendor data - attaches new server to the old network port and cinder volumes. That's just what I want! Alas, when I try this with the actual old VMs in need of update, usually it does a totally different thing: - reboots the server - changes the nova db record to reflect the requested VM (so that the server /looks/ like it was rebuilt in 'openstack server show') - That's it! It absolutely does not rebuild the actual server disk image. So, for instance, any files that were present in '/' of the old server are still present post-rebuild. lsb_release still shows the OS of the original base image, disagreeing with the the OS version displayed by 'openstack server show'. I've been reading logfiles for days but I don't see nova ever complain (in either case) that it's doing something unexpected and in both cases the servers wind up in state 'Active'. Am I misunderstanding what this is supposed to do? If not, does anyone have thoughts about what might be going wrong? I assume nova is encountering a roadbump during the rebuild and doing a graceful revert, but if so it sure isn't telling me. Bobcat, kvm, VM images are hosted on ceph/rbd.

2 3

[rally] possibility to skip admin=True validation
by Benedikt Trefzer 17 Jul '24

17 Jul '24

Hi list I'd like to run rally against barbican. Since all barbican scenarios are tagged with @validation.add("required_platform", platform="openstack", admin=True) we cannot use them since our rally configuration does not have an admin user. (actually test itself should succeed, but the validation fails). Question: Is it somehow possible to skip (or unset admin=True requirement) for validation in the task configuration or do I have to rewrite the scenario ? Cheers Benedikt

1 0

Caracal deliverables for bug 2065145
by Francesco Di Nucci 17 Jul '24

17 Jul '24

Hello, as the bug 2065145 <https://bugs.launchpad.net/neutron/+bug/2065145> fix has been merged into stable/2024.1 branch, are there plans to release a new version of neutron packages that include the fix? If not, should it be advisable to create a proposal for a bugfix release? Thanks -- Francesco Di Nucci System Administrator Compute & Networking Service, INFN Naples Email:francesco.dinucci@na.infn.it

2 2

General network questions installing OpenStack single node guided
by klaus.becker＠webmen.de 16 Jul '24

16 Jul '24

Dear OpenStackers, I started my 2nd try installing OpenStack single node guided using this link: https://microstack.run/docs/single-node-guided At the point sunbeam cluster bootstrap I have to decide network settings: sudo sunbeam cluster bootstrap Management networks shared by hosts (CIDRs, separated by comma) (192.168.196.0/24): The example given is this: Management networks shared by hosts (CIDRs, separated by comma) (10.20.20.0/24): 172.16.1.0/24 MetalLB address allocation range (supports multiple ranges, comma separated) (10.20.20.10-10.20.20.20): 172.16.1.201-172.16.1.220 Unfortunately there is no information about the LAN the server is running in. What does "Management networks shared by hosts“ mean? I have ONE host (OpenStack) What should be shared? To whom ? I do not understand the clue behind that. The MetalLB addresses are for the VMs, right? So they must be in an address range of my LAN During my first installation (also single node, but not guided) I was not asked for different networks. Finally I was unable to reach Horizon from my LAN. I tried many efforts with routing (nft), but I did not work. Everything seemed live inside my host, but unusable, because I am running Ubuntu Server 24.02 in a Rack using ssh - so: no GUI. I hoped to get another chance with this „guided" installation. What I hoped to get: 1. A dedicated network inside OpenStack where all components are communicating with each other without interfering with my LAN. 2. Access to the Dashboard (Horizon) from my LAN 3. Access to VMs I am using inside OpenStack from my LAN Because I am working over ssh, the host uses an IP number from my LAN (192.168.196.11) There are about 20 IPs free in my LAN The question "Management networks shared by hosts (CIDRs, separated by comma) (192.168.196.0/24):“ shows the whole network (/24) as given. I do not want to run into IP conflicts with other PCs, routers etc. in my LAN. So could someone PLEASE give me a bit more background to this network topic? Thanx a lot! Klaus

2 3

[RALLY 4.1.0] Verifier reconfigure fails when in "no admin" mode
by Jean-François TALTAVULL 16 Jul '24

16 Jul '24

Hi All, I'm using a Rally (4.1.0) deployment with existing users and no admin credentials. So the command "rally env show" outputs "admin": null in the json and it's ok. But the problem is that the command "rally verify configure-verifier -reconfigure" fails with: "rally_openstack.common.credential.OpenStackCredential() argument after ** must be a mapping, not NoneType" I've been struggling with this error for a number of hours with no results :-( Any idea ? I looked for a known bug in launchpad but found nothing. Thanks, Jean-Francois

2 4

Re: [neutron] ovs: multiple rows in Manager patch
by Brian Haley 16 Jul '24

16 Jul '24

Just sending response to the list in case anyone else has the same issue. On 6/17/24 1:09 PM, stream_unpicked390(a)simplelogin.com wrote: > Hi, > > On 6/13/24 21:38, Brian Haley - haleyb.dev(a)gmail.com wrote: >> Hi, >> >> On 6/13/24 4:53 AM, stream_unpicked390(a)simplelogin.com wrote: >>> Hi Brian, >>> On 6/12/24 23:04, Brian Haley - haleyb.dev(a)gmail.com wrote: >>>> Hi, >>>> >>>> On 6/12/24 12:43 PM, stream_unpicked390(a)simplelogin.com wrote: >>>>> Hi everyone! >>>>> >>>>> this is my first time on a mailing list, so please excuse any >>>>> "obvious" mistakes and feel free to correct me :) >>>>> >>>>> I'm currently setting up a minimal OpenStack system manually on one >>>>> server. The setup is nearly complete but I'm running into issues >>>>> with neutron. After fully completing the neutron setup as mentioned >>>>> in the docs, the neutron-ovs-service refuses to start, showing the >>>>> error message >>>> >>>> Sorry, but I have to ask which "docs" are you using? >>> >>> thanks for your quick answer! I followed the OpenStack install guide >>> under https://docs.openstack.org/2024.1/install/. For neutron >>> specifically, I used the document linked under >>> https://docs.openstack.org/neutron/2024.1/install/install-ubuntu.html. >>> >>> As I mentioned, I'm currently setting up everything on just one >>> server, the documentations scenario describes two servers. This is >>> the only intentional diversion I made from the docs. I've been trying >>> to setup a self-service network. >>> >>>> I don't see ptcp used in any of the examples here for ovsdb_connection: >>>> >>>> https://docs.openstack.org/neutron/latest/install/ >>> >>> I couldn't find any example of the ovsdb_connection and I don't >>> remember modifying any value like that either. The problem existed >>> "right out of the box". Can you point me to the part of the linked >>> documentation where the ovsdb_connection config is discussed? >> >> Instead of trying to go through the details of anything that might be >> wrong, if you are just trying to get a single-node Openstack running >> have you tried using devstack [0]? It would be much simpler if this is >> just for testing purposes. > This is a production setup so devstack was out of the question. Because > of our very tight constraints, we first have to set up a single node, > before being able to add additional servers. Definitely not ideal.. Your > hint about ovsdb_connection helped me out! While I couldn't find any > setting with the name ovsdb_connection already set to an, apparently > wrong, string, I set ovsdb_connection to the value mentioned by you. > This probably overwrote something I couldn't find, because after > restarting neutron, my issues were gone! Thanks again for looking into > my issue and guiding me on the right path! Glad it worked out, thanks for responding. -Brian >> [0] https://github.com/openstack-dev/devstack.git >> >>>>> multiple rows in Manager patch "ptcp:6640:127.0.0.0" >>>> >>>> Two things look wrong here, both the IP address and port. Typically >>>> this string would be "ptcp:127.0.0.1:6640" or "ptcp:0.0.0.0:6640" in >>>> ml2_conf.ini. >>> >>> Thanks! I will try to take a look at the aforementioned config as >>> soon as possible and report back! >>> >>>> >>>> -Brian >>>> >>>>> I dumped the ovs database which only shows one row in the Manager >>>>> table. After many days of debugging and googleing I'm out of ideas >>>>> how this could be solved.. >>>>> I already posted this issue on serverfault >>>>> (https://serverfault.com/questions/1160418/openstack-ovs-multiple-rows-in-ma…) only after that post finding this mailing list! >>>>> >>>>> Best regards >>>>> Talooola >>>>> >>>> >>

2 1

[swift] Weird log corruption or something more serious?
by Sam Morrison 15 Jul '24

15 Jul '24

Hi, We are debugging a weird issue with our swift cluster (currently running 2.29.2) We got a report from a user that they were getting upload failures and then sent us this from their logs: 2024-06-25 23:30:06,829 | swiftclient.service | ERROR | Object PUT failed: https://swift-host/v1/PROJECT_ID_A/prod_result/195e5e8e-334a-11ef-ba70-0a58… 401 Unauthorized [first 60 chars of response] b'<html><h1>Unauthorized</h1><p>This server could not verify t' (txn: txab25fcfcf3354bbbaf105-00667b52fe) Searching our logs for that transaction ID I found: 203.101.227.241 127.0.0.1 25/Jun/2024/23/30/06 PUT /v1/AUTH_PROJECT_ID_B/nexus-migration/nexus-data/blobs/default/content/vol-27/chap-25/6bc30bae-4f5e-40a8-bc63-096679af1a58.bytes HTTP/1.0 401 - rclone/v1.59.0%2Cpython-swiftclient-4.2.0 gAAAAABme1L7tlfE... - 131 - txab25fcfcf3354bbbaf105-00667b52fe - 0.0440 - - 1719358206.765954018 1719358206.809994936 0 Note this is a completely different project ID from our side, timestamp is correct too. We also have this: ERROR WSGI: code 400, message Bad request syntax ('\x89PNG') (txn: txab25fcfcf3354bbbaf105-00667b52fe) (client_ip: XX.XX.XX.XX) Upon further analysis we see this quite a lot, mostly from attempted hackers but lots from real users and lots seem to be related to rclone. Eg. Jun 27 08:24:46 swift-host proxy-server[70043]: ERROR WSGI: code 400, message Bad request syntax ('PUT /v1/AUTH_PROJECT_ID_B/nexus-migration/nexus-data/elasticsearch/nexus/nodes/0/indices/42b6a6a76dae683a647546ac02a6883108ac7df5/0/index/segments_4PUT /v1/AUTH_PROJECT_ID_B/nexus-migration/nexus-data/elasticsearch/nexus/nodes/0/indices/4db62ff1327dfe236d85b1dd1a6a9e8139e7e4cf/0/index/_67t.cfs HTTP/1.1') (txn: txd57a3d00bb1942c6a390e-00667c952a) (client_ip: XX.XX.XX.XX) Anyone know what is going on here? Thanks, Sam

1 1

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2024.2/R-11)
by Goutham Pacha Ravi 15 Jul '24

15 Jul '24

Hello Stackers, The 2024.2 release train is chugging along, and its final stop is 11 weeks away [1]. In the past week, the OpenStack Technical Committee updated its charter to enforce its affiliation diversity requirements during TC elections [2]. If an election results in over 50% of the TC being affiliated with the same organization, we've adopted a rule to vacate the least favored candidate elected from that organization and accommodate a non-elected candidate with a different affiliation, if one exists. We're also ideating on how to handle situations where there are no candidates from other organizations [3]. Your views are greatly appreciated. Nominations for the TC (and PTL) elections for 2025.2 open on 2024-08-14. You'll begin to receive election-specific emails soon. In other news, Jeremy Stanley (fungi) shared the OpenInfra Foundation's Diversity and Inclusion Working Group updates on this list recently [4]. I encourage you to participate in the survey that the group has put together [5]. It's open year-round, and the feedback gathered will be incorporated continuously. === Weekly Meeting === Last week's Technical Committee meeting was well attended, and the meeting was logged to eavesdrop [6]. We discussed project activity and gate issues in Monasca, Freezer, Skyline, Tacker, Masakari, Zaqar, and Adjutant. We're very glad to see the project teams' responsiveness to the concerns raised. Monasca and Freezer teams have expressed their desire to remain "inactive" through this release cycle. A proposal has been made to move Skyline from the list of "Emerging" projects. The TC will review these proposals in the coming week. The retirement of the "kuryr-kubernetes" repository was held off temporarily to address dependencies within Tacker. The next Technical Committee meeting is tomorrow, 2024-07-16! The meeting will be hosted on OFTC's #openstack-tc channel, and the agenda has been posted to the Meetings Wiki Page [7]. Please consider joining us! === Governance Proposals === ==== Merged ==== Adding a rule to manage the affiliation diversity requirement in the TC | https://review.opendev.org/c/openstack/governance/+/922512 ==== Open for review/discussion ==== Follow up on the affiliation diversity handling | https://review.opendev.org/c/openstack/governance/+/923876 Move Skyline out from the "Emerging Projects" list | https://review.opendev.org/c/openstack/governance/+/924109 Transition Watcher project to DPL | https://review.opendev.org/c/openstack/governance/+/923583 Inactive state extensions: Monasca | https://review.opendev.org/c/openstack/governance/+/923919 Update to include docs and miscellaneous repos for AC status | https://review.opendev.org/c/openstack/governance/+/915021 Update criteria for the inactive projects to become active again | https://review.opendev.org/c/openstack/governance/+/921500 Inactive state extensions: Freezer | https://review.opendev.org/c/openstack/governance/+/923441 Retire kuryr-kubernetes and kuryr-tempest-plugin | https://review.opendev.org/c/openstack/governance/+/922507 Remove Eventlet From OpenStack | https://review.opendev.org/c/openstack/governance/+/902585 The OpenStack Technical Committee serves you! We're always willing to chat with you and would love feedback: === How to contact the TC === You can reach out to the TC in multiple ways: 1. Email: you can send the email with the tag [tc] on this email list. 2. Ping us using the 'tc-members' keyword on the #openstack-tc IRC channel 3. Join us at our weekly meeting: The Technical Committee meets every week on Tuesdays at 1800 UTC [7] === Upcoming Events === 2024-08-14: Nominations open for 2025.1 OpenStack PTL+TC elections 2024-08-24: OpenInfra Days, Vietnam: https://2024.vietopeninfra.org/ 2024-08-29: Dalmatian-3 milestone Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] Release Schedule: https://releases.openstack.org/dalmatian/schedule.html [2] TC Diversity Requirement: https://governance.openstack.org/tc/reference/charter.html#tc-diversity-req… [3] Following up on affiliation diversity: https://review.opendev.org/c/openstack/governance/+/923876 [4] OpenInfra D&I initiatives and survey: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [5] D&I survey: https://openinfrafoundation.formstack.com/forms/openinfra_diversity_inclusi… [6] TC Meeting IRC Logs, 2024-07-09: https://meetings.opendev.org/meetings/tc/2024/tc.2024-07-09-18.00.log.html [7] TC Meeting, 2024-07-16: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

Restricting User Deletion to Project Owners
by Thamanna Farhath 15 Jul '24

15 Jul '24

Hi OpenStack Community, I am working on a policy configuration to ensure that only owners of a project can delete users within their own project. Below is my current setup and the policy rules I have defined. Current Setup OpenStack deployment using Kolla Ansible 2023.1. Created an owner role with permissions to create, delete, and manage users and projects. But it allows you to delete users from other projects as well. So, I need to achieve the below scenario. Policy Rules yaml "admin_required": "role:admin" "admin_or_owner": "rule:admin_required or (role:owner and project_id:%(target.user.project_id)s)" "identity:delete_user": "rule:admin_or_owner" Scenario User demo with owner role of project X creates users newuser1 and newuser2 assigned to project X. User tester with owner role of project Y creates users myuser1 and myuser2 assigned to project Y. With the above configuration, I aim to ensure that: User demo can only delete newuser1 and newuser2 within project X. User tester can only delete myuser1 and myuser2 within project Y. Thank you for your support. Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

2 3

[keystone] Keystone Single Sign-On for REST API Access
by leon.amtmann＠ibm.com 15 Jul '24

15 Jul '24

Hello OpenStack Community, and greetings especially to the Keystone experts among you. I'm currently co-administering an OpenStack 17.1.3 cluster with a few compute nodes and three controllers. For a better user experience, we've enabled Single Sign-On using OIDC for Horizon, which is working fabulously. To do that, the [mod_auth_openidc module](https://github.com/OpenIDC/mod_auth_openidc) is configured for httpd on the controller nodes. However, we're now facing a novel scenario for us. Basically, we want to interact with the OpenStack REST APIs - but not using a usual token request to `/v3/auth/tokens`, but using Single Sign-On. We've looked at the authentication request that Horizon makes to Keystone after successful authentication with the SSO provider. As far as we can tell: 1. After successful authentication with the SSO identity provider, Keystone redirects back to the redirect URI with a grant_id, id_token and state in the URL fragments (after # in the URL). 2. The redirect URI is `/v3/auth/OS-FEDERATION/websso/openid`, which contains HTML that will read the fragments and POST them as form data to somewhere. 3. Those fragments get POSTed to `/v3/auth/OS-FEDERATION/websso/openid`. 4. The response from Keystone is an HTML document containing an auth token, rigged to get POSTed to Horizon at `/dashboard/auth/websso/` (presumably to start a fresh session). So far, so good. The [Keystone federation documentation](https://docs.openstack.org/keystone/latest/admin/federation/… does show the [Horizon flow](https://docs.openstack.org/keystone/latest/admin/federation/introduct… in action. I can replicate it in part using cURL, by doing the same requests to the same endpoints that my browser does. However, I do not understand or see a request to `/v3/auth/OS-FEDERATION/websso/mapped` where Keystone actually issues a token. Instead, after the `id_token`, `grant_id`, etc. are POSTed to `/v3/auth/OS-FEDERATION/websso/openid`, Keystone responds with an HTML document containing the token (see step 4. in my earlier list). I've only seen this working for Horizon in an actual browser, though. Am I misunderstanding the documentation? Right above that in the documentation, there is a section about an unrelated [OpenID Connect Authentication Flow](https://docs.openstack.org/keystone/latest/admin/federation/introduct…. It contains a graphic in which the first request made is a GET request to `/secure`. A direct GET request yields a `404`, so I assume this is a placeholder for any protected resource. However, if I request any protected resource without a valid token, I do not receive a 302 as the graphic would seem to suggest. Is this a behavior that would need to be configured, first? In short: How does one end up with an unscoped token when trying to SSO against Keystone from something that is not Horizon? I'm just wondering if I am on the correct path in trying to modify the `10-keystone.wsgi` in `/etc/httpd/conf.d/` to use [multiple providers](https://github.com/OpenIDC/mod_auth_openidc/wiki/Multiple-Provid…. If, after starting the SSO process, I end up with a `grant_id` and an `id_token`, where do I post this to to get a Keystone token in return? After going through the documentation, I am still at a loss. I would appreciate any help. Thank you for your time.

4 3

[tacker] Cancelling IRC meeting
by Yasufumi Ogawa 15 Jul '24

15 Jul '24

Hi team, I'd like to skip IRC meeting this week since I'll be out of office. Thanks, Yasufumi

1 0

[barbican] Weekly meeting canceled Jul 15th
by Grzegorz Grasza 15 Jul '24

15 Jul '24

Hi all, I'll be traveling during our usual meeting hour, so I'm cancelling it this week. / Greg

1 0

[neutron] Bug Deputy Report July 8 - July 14
by Yatin Karel 14 Jul '24

14 Jul '24

Hi Neutrinos, Below is the bug deputy report for last week:- High ------ - https://bugs.launchpad.net/neutron/+bug/2072754 - Restarting octavia breaks IPv4 Load Balancers with health checks Assigned to Pavlo Shchelokovskyy - https://bugs.launchpad.net/neutron/+bug/2072567 - neutron-ovn-tempest-postgres-full job fails as oslo_db.exception.DBError: (psycopg2.errors.GroupingError) column "standardattributes_1.id" must appear in the GROUP BY clause or be used in an aggregate function Assigned to Rodolfo Alonso Medium ---------- - https://bugs.launchpad.net/neutron/+bug/2072978 - Show some error in logs when failing to load nb connection certificate Assigned to Chris Buggy Wishlist ---------- - https://bugs.launchpad.net/neutron/+bug/2072505 - [RFE] "External network not reachable from subnet" could take static routes into account Unassigned -- Thanks and Regards Yatin Karel

1 0

Making stestr scheduler smarter with a lot of threads
by Thomas Goirand 14 Jul '24

14 Jul '24

Hi, Currently, it looks like stestr calculate which thread should run which test at the beginning, by calculating some partition, and then launch all tests at once. With a lot of threads, the result is, at the end, only a few cores are in use, with all the others being idle. Would there be a way to have a pool of available threads instead, and have stestr to give threads something to eat when they are available, instead of the current way? How much work would that be? Cheers, Thomas Goirand (zigo)

4 3

Tenacity 8.4.2 breaking at least 4 OpenStack projects
by Thomas Goirand 14 Jul '24

14 Jul '24

Hi, It looks like Tenacity 8.4.2 broke a number of projects: - heat - ironic - octavia - python-octavia-lib I filed this for Octavia: https://bugs.launchpad.net/octavia/+bug/2073044 though I should probably do the same for the other projects. Unfortunately, someone uploaded Tenacity 8.4.2 to Debian unstable, so the packages are currently broken. :/ I'll see if the upload can be temporarily reverted, but the more long term solution is to fix the above projects and up the global requirements. Cheers, Thomas Goirand (zigo)

2 2

Subject: Assistance Needed: Restricting Owner Role to Manage Users and Projects Within Their Own Project
by Thamanna Farhath 13 Jul '24

13 Jul '24

Hi OpenStack Community, I am currently using openstack kolla ansible 2023.1, now facing an issue with the role-based access control in my OpenStack deployment.Could you please assist me in achieving my requirements for proper role configuration and access management? I have created a custom role named "owner" with permissions to create, delete, and manage users and projects, and also to assign users to projects. However, I want to restrict the "owner" role to manage only the users and projects within their own project, and not across other projects. Here is the policy file configuration(/etc/kolla/keystone/policy.yaml) I am using: yaml # Custom rule to allow users with the 'owner' role to manage users and projects identity:create_project: "role:owner or role:admin" admin_or_owner: "rule:admin_required or project_id:%(target.project.id)s" identity:delete_project: "rule:admin_or_owner" identity:delete_user: "rule:admin_or_owner" identity:create_user: "role:owner or role:admin" identity:get_user: "role:owner or role:admin or role:member or role:user" identity:list_users: "role:owner or role:admin" identity:update_user: "role:owner or role:admin" identity:create_grant: "role:owner or role:admin" identity:list_grants: "role:owner or role:admin" I have the following questions: Is the above policy configuration correct for achieving the desired restrictions? Are there any other possibilities or best practices to ensure that the "owner" role can only delete users and projects within their own project? How can I improve or adjust the policy rules to meet these requirements effectively? Thank you in advance for your assistance. ---- On Fri, 12 Jul 2024 10:49:22 +0530 Artem Goncharov <artem.goncharov(a)gmail.com> wrote --- Hi, 1) in Keystone we are currently working on implementing domain manager persona that allows certain person on a "customer" side to do project and user operations. This includes explicit listing of users in a domain 2) this is conceptually wrong to talk about users in a project, since users are owned by domains and can get access to multiple projects within the domain (and with some tricks to other domains). So you are partially right that role assignments is the way of figuring out access of users to projects. One more challenge though is that such access can be granted using user groups. In this case you actually should also resolve groups access to the desired project. Hope this helps, Artem ---- typed from mobile, auto-correct typos assumed ---- On Thu, Jul 11, 2024, 20:34 Thamanna Farhath <mailto:thamanna.f@zybisys.com> wrote: Hello openstack community, I am currently working on an OpenStack kolla ansible 2023.1. I have created a custom role " owner " Objective: Users with the owner role can only list users within their project. Avoid additional API calls for filtering users by project. Current Challenge: Using direct API calls, the list users API currently returns all users without any project-specific filtering. While additional API calls to the role_assignments endpoint can achieve the desired filtering, this approach is not efficient and doesn't restrict access at the API level. I am considering defining custom Keystone policies to achieve this restriction. However, I am unsure how to properly configure these policies to enforce project-specific access control effectively. Thanks in advance Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email." Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

1 0

[keystone] Weekly Meeting Canceled 17-Jul-2024
by Dave Wilde 12 Jul '24

12 Jul '24

Hello, I'm canceling the Keystone weekly meeting for 17-Jul-2024 as I'll be on PTO. Please reach out to the mailing list or #openstack-keystone on IRC for assistance. Thanks, /Dave

1 0

Assistance Needed: Restricting User List API Access Based on Project Context
by Artem Goncharov 11 Jul '24

11 Jul '24

Hi, 1) in Keystone we are currently working on implementing domain manager persona that allows certain person on a "customer" side to do project and user operations. This includes explicit listing of users in a domain 2) this is conceptually wrong to talk about users in a project, since users are owned by domains and can get access to multiple projects within the domain (and with some tricks to other domains). So you are partially right that role assignments is the way of figuring out access of users to projects. One more challenge though is that such access can be granted using user groups. In this case you actually should also resolve groups access to the desired project. Hope this helps, Artem ---- typed from mobile, auto-correct typos assumed ---- On Thu, Jul 11, 2024, 20:34 Thamanna Farhath <thamanna.f(a)zybisys.com> wrote: > Hello openstack community, > I am currently working on an OpenStack kolla ansible 2023.1. I > have created a custom role " owner " > > *Objective:* > > 1. Users with the owner role can only list users within their project. > 2. Avoid additional API calls for filtering users by project. > > *Current Challenge:* > Using direct API calls, the list users API currently returns all > users without any project-specific filtering. While additional API calls to > the role_assignments endpoint can achieve the desired filtering, this > approach is not efficient and doesn't restrict access at the API level. > > I am considering defining custom Keystone policies to achieve this > restriction. However, I am unsure how to properly configure these policies > to enforce project-specific access control effectively. > > Thanks in advance > > > ------------------------------ > *Disclaimer :** The content of this email and any files transmitted with > it are confidential and intended solely for the use of the individual or > entity to which they are addressed. If you have received this email in > error, please notify the sender and remove the messages from your system. > If you are not the named addressee, it is strictly forbidden for you to > share, circulate, distribute or copy any part of this e-mail to any third > party without the written consent of the sender.* > > > > *E-mail transmission cannot be guaranteed to be secured or error free as > information could be intercepted, corrupted, lost, destroyed, arrive late, > incomplete, or may contain viruses. Therefore, we do not accept liability > for any errors or omissions in the contents of this message, which arise as > a result of e-mail transmission. The recipient should check this e-mail and > any attachments for the presence of viruses. The company accepts no > liability for any damage caused by any virus transmitted by this email."* > >

1 0

[ironic] SNMP hardware type
by Julia Kreger 11 Jul '24

11 Jul '24

Greetings folks, For those that are unaware of it, Ironic has an snmp[0] hardware type which allows for the remote control of power ports on an PDU, which makes Ironic powerful for a variety of non-traditional computing hardware to be leveraged with Ironic. If you are aware of this hardware type and happen to use it, you may want to engage with the Upstream community. Historically we've used a tool called virtualpdu[1] to facilitate testing, and we're facing some challenges with newer pysnmp[2] versions. These challenges have us wondering if there is value for us to fix virtualpdu and continue upstream testing. Realistically, if nobody steps forward to help maintain virtualpdu and the snmp hardware type, upstream testing may become limited or be discontinued for the snmp hardware type entirely, which is likely to impact users of this hardware type. If you're interested in helping maintain the hardware type and its testing, please reach out to the Ironic team via the weekly team meeting[3]. Thanks, -Julia [0]: https://docs.openstack.org/ironic/latest/admin/drivers/snmp.html [1]: https://opendev.org/openstack/virtualpdu [2]: https://pypi.org/project/pysnmp-lextudio/#history [3]: https://wiki.openstack.org/wiki/Meetings/Ironic

1 0

Assistance Needed: Restricting User List API Access Based on Project Context
by Thamanna Farhath 11 Jul '24

11 Jul '24

Hello openstack community, I am currently working on an OpenStack kolla ansible 2023.1. I have created a custom role " owner " Objective: Users with the owner role can only list users within their project. Avoid additional API calls for filtering users by project. Current Challenge: Using direct API calls, the list users API currently returns all users without any project-specific filtering. While additional API calls to the role_assignments endpoint can achieve the desired filtering, this approach is not efficient and doesn't restrict access at the API level. I am considering defining custom Keystone policies to achieve this restriction. However, I am unsure how to properly configure these policies to enforce project-specific access control effectively. Thanks in advance Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

2 1

[nova] "failure domain" scheduler
by Mohammed Naser 11 Jul '24

11 Jul '24

Hi everyone I had a discussion a few days ago in the #openstack-nova about the possibility of introducing the concept of different failure domains, which would be a component inside of an availability zone. The concept is similar to Azure’s availability set feature: https://learn.microsoft.com/en-us/azure/virtual-machines/availability-set-o… I’ve built a very similar nova scheduler filter which leverages server groups + another scheduler hint called `different_failure_domain` then it will actually build them inside different failure domains. The failure domains are built similar to AZs in terms of being modeled are using aggregates, so it would use a `failure_domain` aggregate metadata. I’ve managed to build and validate this functionality with my team here: https://github.com/vexxhost/nova-scheduler-filters Now, based on that, I’m wondering if: 1. Is this the best approach to take based on this? 2. Is this something we can upstream into Nova easily as an extra filter? I think this is helpful for a lot of the “VMware” world people. 😊 Thanks, Mohammed

2 1

numa filtering
by Gk Gk 11 Jul '24

11 Jul '24

How does the scheduler check the numa nodes availability in the background ? Does it use any db like placement to calculate the allocation, resource etc for numa ? I need just the logic in the scheduler for this check ? Thanks Y.G

2 1

[nova] Instance stuck in hard reboot
by Francesco Di Nucci 11 Jul '24

11 Jul '24

Hi all, I hard rebooted an instance while a compute node was down, and now on the dashboard/CLI it is stuck in hard rebooting status. I am able to start/stop/reset it through virsh on the compute node, but I can't get it out of this loop in OpenStack, any suggestions? Thanks in advance -- Francesco Di Nucci System Administrator Compute & Networking Service, INFN Naples Email: francesco.dinucci(a)na.infn.it

2 2

[Swift - proxy server] manually exclude nodes via new middleware
by Olivier Chaze 11 Jul '24

11 Jul '24

Hi all, I am writing to introduce a new middleware component, ExcludeNode, which we've just started to develop to enhance node management within the Swift proxy server. [Middleware Overview] ExcludeNode is designed to dynamically exclude specific nodes from the proxy's operations. This middleware reads a list of nodes to be excluded from a specified file and prevents these nodes from being considered during normal operations. Main reason for developing this middleware is to avoid objects request latency/failure when nodes are down or when rebalancing objects after adding new nodes for example. The main functionalities include: - Node Exclusion Check: Validates if a node (IP:PORT) is listed in the exclusion file. - Dynamic Updates: Allows updating the exclusion list by writing node information to the specified file (IP:PORT). - Clear the exclusion list How to use it ? Assuming 1.2.3.4 storage node is down, requests will fail with : proxy-server: ERROR with Object server 1.2.3.4:6210/sdm re: Trying to GET /v1/AUTH_[...]: ConnectionTimeout (2.0s) (txn: tx19b7067dcb0c442e96a10-0066867068) To ban the disk 1.2.3.4:6210/sdm , on each proxy-server : curl -X POST `hostname`:8080/exclude_node -H "Content-Type: application/json" -d {"ip":"'1.2.3.4","port":6210}"' in proxy-server.log will be logged : proxy-server: Node added to exclusion list: 1.2.3.4:6210 (txn: tx0f292e3b08f940b5be3d9-0066867068) This way, the node won't serve any request and depending the request type, this latter will be redirected to another primary or handoff node. It will remain like this until the disk is manually removed from the exclusion list and every time this node would have been chosen to serve a request you'll get a log : proxy-server: Node 1.2.3.4:6210 (sdm) explicitly excluded by exclude_node middleware. (txn: txeb9f5953ee3343dbb9699-006686707a) I would greatly appreciate the community's feedback on the following: * FUNCTIONALITY: Does the middleware fulfill a common need? * IMPLEMENTATION: Are there any concerns regarding the code implementation or its impact on existing functionalities? We'd like to submmit this middleware upstream but to be efficient and avoid wasting everybody's time I'd like some feedbacks first :) The code itself is quite simple today but we plan to add features like auto excluding a node reported as unmounted by swift-recon for example. exclude_node.py middleware : https://kpaste.infomaniak.com/nRp8AdJGnMuLCoP7czIbL9IBJjlSkyYS#ARXTJo5ZnGa5… server.py modification : https://kpaste.infomaniak.com/Qvy992wlRbc6yEgrIPhAhyrh2V3-Gipy#E2TaKDAXQdXp… proxy-server.conf : [pipeline:main] - pipeline = [...] proxy-server + pipeline = [...] exclude_node proxy-server [filter:exclude_node] use=egg:swift#exclude_node exclude_nodes_filename = /dev/shm/excluded_nodes Thank you in advance for your feedback, Olivier

3 3

Cannot create windows image on Openstack
by Ramzi Abdallah 11 Jul '24

11 Jul '24

I have installed openstack version bobcat on Centos 9 Stream. The installation went fine and I am able to login and launch the cirros image. The problem I am facing is with windows images. When I try to upload the Windows Server 2012 R2 Standard Evaluation for OpenStack which I downloaded from cloudebase I get this error: TypeError: Cannot read properties of undefined (reading 'data') - Error: Unable to create the image. At first I though the problem is from the image so I created my own with all the needed drivers and got the same problem. Appreciate any help is someone experienced this before Ramzi

3 4

[nova][libvirt] Timed out during operation: cannot acquire state change lock
by Satish Patel 10 Jul '24

10 Jul '24

Folks, We had ceph storage failure and finally its up and running and in HEALTH_OK state but now I am not able to bring up openstack vms. I am running the Zed release of openstack. I have noticed a very odd error on nova_libvirt and am not sure what the solution is here. root@comp1:~# tail -f /var/log/kolla/libvirt/libvirtd.log 2024-07-08 07:22:07.420+0000: 3160: warning : qemuDomainObjBeginJobInternal:934 : Cannot start job (query, none, none) for domain instance-000014b0; current job is (async nested, none, start) owned by (3159 remoteDispatchDomainCreateWithFlags, 0 <null>, 3159 remoteDispatchDomainCreateWithFlags (flags=0x1)) for (1563s, 0s, 1563s) 2024-07-08 07:22:07.420+0000: 3160: error : qemuDomainObjBeginJobInternal:968 : Timed out during operation: cannot acquire state change lock (held by monitor=remoteDispatchDomainCreateWithFlags) 2024-07-08 07:22:37.444+0000: 41620: warning : qemuDomainObjBeginJobInternal:934 : Cannot start job (query, none, none) for domain instance-000015b2; current job is (async nested, none, start) owned by (3162 remoteDispatchDomainCreateWithFlags, 0 <null>, 3162 remoteDispatchDomainCreateWithFlags (flags=0x1)) for (1671s, 0s, 1672s) 2024-07-08 07:22:37.444+0000: 41620: error : qemuDomainObjBeginJobInternal:968 : Timed out during operation: cannot acquire state change lock (held by monitor=remoteDispatchDomainCreateWithFlags) I am noticing the VM in a pause state. Any idea what I can do to fix it. I have rebooted compute nodes also in hope it will free up lock or bad process but no luck root@comp1:~# docker exec -it nova_libvirt virsh list Id Name State ---------------------------------- 1 instance-00001751 paused 2 instance-000015b2 paused 3 instance-000014b0 paused When I try to reboot an instance I see the following error. look like qemu acting up.. root@comp1:~# docker exec -it nova_libvirt virsh reboot instance-00001751 error: Failed to reboot domain 'instance-00001751' error: Timed out during operation: cannot acquire state change lock (held by monitor=remoteDispatchDomainMigratePerform3Params)

2 1

[All] CentOS stream 8 jobs
by Tony Breeds 10 Jul '24

10 Jul '24

Hello all, Letting you all know that I, on behalf of the OpenDev infra team bypassed Zuul to merge: https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/922649 This means that any job on any branch testing with FIPS need to be switched to CentOS-Stream-9. Yours Tony.

1 0

[python-openstackclient] openstack client can't use swift command
by chunyang wu 10 Jul '24

10 Jul '24

Hello team: The latest version of python-openstackclient on pypi.org is 6.0.0[0], but this version lacks the ability to use commands related to the Swift project due to the absence of the simplejson package. Although this issue was addressed in a recent commit [1]. The fix has not yet been released on pypi.org. I kindly request that the team release a new version to address this issue as soon as possible. thanks [0]: https://pypi.org/project/python-openstackclient/6.6.0/#history [1]: https://review.opendev.org/c/openstack/python-openstackclient/+/920001

1 0

[NON-ZIP][tacker]Troubleshooting Error in CNF Deployment with Tacker and Helm
by yo-kumamoto＠kddi.com 10 Jul '24

10 Jul '24

Hello, I hope this message finds you well. I am currently working on deploying a Cloud-Native Network Function (CNF) using Tacker. For this deployment, I am utilizing Helm charts as the method of deployment. Following the guidelines provided in the official Tacker documentation (https://docs.openstack.org/tacker/latest/user/v2/cnf/deployment_using_helm/…) I encountered an error during the instantiation process. Despite following the steps closely, the deployment did not proceed as expected. I have attached the error logs to this email for your reference. Could you please provide assistance in troubleshooting this issue? Any insights or suggestions would be greatly appreciated. Thank you for your support. Best regards, Yohei Kumamoto

3 3

Magnum Caracal: magnum-conductor fails with after sql QueuePool limit exhaustion
by Michel Jouvin 10 Jul '24

10 Jul '24

Hi, In our tests of Magnum Caracal, we are experiencing a bad error in openstack-magnum-conductor causing it not to work until it is restarted. The problem is caused by a high number of connections to the magnum DB (compared to what can be seen in Antelope version), exceeding the QueuePool limit configured with max_pool_size (+ an overflow of 50). This is described with more details at https://bugs.launchpad.net/magnum/+bug/2067345 that remained mostly unanswered (but with several people joining to say they were experiencing the same issue). Are they people running Magnum Caracal on this list? Are you experiencing the same issue? In our case we are using RDO distribution (EL9-based) and the Magnum DB is on a remote server. Any suggestion welcome! Best regards, Michel

2 2

[magnum] Cancelling meeting
by Jake Yip 10 Jul '24

10 Jul '24

Hi all, I have to cancel today's meeting due to clashing appointments. Regards, Jake

1 0

[adjutant][tc] Checking in on project health
by Goutham Pacha Ravi 09 Jul '24

09 Jul '24

Hello, During a Technical Committee meeting [1], we were made aware that adjutant's CI jobs are perma-failing; and trivial bot-created patches as well as patches from folks addressing CI issues [2] haven't received any review feedback. I wanted to check on the status of development of this project and its deliverables (adjutant, python-adjutantclient, adjutant-ui). If these are not actively developed, perhaps it is time to retire them [3]. Let me know what you think. Thanks, Goutham [1] https://meetings.opendev.org/meetings/tc/2024/tc.2024-07-09-18.00.log.html#… [2] https://review.opendev.org/q/project:openstack/adjutant+status:open [3] https://docs.openstack.org/project-team-guide/repository.html#retiring-a-re…

2 1

Network demand for controller nodes
by William Muriithi 09 Jul '24

09 Jul '24

Hello, Does anyone know if controller nodes are demanding on network needs? I have a situation where I don't have enough 10GB/s port in the switch. Would a cluster be network constrained if I have three controllers using 1GB/s connections and the execution and storage nodes are on 10GB/s connections? Does the VM traffic ever transverse the controller's network cards? In another word, is there a valid requirement for 10GB/s on openstack controllers? Regards, William

4 3

[skyline][tc] Graduating skyline out of the "Emerging Projects" list
by Goutham Pacha Ravi 09 Jul '24

09 Jul '24

Hi, We were discussing in a recent TC meeting that Skyline seems to be ready to lose the "Emerging project" tag [1]. The arguments supporting it's graduation are: - The project is actively maintained, and has had consistent releases since the Zed release [2][3][4][5] - The project is being used in production cloud deployments [6] - Operators have requested skyline's inclusion in deployment and configuration management projects [7][8] Wu Wenxiang / Skyline team - is this assessment correct? If yes, are there any objections to removing Skyline from [1]? If it helps, the project team can re-assess the OpenStack project requirements we have [9] and work on any deficiencies. Thanks, On behalf of the OpenStack TC, Goutham Pacha Ravi OpenStack TC Chair [1] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [2] https://releases.openstack.org/zed/#skyline-apiserver [3] https://releases.openstack.org/antelope/#skyline-apiserver [4] https://releases.openstack.org/bobcat/#skyline-apiserver [5] https://releases.openstack.org/caracal/#skyline-apiserver [6] https://governance.openstack.org/tc/user_survey/analysis-2023.html#how-do-y… [7] https://review.opendev.org/q/topic:%22osa/skyline%22 [8] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack…

2 2

[all] New page for OpenInfra D&I initiatives and survey
by Jeremy Stanley 09 Jul '24

09 Jul '24

For those who weren't aware, the OpenInfra web site has a new page for Diversity and Inclusion initiatives coordinated by the D&I Working Group: https://openinfra.dev/about/diversity Diversity and inclusion efforts are essential to the success and sustainability of open source communities. The D&I WG is chartered by the foundation board of directors to support OpenStack and other OpenInfra communities in identifying and overcoming barriers to creating and maintaining an inclusive and welcoming environment for all of us. This new web page is a dedicated place to learn more about getting involved in ongoing efforts that promote D&I within OpenInfra. In order to assist the foundation in better understanding the diversity and inclusion challenges faced by those participating in (or attempting to participate in) various OpenInfra communities, please complete the latest version of the Diversity and Inclusion Survey: https://openinfrafoundation.formstack.com/forms/openinfra_diversity_inclusi… This survey is completely anonymous, and your responses will remain confidential. Individual responses are not shared outside OpenInfra Foundation staff and board delegates in the D&I Working Group. If you're involved in multiple OpenInfra communities, please complete the survey multiple times (once for each community), and if possible also share the survey link in team meetings or similar community discussions. Thanks for reading, and I look forward to seeing you in future D&I activities! -- Jeremy Stanley

1 0

[publiccloud-sig] Reminder - next meeting July 10th - 0700 UTC
by Felix Kronlage-Dammers 09 Jul '24

09 Jul '24

Hi everyone, This Wednesday (July 10th) the next meeting of the public cloud sig is going to happen. We’ll be meeting at 0700 UTC here: https://meetpad.opendev.org/publiccloud-sig <https://meetpad.opendev.org/publiccloud-sig> See also here for all other details: https://wiki.openstack.org/wiki/PublicCloudSIG <https://wiki.openstack.org/wiki/PublicCloudSIG> See you on Wednesday! felix -- Felix Kronlage-Dammers Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2024.2/R-12)
by Goutham Pacha Ravi 08 Jul '24

08 Jul '24

Hello Stackers, Time flies, we're over halfway through our 2024.2 (Dalmatian) release cycle! This second half is usually chock-full of release deadlines, specification and code freezes, and more release action [1]. The past week was a short work week for many folks in the Technical Committee due to local holidays, so this update will be brief. Hervé Beraud (hberaud) and Mike Bayer (zzzeek) joined us at the TC's video meeting last week, and they took us through the goal proposal to remove eventlet from OpenStack [2][3][4][5][6]. The meeting was well attended, and it involved a walkthrough of SQLAlchemy, where zzzeek presented how it supports asyncio and a library that allows non-async definitions to invoke awaitables inside an asyncio application. We didn't merge any new governance proposals this week, but several are in review as noted below. === Weekly Meeting === The weekly meeting was held simultaneously on video and IRC. Please check out the recording posted to the TC's YouTube channel [7] as well as the meeting log on eavesdrop [8]. We have new governance proposals that concern Freezer and Monasca, the two projects that were marked "inactive" during the last release cycle. We discussed providing the Skyline team instructions and time to graduate from being an "emerging" project. The TC reiterated its commitment to ensure that all decisions are made asynchronously and recorded on Gerrit to involve community members in different time zones. The rest of the meeting was focused on the eventlet removal discussion. It was a very informative presentation that I'd recommend all OpenStack maintainers review. We took a note to time-box the reviews on the goal proposal. I'd like to request reviews and any strong objections to be expressed on the proposal as soon as possible. The next Technical Committee meeting is today, 2024-07-09 at 1800 UTC! The meeting will be hosted on OFTC's #openstack-tc channel, and the agenda has been posted to the Meetings Wiki Page [9]. Please consider joining us! === Governance Proposals === Remove Eventlet From Openstack | https://review.opendev.org/c/openstack/governance/+/902585 Update criteria for the inactive projects to become active again | https://review.opendev.org/c/openstack/governance/+/921500 Remove Monasca from inactive list | https://review.opendev.org/c/openstack/governance/+/923466 Transition Watcher project to DPL | https://review.opendev.org/c/openstack/governance/+/923583 Inactive state extensions: Freezer | https://review.opendev.org/c/openstack/governance/+/923441 Retire Kuryr-Kubernetes and Kuryr-Tempest-Plugin | https://review.opendev.org/c/openstack/governance/+/922507 Update to include docs and miscellaneous repos for AC status | https://review.opendev.org/c/openstack/governance/+/915021 Adding a rule to manage the affiliation diversity requirement in the TC | https://review.opendev.org/c/openstack/governance/+/922512 === Upcoming Events === 2024-08-14: Nominations open for 2025.1 OpenStack PTL+TC elections 2024-08-29: Dalmatian-3 milestone Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] Release Schedule: https://releases.openstack.org/dalmatian/schedule.html [2] Remove eventlet from OpenStack: https://review.opendev.org/c/openstack/governance/+/902585 [3] Awaitlet SQLAlchemy: https://awaitlet.sqlalchemy.org/en/latest/ [4] SQLAlchemy Asyncio Facade for oslo.db: https://review.opendev.org/c/openstack/oslo.db/+/922976 [5] zzzeek's async demo: https://github.com/zzzeek/async_demo [6] Presentation Slides: https://docs.google.com/presentation/d/169Bc_Uhv-L-0HzM6WuE6uBsC-1hIlgCOQ4l… [7] TC Meeting Recording, 2024-07-02: https://www.youtube.com/watch?v=KnTDP7Dqit8 [8] TC Meeting IRC Logs, 2024-07-02: https://meetings.opendev.org/meetings/tc/2024/tc.2024-07-02-18.00.log.html [9] TC Meeting, 2024-07-09: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

Re: openstack-discuss Digest, Vol 69, Issue 20
by Gk Gk 08 Jul '24

08 Jul '24

Regarding my question, I mean cli tool which will list all the api calling history of a tenant for nova, neutron, storage etc... On Mon, 8 Jul 2024, 18:35 , <openstack-discuss-request(a)lists.openstack.org> wrote: > Send openstack-discuss mailing list submissions to > openstack-discuss(a)lists.openstack.org > > To subscribe or unsubscribe via email, send a message with subject or > body 'help' to > openstack-discuss-request(a)lists.openstack.org > > You can reach the person managing the list at > openstack-discuss-owner(a)lists.openstack.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of openstack-discuss digest..." > > Today's Topics: > > 1. Re: [nova] Instance stuck in hard reboot (smooney(a)redhat.com) > 2. openstack cli (Gk Gk) > 3. Re: openstack cli (Eugen Block) > 4. Network demand for controller nodes (William Muriithi) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 08 Jul 2024 10:55:20 +0100 > From: smooney(a)redhat.com > Subject: Re: [nova] Instance stuck in hard reboot > To: Francesco Di Nucci <francesco.dinucci(a)na.infn.it>, > openstack-discuss <openstack-discuss(a)lists.openstack.org> > Message-ID: > <940e8f9eeb9314936957807719eaa83b65de489c.camel(a)redhat.com> > Content-Type: text/plain; charset="UTF-8" > > On Mon, 2024-07-08 at 09:32 +0200, Francesco Di Nucci wrote: > > Hi all, > > > > I hard rebooted an instance while a compute node was down, and now on > > the dashboard/CLI it is stuck in hard rebooting status. > > > > I am able to start/stop/reset it through virsh on the compute node, but > > I can't get it out of this loop in OpenStack, any suggestions? > as an admin reset state to error then hard reboot. > as a user you could try issuing a hard reboot again but im not sure > that the preconditions will allow that to complete. > > as always when i mention reset-sate i want to be clear that its is a > action of last resort > and generally unsafe if the error is cause by a move operations. This > situration however > is basically the reason that api action extists. you ahve an instance > stuck in a transitory > state due to an interupted or patial lifecycle operation that does not > change the host. > > if soemting is stuck in rebooting like this reset-set is the tool to reach > for and hard reboot > is the default action that you shoudl recach for to bring it form ERROR to > active. > > > > Thanks in advance > > > > > ------------------------------ > > Message: 2 > Date: Mon, 8 Jul 2024 15:58:38 +0530 > From: Gk Gk <ygk.kmr(a)gmail.com> > Subject: openstack cli > To: openstack-discuss <openstack-discuss(a)lists.openstack.org> > Message-ID: > < > CAJreVFwjABsX+fQkTCeR0tqZa5wEv8nYHTarqJDUFShMxGC0dA(a)mail.gmail.com> > Content-Type: multipart/alternative; > boundary="000000000000192a81061cb9e1d9" > > Hi all, > > Is there a tool available in openstack, using which, as admin, we can list > the operations executed by users like, vm operations , network related etc > given their tenant uuid ? Do you think it is a good idea to have such a > tool ? > > Thanks > Y.G >

1 0

[kolla-ansible] No such file or directory: '/var/lib/kolla/config_files/config.json'"]
by Satish Patel 08 Jul '24

08 Jul '24

Folks, Running openstack Zed release. I am adding new compute nodes and getting this error at nova-cell task., any idea what is wrong here? I searched for bugs and people reported but never found any solution. Any idea what we can do? TASK [nova-cell : Get a list of existing cells] *************************************************************************************************************************************************************************************************************************************************** fatal: [comp7 -> ctrl3]: FAILED! => {"changed": false, "failed_when_result": true, "msg": "Container exited with non-zero return code 2", "rc": 2, "stderr": "INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json\nERROR:__main__:Unexpected error:\nTraceback (most recent call last):\n File \"/usr/local/bin/kolla_set_configs\", line 417, in main\n config = load_config()\n File \"/usr/local/bin/kolla_set_configs\", line 290, in load_config\n config = load_from_file()\n File \"/usr/local/bin/kolla_set_configs\", line 280, in load_from_file\n with open(config_file) as f:\nFileNotFoundError: [Errno 2] No such file or directory: '/var/lib/kolla/config_files/config.json'\n", "stderr_lines": ["INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json", "ERROR:__main__:Unexpected error:", "Traceback (most recent call last):", " File \"/usr/local/bin/kolla_set_configs\", line 417, in main", " config = load_config()", " File \"/usr/local/bin/kolla_set_configs\", line 290, in load_config", " config = load_from_file()", " File \"/usr/local/bin/kolla_set_configs\", line 280, in load_from_file", " with open(config_file) as f:", "FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/kolla/config_files/config.json'"], "stdout": "", "stdout_lines": []}

1 0

[ironic] Docs audit complete; results available
by Jay Faulkner 08 Jul '24

08 Jul '24

Hi all, Two PTGs ago, Ironic contributors had a topic about documentation -- we acknowledged our documentation was at times comprehensive, but at times incomprehensible :). This led us to enlist a professional technical writer, Dave Welsch, to assist. Dave has been working over the last few months to get feedback from Ironic users and contributors, as well as auditing our documentation and has put his final findings here: https://docs.google.com/document/d/1e9URuPHKNTx5QXdkCFAzsS0EAAxPJ-xQC4BEpDw… . Actionable bugs were a desired outcome of this, which was achieved. We filed several bugs against Ironic, tagged "docs-audit-2024" -- you can see the list here: https://bugs.launchpad.net/ironic/+bugs?field.tag=docs-audit-2024. I know that identifying issues is only part of the work needed; but it's our hope that by breaking this down and creating a path towards better docs, we'll be able to make it a less daunting task for contributors -- possibly even doable by folks from an outreachy-like program. Thanks for those who participated in this process. Feel free to reach out here if you have any questions, or to me privately. Thanks, Jay Faulkner

1 0

How to setup filters into oslo_notification/cinder
by Helder Ardachnikoff Gomes 08 Jul '24

08 Jul '24

I am working on a project with cinder and we keep listening to events from cinder through oslo messaging. We wish to set up cinder config in order to not receive events with the following event_type: volumes.exist. I read on chatgpt that I could setup my config like that: [oslo_messaging_notifications] driver = messagingv2 topics = notifications notification_format = both notification_filters = -volume.exist But, I am not quite confident about that, may someone help me ? -- _‘Esta mensagem é direcionada apenas para os endereços constantes no cabeçalho inicial. Se você não está listado nos endereços constantes no cabeçalho, pedimos-lhe que desconsidere completamente o conteúdo dessa mensagem e cuja cópia, encaminhamento e/ou execução das ações citadas estão imediatamente anuladas e proibidas’._ * **‘Apesar do Magazine Luiza tomar todas as precauções razoáveis para assegurar que nenhum vírus esteja presente nesse e-mail, a empresa não poderá aceitar a responsabilidade por quaisquer perdas ou danos causados por esse e-mail ou por seus anexos’.*

1 0

[Neutron] Is there a way to work around "External network not reachable from subnet" by setting a static route?
by Florian Haas 08 Jul '24

08 Jul '24

Hi everyone, here's a question I've been chewing on for some time. Please bear with me while I illustrate the background. 1. Suppose you have a regular (internal) Neutron network, subnet, and router. You now plug the subnet into the router and set an external gateway. You then create a port on the subnet. You associate a floating IP with that port. All is well. 2. Suppose instead you do plug the subnet into the router, but you don't set an external gateway on it. You try to associate a floating IP with that port. This creates a BadRequest because: "External network <uuid> is not reachable from subnet <uuid>". 3. Suppose instead you do set an external gateway on the router, but you don't connect the subnet to it. Again, you try to associate a floating IP with that port. This also creates a BadRequest because: "External network <uuid> is not reachable from subnet <uuid>". So far, so good. Up to this point, everything is behaving according to intuitive expectations. Now suppose we have * the external network, * an "outer" internal network, * a Neutron router connecting the two (that is, the router has the external network as its gateway, and has an interface to the "outer" subnet), * an "inner" internal network, * a Nova server (let's call that the "gateway"), which connects the "inner" and "outer" networks, a * another Nova server (the "worker") that sits on the "inner" network. (Suppose, for the sake of discussion, that the "gateway" runs some kind of web application firewall or other virtual security appliance, and the "worker" is a general-purpose Linux instance.) Now I can give the Neutron router a static route so that traffic to the "inner" network uses the "gateway" as its nexthop. Provided the "gateway" does its forwarding correctly, this means that the "worker" has internet connectivity via the "gateway" (which, in turn, achieves this via the router). This, too, works fine for outbound connections originating with the "worker". If I now want to assign a floating IP to the "worker", then routing-wise there really should be no problem: the router should take care of SNAT/DNAT, it would translate the floating IP into a private IP on the "inner" network, would consult its routing table, and throw packets for that "inner" network to the "gateway" to handle, per the static route that's been set. But when I try to associate a floating IP to the "worker" in this scenario, I *still* get the "External network not reachable from subnet" error, because the associated internal check[1] does not take the existing static route into account. This appears to have been an issue in other contexts as well: the defunct Tricircle project appears to have worked around this limitation via a service plugin that just omitted that check.[2] Now my questions are these: 1. Shouldn't this be fixed in core, instead? It would seem to me like the default check is too simplistic, but I might be missing something. 2. Is there a way to make this check take static routes into account, other than providing a service plugin like Tricircle used to do? Thanks! Cheers, Florian References: [1] get_router_for_floatingip(): https://opendev.org/openstack/neutron/src/commit/ebed620aafdff75a05fb5d42ad… [2] get_router_for_floatingip(): https://opendev.org/openstack/tricircle/src/commit/8db8fb30f5757c46a953962d…

2 4

openstack cli
by Gk Gk 08 Jul '24

08 Jul '24

Hi all, Is there a tool available in openstack, using which, as admin, we can list the operations executed by users like, vm operations , network related etc given their tenant uuid ? Do you think it is a good idea to have such a tool ? Thanks Y.G

2 1

when deleting an instance, its volume is not deleted
by kjme001＠gmail.com 08 Jul '24

08 Jul '24

I have the following problem, when deleting an instance, its volume is not deleted along with it, the volume remains and I then have to execute the command to detach the volume from the non-existent instance and then delete it. I don't know what could be causing this, the volume should probably be deleted together with the instance being deleted Thank you for your help

4 3

[neutron] Bug deputy July 01 - July 07
by Lajos Katona 08 Jul '24

08 Jul '24

Hi Neutrinos, This is the bug report from the first week of July: *High*: * https://bugs.launchpad.net/neutron/+bug/2071818 ("over 4096 resubmit actions" error occurs when there are ~250 neutron routers on a provider network) ** In progress: https://review.opendev.org/c/openstack/neutron/+/923391 * https://bugs.launchpad.net/neutron/+bug/2072154 (Port status goes BUILD when migrating non-sriov instance in sriov setting. ) ** In Progress: https://review.opendev.org/c/openstack/neutron/+/923467 * https://bugs.launchpad.net/neutron/+bug/2072466 (FWaaS does not apply rules to routers under DVR agents) ** In Progress: https://review.opendev.org/c/openstack/neutron-fwaas/+/922963 *Medium*: * https://bugs.launchpad.net/neutron/+bug/2071920 (The periodic workers service should run only ``PeriodicWorker`` instances) ** In Progress: https://review.opendev.org/c/openstack/neutron/+/923500 *Invalid*: * https://bugs.launchpad.net/neutron/+bug/2071721 ([OVN] unshelve not working when referenced by allowed address ) Best Wishes Lajos Katona (lajoskatona)

1 0

attach both virtio-scsi and virtio-blk to an instance
by Jiatong Shen 07 Jul '24

07 Jul '24

Hello, I am searching whether it is possible to use both virtio-scsi and virtio-blk at the same time for one instance. I knew it was possible to use each of them separately. I have this requirement because virtio-blk is not possible to support as many as devices like virtio-scsi, but we still want to keep virtio-blk for performance reasons and for legacy instances. Thank you for help. -- Best Regards, Jiatong Shen

2 2

Kolla-Ansible Issues
by Alexandra Rong 05 Jul '24

05 Jul '24

Hello OpenStack-Ansible team, We are students in Germany studying Business Informatics. Currently, we have a project where we need to build an OpenStack environment using Kolla-Ansible. Our goal is to use Gnocchi and CloudKitty to gather metrics on the usage of the VMs in OpenStack. We tried following the official Kolla-Ansible deployment guide but encountered issues starting Kolla-Ansible, as shown in the attached screenshots. Could you possibly help us? Best regards, Felix and Alexandra Ansible.cfg: Error: All-in-one:

5 4

[openstack-helm] SQLAlchemy 2.0 support
by Mohammed Naser 05 Jul '24

05 Jul '24

Hi everyone! It seems like our db-init jobs are broken with the latest version of OpenStack because it doesn’t work with SQLAlchemy 2.0 2024-05-31 03:21:34,290 - OpenStack-Helm DB Init - INFO - Got DB root connection 2024-05-31 03:21:34,290 - OpenStack-Helm DB Init - INFO - Using /etc/keystone/keystone.conf as db config source 2024-05-31 03:21:34,292 - OpenStack-Helm DB Init - INFO - Trying to load db config from database:connection 2024-05-31 03:21:34,292 - OpenStack-Helm DB Init - INFO - Got config from /etc/keystone/keystone.conf 2024-05-31 03:21:34,331 - OpenStack-Helm DB Init - INFO - Tested connection to DB @ percona-xtradb-haproxy.openstack.svc.cluster.local:3306 as root 2024-05-31 03:21:34,332 - OpenStack-Helm DB Init - INFO - Got user db config 2024-05-31 03:21:34,332 - OpenStack-Helm DB Init - CRITICAL - Could not create database keystone Traceback (most recent call last): File "/tmp/db-init.py", line 112, in <module> root_engine.execute("CREATE DATABASE IF NOT EXISTS {0}".format(database)) AttributeError: 'Engine' object has no attribute 'execute' I was wondering if it makes sense to fix this, or if this is something that we should just take the time to simply replace with some sort of simple operator that creates manages/watches database CRs (if one doesn’t exist?) Thanks Mohammed

2 2

[ironic] the fate of ironic-hardware-inventory-recorder-image
by Dmitry Tantsur 05 Jul '24

05 Jul '24

Hi folks! The project in question (https://github.com/metal3-io/ironic-hardware-inventory-recorder-image) was created as part of our effort to allow updating introspection data in ironic-inspector from inside of already running ("active" in the ironic terminology) instances. It is a container image that packages ironic-python-agent in a way that limits its activity to finding ironic-inspector and posting data to it. The initial plans did not happen, ironic-inspector is pending deprecation in favor of ironic's built-in inspection, and the project is unused by anyone in the Metal3 ecosystem. We want to archive and eventually remove it, but we've heard that some members of the ironic community (CERN? CC'ing Arne) are using it. If you do have an interest in this approach, please reach out to me to organize a take-over of the project. Dmitry

4 5

[release] Release countdown for week R-12, Jul 08-12
by Elõd Illés 05 Jul '24

05 Jul '24

Development Focus ----------------- We are now past the Dalmatian-2 milestone, and entering the last development phase of the cycle. Teams should be focused on implementing planned work for the cycle. Now is a good time to review those plans and reprioritize anything if needed based on the what progress has been made and what looks realistic to complete in the next few weeks. General Information ------------------- Looking ahead to the end of the release cycle, please be aware of the feature freeze dates. Those vary depending on deliverable type: * General libraries (except client libraries) need to have their last feature release before Non-client library freeze (August 22nd, 2024). Their stable branches are cut early. * Client libraries (think python-*client libraries) need to have their last feature release before Client library freeze (August 29th, 2024) * Deliverables following a cycle-with-rc model (that would be most services) observe a Feature freeze on that same date, August 29th, 2024. Any feature addition beyond that date should be discussed on the mailing-list and get PTL approval. After feature freeze, cycle-with-rc deliverables need to produce a first release candidate (and a stable branch) before RC1 deadline (September 12th, 2024) * Deliverables following cycle-with-intermediary model can release as necessary, but in all cases before Final RC deadline (September 25th, 2024) Upcoming Deadlines & Dates -------------------------- Non-client library freeze: August 22nd, 2024 (R-6 week) Client library freeze: August 29th, 2024 (R-5 week) Dalmatian-3 milestone: August 29th, 2024 (R-5 week) Előd Illés irc: elodilles @ #openstack-release

1 0

[neutron] Neutron drivers meeting cancelled
by Rodolfo Alonso Hernandez 05 Jul '24

05 Jul '24

Hello Neutrinos: Due to the lack of quorum, today's Neutron drivers meeting will be cancelled. Have a nice weekend.

1 0

[horizon][python-openstackclient] Hypervisor listed in dashboard but not in CLI
by Francesco Di Nucci 05 Jul '24

05 Jul '24

Hello, I am having a strange "error" with an hypervisor. It works and it is shown in Horizon under /Admin/System/System Information/Compute Services/, but it is not listed through the CLI with /openstack hypervisor list/ Any suggestion on how to prevent this? Thanks in advance -- Francesco Di Nucci System Administrator Compute & Networking Service, INFN Naples Email:francesco.dinucci@na.infn.it

1 0

Magnum Caracal: how to use the K8s cluster API instead of Heat driver
by Michel Jouvin 05 Jul '24

05 Jul '24

Hi, In our testing of Caracal Magnum, we saw that the cluster upgrade feature is now disabled for clusters using the Heat driver. There are many good reasons for this explained at [1] and [2] in particular. But I don't see any Magnum documentation of this new approach to K8s cluster provisionning. Is https://vexxhost.github.io/magnum-cluster-api/ the authoritative place? I had a quick look to the URL above but I am not sure I understood what configures Magnum to use Cluster API rather than Heat driver. Is Magnum configured to use one or the other for all clusters it manages? Is it a per cluster choice? How it works to transition from Heat driver to Cluster API, ie how it works if you have existing clusters configured with Heat drivers and if you are creating new ones with Cluster API? Best regards, Michel [1] https://opendev.org/openstack/magnum/commit/2fd3059f3885ff4a14f7f0141016196… (see comments in the code, at line 313) [2] https://vexxhost.com/blog/cluster-api-driver-openstack-magnum/

1 0

[manila] Bugsquash event: Jul 10th to 12th
by Carlos Silva 04 Jul '24

04 Jul '24

Hello, zorilla and interested stackers! We are going to have our first bugsquash of this cycle in the next week. For this event, we will be focusing on pursuing closure for bugs that are currently in progress. - It will start next Wednesday (July 10) and ends on Friday (July 12th). - We will have a kick-off meeting to go over the list of bugs as usual on Wednesday, 14 UTC, and work on assignments if needed. - We will use half of the next upstream weekly meeting to talk about the progress and discuss ongoing reviews. All of the details will be added to the bugsquash etherpad [1]. Looking forward to closing a lot of bugs with your help! [1] https://etherpad.opendev.org/p/manila-dalmatian-m2-bugsquash Regards, carloss

1 0

[CloudKitty] query regarding ElasticSearch usage
by Rafael Weingärtner 04 Jul '24

04 Jul '24

Hello OpenStack community and users of CloudKitty! There have been some discussions in our bi-weekly meetings and vPTG meetings regarding the deprecation and removal of ElasticSearch support as a storage backend. Therefore, this e-mail is to consult/query users regarding ElasticSearch users. If there are no people using it anymore with CloudKitty, we are considering the alternative of removing it and maintaining only the OpenSearch support. We would like to hear back from you guys. Let me know if you are using ElasticSearch, and if you intend to stick with it. Also, do you guys feel the need to improve something in its support in CloudKitty. Thanks in advance for your feedback! -- Rafael Weingärtner

5 5

Magnum issues http requests to other services when internal/admin endpoint is https
by Michel Jouvin 04 Jul '24

04 Jul '24

Hi, We have reconfigured our cloud with a Nginx sever handling TLS in front of every API service, as described in https://docs.openstack.org/security-guide/secure-communication/secure-refer…. Previously we used to have Magnum/Heat/Barbican endpoints using http rather than https (not good!). Since this change, we have seen problems in Magnum (starting in Antelope when we did the change but still present in Caracal) where some "internal" requests to Barbican and Heat are done in http rather than https. We worked around this defining a redirect to https on 497 status code, as documented in Nginx documentation but it is not really satisfying as it means all clients, not only Magnum service, could use http on the https port and get serviced. I remember seing an issue on Launchpad about this issue but forgot the issue number. We were wondering wether it would not be better, at least until the problem is fixed, to define the internal+admin endpoints with http rather than https. According to our tests, it fixes the problem, as expected. Thanks in advance for any thought or experience on this. Best regards, Michel

2 2

[nova] volume information was duplicated when Live Migration failed
by Junya Noguchi (Fujitsu) 04 Jul '24

04 Jul '24

To whom it may concern Hi. I encountered an issue with OpenStack where volume information was duplicated when Live Migration failed. Could you please provide a solution to this issue? [Issue] After a Live Migration failure, when I checked the volume where the error occurred using the 'openstack volume list' command, the following was displayed: +--------------------------------------+-----------------------------------+-----------+------+---------------------------------------------------------------------------------------------------------------------------------------------+ | ID | Name | Status | Size | Attached to | +--------------------------------------+-----------------------------------+-----------+------+---------------------------------------------------------------------------------------------------------------------------------------------+ | 9ba9734f-575e-41de-8bb4-60839388e0ad | | in-use | 10 | Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to | | | | | | 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to | | | | | | 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to | | | | | | 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda | I expect that the 'Attached to' display should only appear once as there is one volume connected, but it is duplicated for each failed attempt. The versions of OpenStack and nova are as follows: OpenStack Zed $ openstack --version openstack 6.0.0 $ nova --version nova CLI is deprecated and will be a removed in a future release 18.1.0 [Details] The volume display is duplicated under the following conditions: - When Live Migration of an instance fails If the communication between the ports required for Live Migration is closed, Live Migration will fail when executed. $ sudo netstat -tuln | grep 16509 $ The output of nova-compute.log at the time of Live Migration failure is as follows: --- 2024-02-28 17:04:32.119 2785 ERROR nova.virt.libvirt.driver [-] [instance: 521eb55f-535d-4fa7-a27e-447b0bbae9b4] Live Migration failure: unable to connect to server at 'XXXXX.com:16509': Connection refused: libvirt.libvirtError: unable to connect to server at 'XXXXX.com:16509': Connection refused 2024-02-28 17:04:32.119 2785 DEBUG nova.virt.libvirt.driver [-] [instance: 521eb55f-535d-4fa7-a27e-447b0bbae9b4] Migration operation thread notification thread_finished /usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py:10516 2024-02-28 17:04:32.607 2785 DEBUG nova.virt.libvirt.migration [-] [instance: 521eb55f-535d-4fa7-a27e-447b0bbae9b4] VM running on src, migration failed _log /usr/lib/python3.9/site-packages/nova/virt/libvirt/migration.py:432 2024-02-28 17:04:32.608 2785 DEBUG nova.virt.libvirt.driver [-] [instance: 521eb55f-535d-4fa7-a27e-447b0bbae9b4] Fixed incorrect job type to be 4 _live_migration_monitor /usr/lib/python3.9/site-packages/nova/virt/libvirt/driver.py:10330 2024-02-28 17:04:32.608 2785 ERROR nova.virt.libvirt.driver [-] [instance: 521eb55f-535d-4fa7-a27e-447b0bbae9b4] Migration operation has aborted --- After Live Migration, when I check the volume where the error occurred using the 'openstack volume list' command, the following is displayed: +--------------------------------------+-----------------------------------+-----------+------+---------------------------------------------------------------------------------------------------------------------------------------------+ | ID | Name | Status | Size | Attached to | +--------------------------------------+-----------------------------------+-----------+------+---------------------------------------------------------------------------------------------------------------------------------------------+ | 9ba9734f-575e-41de-8bb4-60839388e0ad | | in-use | 10 | Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to | | | | | | 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to | | | | | | 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to | | | | | | 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda Attached to 73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda | '73c4c6e8-fe34-4f87-9f26-e70b4cc593ba on /dev/vda' is displayed as many times as Live Migration failed. The duplicate display did not disappear even after Live Migration was successful. Thanks,

2 2

[kolla-ansible][letsencrypt] containers are running but not getting certificates.
by Jonathan Proulx 03 Jul '24

03 Jul '24

Hi All, I'm trying to do a test multinode deploy using 2023.2 I have letsencrypt_webserver and letsencrypt_lego contsainers running and I'm seeing random traffic in the /var/log/kolla/letsencrypt/letsencrypt-webserver-access.log so fairly confident they're plumbed through to the public internet properly, but I don't seem to be getting certificates. how can I trigger a renewal attempt so I can maybe see what I've screwed up? Thanks, -Jon -- Jonathan Proulx (he/him) Sr. Technical Architect The Infrastructure Group MIT CSAIL

3 6

[OSSA-2024-001] OpenStack Cinder, Glance, Nova: Arbitrary file access through custom QCOW2 external data (CVE-2024-32498)
by Jeremy Stanley 02 Jul '24

02 Jul '24

======================================================================= OSSA-2024-001: Arbitrary file access through custom QCOW2 external data ======================================================================= :Date: July 02, 2024 :CVE: CVE-2024-32498 Affects ~~~~~~~ - Cinder: <22.1.3, >=23.0.0 <23.1.1, ==24.0.0 - Glance: <26.0.1, ==27.0.0, >=28.0.0 <28.0.2 - Nova: <27.3.1, >=28.0.0 <28.1.1, >=29.0.0 <29.0.3 Description ~~~~~~~~~~~ Martin Kaesberger reported a vulnerability in QCOW2 image processing for Cinder, Glance and Nova. By supplying a specially created QCOW2 image which references a specific data file path, an authenticated user may convince systems to return a copy of that file's contents from the server resulting in unauthorized access to potentially sensitive data. All Cinder deployments are affected; only Glance deployments with image conversion enabled are affected; all Nova deployments are affected. Patches ~~~~~~~ - https://review.opendev.org/923247 (2023.1/antelope(cinder)) - https://review.opendev.org/923277 (2023.1/antelope(glance)) - https://review.opendev.org/923278 (2023.1/antelope(glance)) - https://review.opendev.org/923279 (2023.1/antelope(glance)) - https://review.opendev.org/923280 (2023.1/antelope(glance)) - https://review.opendev.org/923281 (2023.1/antelope(glance)) - https://review.opendev.org/923282 (2023.1/antelope(glance)) - https://review.opendev.org/923283 (2023.1/antelope(glance)) - https://review.opendev.org/923288 (2023.1/antelope(nova)) - https://review.opendev.org/923289 (2023.1/antelope(nova)) - https://review.opendev.org/923290 (2023.1/antelope(nova)) - https://review.opendev.org/923281 (2023.1/antelope(nova)) - https://review.opendev.org/923246 (2023.2/bobcat(cinder)) - https://review.opendev.org/923266 (2023.2/bobcat(glance)) - https://review.opendev.org/923267 (2023.2/bobcat(glance)) - https://review.opendev.org/923268 (2023.2/bobcat(glance)) - https://review.opendev.org/923269 (2023.2/bobcat(glance)) - https://review.opendev.org/923270 (2023.2/bobcat(glance)) - https://review.opendev.org/923271 (2023.2/bobcat(glance)) - https://review.opendev.org/923272 (2023.2/bobcat(glance)) - https://review.opendev.org/923284 (2023.2/bobcat(nova)) - https://review.opendev.org/923285 (2023.2/bobcat(nova)) - https://review.opendev.org/923286 (2023.2/bobcat(nova)) - https://review.opendev.org/923287 (2023.2/bobcat(nova)) - https://review.opendev.org/923245 (2024.1/caracal(cinder)) - https://review.opendev.org/923259 (2024.1/caracal(glance)) - https://review.opendev.org/923260 (2024.1/caracal(glance)) - https://review.opendev.org/923261 (2024.1/caracal(glance)) - https://review.opendev.org/923262 (2024.1/caracal(glance)) - https://review.opendev.org/923263 (2024.1/caracal(glance)) - https://review.opendev.org/923264 (2024.1/caracal(glance)) - https://review.opendev.org/923265 (2024.1/caracal(glance)) - https://review.opendev.org/923273 (2024.1/caracal(nova)) - https://review.opendev.org/923274 (2024.1/caracal(nova)) - https://review.opendev.org/923275 (2024.1/caracal(nova)) - https://review.opendev.org/923276 (2024.1/caracal(nova)) - https://review.opendev.org/923244 (2024.2/dalmatian(cinder)) - https://review.opendev.org/923248 (2024.2/dalmatian(glance)) - https://review.opendev.org/923249 (2024.2/dalmatian(glance)) - https://review.opendev.org/923250 (2024.2/dalmatian(glance)) - https://review.opendev.org/923251 (2024.2/dalmatian(glance)) - https://review.opendev.org/923252 (2024.2/dalmatian(glance)) - https://review.opendev.org/923253 (2024.2/dalmatian(glance)) - https://review.opendev.org/923254 (2024.2/dalmatian(glance)) - https://review.opendev.org/923255 (2024.2/dalmatian(nova)) - https://review.opendev.org/923256 (2024.2/dalmatian(nova)) - https://review.opendev.org/923257 (2024.2/dalmatian(nova)) - https://review.opendev.org/923258 (2024.2/dalmatian(nova)) Credits ~~~~~~~ - Martin Kaesberger (CVE-2024-32498) References ~~~~~~~~~~ - https://launchpad.net/bugs/2059809 - http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-32498 Notes ~~~~~ - Due to the scope of the problem and complexity of the resulting fixes, regressions and additional bypasses were reported in the original bug by downstream stakeholders during the coordinated disclosure period. As a result, our initially chosen publication date was rescheduled, which put the advisory four days past our promised ninety day maximum embargo length. Additional revised patches and regression fixes were supplied to stakeholders as soon as possible, but we understand the unfortunate timing of these last-minute changes resulted in a lot of additional work for everyone involved. -- Jeremy Stanley OpenStack Vulnerability Management Team

2 1

[Cinder] Extending Specs Freeze Deadline to Jun 7
by Jon Bernard 02 Jul '24

02 Jul '24

Hello, As discussed in the last cinder meeting [1] we are extending the cinder specs freeze deadline to June 7. The previous week contains holidays for some on our team, so the weekend days are added to make sure everything that needs to merge has sufficient time to do so. Please make sure your specs are updated before the new deadline. [1]: https://meetings.opendev.org/meetings/cinder/2024/cinder.2024-06-26-14.00.l… Thanks, -- Jon

1 0

any idiomatic testtools / stestr patterns for asyncio?
by Mike Bayer 02 Jul '24

02 Jul '24

hey gang - I'm adding asyncio support to oslo.db. Fairly major block hit, which I can work around, is that testtools / stestr do not seem to have any provision for asyncio present. An initial attempt working with it gracefully looked like this (using oslotests base, which is the testtoolsTestBase): import unittest from oslotest import base class MyTestSuite(unittest.IsolatedAsyncioTestCase, base.BaseTestCase): # ... but that didn't work because the degree to which testtools modifies the run() method of unittest.TestCase prevents IsolatedAsyncioTestCase from running awaitable test cases. Next attempt is this, remove testtools / oslotest from the class hierarchy (which is bad, because we lose all of our testtools patterns): class MyTestSuite(unittest.IsolatedAsyncioTestCase): # ... this runs, however stestr sends into the test suite a Result object that is not a subclass of unittest.result.TestResult, it's a "subunit.test_results.AutoTimingTestResultDecorator" that does not TestResult in its MRO; unittest shows a warning that there is no addDuration() method on the result object. final workaround, patch one on, here's my base: class BaseAsyncioCase(unittest.IsolatedAsyncioTestCase): def run(self, result=None): # work around stestr sending a result object that's not a # unittest.TestResult result.addDuration = lambda test, elapsed: None return super().run(result=result) now I can run asyncio tests. But only by ditching testtools entirely. What would be the correct approach to introducing asyncio support in our test suites given our dependency on testtools?

4 6

Regarding Cinder Volume as a backend not mounting on Compute Node
by Adivya Singh 01 Jul '24

01 Jul '24

Hi Team, I am facing a issue, where my Cinder Volume is not getting mounted on Compute node, it creates Successfully, Here is my export file input on storage server /cinder/volume 172.29.244.0/22(rw,no_subtree_check,no_root_squash) and here is my NFS mount Option on Cinder as a backend nfs_mount_options: noatime,rw,bg,intr,async,timeo=60 it mounts successfully on Controller, but compute it get struck Regards Adivya Singh

1 0

[tacker] Cancel weekly meeting
by Yasufumi Ogawa 01 Jul '24

01 Jul '24

Hi team, I'm canceling today's meeting since I'm not available no new topic on the etherpad. Please feel free to host someone else if you have any topic. Thanks, Yasufumi

1 0

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2024.2/R-13)
by Goutham Pacha Ravi 01 Jul '24

01 Jul '24

Hello Stackers, We're officially at Milestone 2 of the 2024.2 ("Dalmatian") release cycle [1]. Alongside spec reviews and other project-specific deadlines, this is the week to look out for intermediary releases of several OpenStack projects and close out bug fixes, or re-target open ones for a future milestone. We're soon going to see some patches to add the code name that was picked for the 2025.1 release (31st release!) of OpenStack: "Epoxy" [2]. I probably jinxed the animal theme, but Kendall Nelson (diablo_rojo) told me that Epoxy will be "the glue that holds our clouds together." The Technical Committee was reviewing project activity in Monasca [3], Freezer, and Skyline [4]. As you've probably gathered from the discussion on this list and elsewhere, Monasca seems ready to leave the "inactive" project list and Skyline seems set to graduate from the "emerging" project list [5]. The respective project teams will need to make these proposals, and I'll keep you posted on how that evolves. The Freezer project team would like to continue to keep the "inactive" project tag through this release as they continue to stabilize the codebase and the CI jobs. This means that there will be no releases from the Freezer team during the 2024.2 ("Dalmatian") release cycle. Hervé Beraud (hberaud) and Mike Bayer (zzzeek) will be joining us at the Technical Committee monthly video meeting tomorrow (2024-07-02) to preview the work ahead of us to remove eventlet from OpenStack [6][7]. If you are an OpenStack project maintainer, packager, or operator, please consider joining so we can have a productive discussion. === Weekly Meeting === The Technical Committee met last week on OFTC's #openstack-tc IRC channel [8]. We discussed several in-progress changes, including a proposal to address affiliation diversity violations that may occur during TC elections. There was an emphasis on gate health. Thanks to a few patches to devstack, tempest, and devstack plugins, several integrated gate jobs were more stable over the past couple of weeks. As noted last week, CentOS Stream 8 is now EOL, and hence any test jobs that were relying on this OS are now perma-failing. This is a call-out to project maintainers to clean up their test jobs across all code branches. Currently, a search of trunk branches reveals that centos-stream-8 nodes are required for test jobs in Tempest, Rally, Tenks, Ansible tests, and tempest plugins of Octavia, Telemetry, and Cinder. Please do look at cruft left behind in older stable and "unmaintained" branches as well. TC members also brainstormed on projects that are seemingly active, but the project's core team is unresponsive to new contributors. This is against the Open Community principles that we adhere to [9]. If you're aware of instances of this happening, please do let the TC know. Our next weekly meeting is held simultaneously on Video and IRC. Details, including the meeting's agenda, are in [7]. I hope you'll be able to participate. As always, our IRC meetings are logged to eavesdrop [10], and our Video meeting recordings are uploaded to YouTube [11]. === Governance Proposals === ==== Merged ==== Show TC and SIG repos under reference | https://review.opendev.org/c/openstack/governance/+/918488 ==== Open for review/discussion ==== Retire kuryr-kubernetes and kuryr-tempest-plugin | https://review.opendev.org/c/openstack/governance/+/922507 Update to include docs and miscellaneous repos for AC status | https://review.opendev.org/c/openstack/governance/+/915021 Adding a rule to manage the affiliation diversity requirement in the TC | https://review.opendev.org/c/openstack/governance/+/922512 Update criteria for the inactive projects to become active again | https://review.opendev.org/c/openstack/governance/+/921500 Remove Eventlet From OpenStack | https://review.opendev.org/c/openstack/governance/+/902585 === Upcoming Events === 2024-07-02: Monthly Technical Committee Video+IRC meeting 2024-07-02: OpenInfra Board Meeting -: https://board.openinfra.dev/meetings/ 2024-07-04: Milestone-2 of the 2024.2 release cycle Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] Release Schedule: https://releases.openstack.org/dalmatian/schedule.html [2] OpenStack "Epoxy": https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [3] Reviving Monasca: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [4] Graduating Skyline: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [5] Emerging Technology and Inactive Projects: https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [6] Goal proposal to "Remove Eventlet from OpenStack": https://review.opendev.org/c/openstack/governance/+/902585 [7] Agenda for the OpenStack TC Meeting on 1st July 2024: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [8] TC Meeting log (2024-06-25): https://meetings.opendev.org/meetings/tc/2024/tc.2024-06-25-18.00.log.html [9] "Open Community": https://opendev.org/openinfra/four-opens/src/branch/master/doc/source/openc… [10] Past TC meetings: https://meetings.opendev.org/meetings/tc/ [11] YouTube channel recordings of past TC meetings: https://www.youtube.com/watch?v=34w1Aoai1EA&list=PLhwOhbQKWT7Ui1KPL7MLIq4ye…

1 0

problem with deleting a volume from OpenStack
by kjme001＠gmail.com 01 Jul '24

01 Jul '24

I have a problem with deleting a volume from OpenStack from a machine that no longer exists because it has crashed. If it executes the command: openstack volume delete volume_name receives the deleting status and the volume is not deleted, it is still in the deleting state. How can I delete such a volume? Can I remove it from the postgres database? Regards

2 4

[horizon][kolla-ansible][antelope]
by Andrej Moravčík 01 Jul '24

01 Jul '24

Hello, I have just observed Horizon behavior with Project Names displayed incorrectly in Admin / Overview page and I would like to ask if anybody else can confirm the same issue. It is apparently related to Horizon Multidomain support, which is not enabled by default and maybe it is not very used / tested feature. The issue is reported here: https://bugs.launchpad.net/horizon/+bug/2071630 Thanks. — Andrej Moravčík ________________________________ Táto správa a jej prílohy sú dôverné. Prosím, narábajte s týmito informáciami ako s dôvernými a nezverejňujte, nekopírujte ani neposielajte tieto informácie bez predchádzajúceho súhlasu spoločnosti Aliter Technologies, a.s. Ak nie ste adresátom indikovaným v záhlaví tejto správy alebo želaný príjemca, upovedomte prosím spoločnosť Aliter Technologies, a.s. odpoveďou na tento e-mail a pôvodnú správu vymažte zo svojho systému. Aliter Technologies - registrovaná ochranná známka pre EU, USA This message and any attachments are confidential. Please treat the information as confidential, and do not disclose, copy or deliver this message to anyone without Aliter Technologies' approval. If you are not the addressee indicated in this message or an intended recipient please notify Aliter Technologies a.s. by return e-mail and delete the message from your system. Aliter Technologies - Registered in WIPO & U.S. Patent and Trademark Office

1 0

[neutron] DHCP agent event processing multithread LP#2070376
by Rodolfo Alonso Hernandez 01 Jul '24

01 Jul '24

Hello Neutrinos: As discussed during the last Neutron drivers meeting, I said that I was going to benchmark the performance of the DHCP agent with and without multiple event processing threads. The result is in [2]. As expected, the number of threads attending the received events (network creation, port deletion, etc) does not affect the speed of the DHCP agent. In particular I tested the resync transient that happens during the DHCP agent restart, that is easy to reproduce and provides consistent results. Regards. [1]https://bugs.launchpad.net/neutron/+bug/2070376 [2]https://bugs.launchpad.net/neutron/+bug/2070376/comments/1

1 0

[neutron] Bug deputy Jun 24 - Jun 30
by Rodolfo Alonso Hernandez 01 Jul '24

01 Jul '24

Hello Neutrinos: This is the bug report from the last week: High: * https://bugs.launchpad.net/neutron/+bug/2071426: [DHCP] DHCP agent failing in ``DHCPResourceUpdate.__lt__`` method. Assigned to Rodolfo. Patch: https://review.opendev.org/c/openstack/neutron/+/923026 * https://bugs.launchpad.net/neutron/+bug/2071374: https://bugs.launchpad.net/neutron/+bug/2071374. Assigned to Roberto. Patch: https://review.opendev.org/c/openstack/neutron/+/922972 Medium: * https://bugs.launchpad.net/neutron/+bug/2071451: [ml2][ovs][firewall] port changes ofport and local_vlan will lead to wrong flow installation. Assigned to Liu. Patch: https://review.opendev.org/c/openstack/neutron/+/923040 Low: * https://bugs.launchpad.net/neutron/+bug/2070376: [eventlet-deprecation] Reduce the DHCP agent pool processing to a single thread. Assigned to Rodolfo. In progress. Whislist/RFE: * https://bugs.launchpad.net/neutron/+bug/2071323: [RFE] Flow logs support for OpenStack Networking. Not assigned but most probably Liu, the reporter, will take care of it once triaged and approved by the drivers team. Have a nice week.

1 0

Warehouse failure
by ゞPositive î 01 Jul '24

01 Jul '24

The Centos system has stopped updating now, and we hope that the official website manual Centos can be changed to Rocky Linux. Now we are using Rocky Linux to install OpenStack using the Centos system installation method. Among them, the installation source link for yum install Centos release openstack ussuri and yum install Centos release openstack xena is invalid. We hope to receive your solution. ゞPositive ? 451526204(a)qq.com  

2 1