July 2024 - openstack-discuss - lists.openstack.org

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

How to config DPDK compute in OVN
by dangerzone ar 22 Apr '25

22 Apr '25

Hi....as far I understand ovs does support dpdk compute to be deployed. I'm referring to the guideline ' *https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html <https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html>* ' ... It's too brief.. May I know how to config dpdk compute in OVN networking openstack? Hope someone could help what is the config to be done.... Currently compute is run without dpdk over openstack train ovn based. Please need some info on how I can proceed. Thank you

2 1

[oslo] python-consul2 is unmaintained
by Takashi Kajinami 21 Apr '25

21 Apr '25

Hello, I recently noticed that python-consul2 library[1], which is used by the consul coordination driver in tooz, hasn't been updated for 3+ years. Looking at issues in the project, there is an indication that the library is not compatible with ACL API in consul >= 1.11[2]. There is another fork created[3] and this includes fixes to support new API endpoints. Although the driver in tooz does not directly use the feature, I think it's better to discuss our plan to mitigate risks caused by the unmaintained library. As far as I can think of, there are three options now 1. Deprecate consul driver and remove it 2. Keep using python-consul2 now. At least basic functionality is proven to work by our functional tests (Note that the tests do not cover all configuration patterns) 3. Replace python-consul2 by py-consul Before we start the overall discussions I'd like to ask a few questions to the wider audience, as our inputs. - Is anyone using the consul coordination driver now ? - Does anyone have experience with python-consul2 and/or py-consul ? Note that this discussion affects the http messaging driver[4] currently proposed, because it also requires client library to communicate with Consul. Actually I noticed the situation when I noticed that the requirement check job is failing because the library is not part of global requirements list... Thank you, Takashi Kajinami [1] https://github.com/poppyred/python-consul2 [2] https://github.com/poppyred/python-consul2/issues/42 [3] https://github.com/criteo-forks/py-consul [4] https://review.opendev.org/c/openstack/oslo.messaging/+/912499 -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

2 3

[octavia] Purging ERROR'ed amphora and load balancers stuck in transitional states
by Paul Browne 11 Apr '25

11 Apr '25

Hello Octavia users, For a while I've been trying to find a systematic way to remove Octavia LBs+Amphora that have become stuck in ERROR or transitional PENDING_* states ; $ openstack loadbalancer amphora list -f value | grep ERROR a2323f14-3aaa-418a-8249-111aaa9c21fe 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR MASTER 10.8.0.242 192.168.3.247 e5b236ba-e7ee-4ed7-9f58-57ce7a408489 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR BACKUP 10.8.0.190 192.168.3.247 6b556f28-93c9-49dd-b6ee-4379288e7957 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR MASTER 10.8.0.39 192.168.3.126 c669db5d-8686-4d5c-9e95-e02030b34301 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR BACKUP 10.8.0.174 192.168.3.126 $ openstack loadbalancer list -f value | grep -vi active 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe k8s-clusterapi-cluster-default-ci-6386871107-kube-upgrade-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.247 PENDING_UPDATE amphora d5e402fe-2c4b-49af-a700-532cb408cee5 k8s-clusterapi-cluster-default-ci-6386871107-latest-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.126 PENDING_DELETE amphora These resources are marked immutable and so cannot be failed over or deleted; $ openstack loadbalancer amphora failover a2323f14-3aaa-418a-8249-111aaa9c21fe Load Balancer 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe is immutable and cannot be updated. (HTTP 409) (Request-ID: req-6e66c4e8-c3d3-4549-a03c-367017c8c8b3) $ openstack loadbalancer failover 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe Invalid state PENDING_UPDATE of loadbalancer resource 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe (HTTP 409) (Request-ID: req-84b44212-e7a8-4101-a16f-18c774c0577e) The backing Nova instances for these Amphora do seem to exist and be in good working order. Is there any API way to purge these out of Octavia's service state, or would (very careful) DB hackery be required here? Many thanks, Paul Browne ******************* Paul Browne Research Computing Platforms University Information Services Roger Needham Building JJ Thompson Avenue University of Cambridge Cambridge United Kingdom E-Mail: pfb29(a)cam.ac.uk<mailto:pfb29@cam.ac.uk> Tel: 0044-1223-746548 *******************

4 6

[tc][all][monasca ] Retiring the Monasca project
by Ghanshyam Mann 16 Feb '25

16 Feb '25

Hello everyone, As you might know, the Monasca project is not active and was marked 'inactive' since 2024.1 cycle[1]. In this cycle also, this project is leaderless[2] and no maintainer stepped up to maintain it. As discussed in the Leaderless project etherpad[2], the next step is to retire the project. I'm sending this email to propose the Monasca retirement and based on the response, I will proceed next. [1] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [2] https://etherpad.opendev.org/p/2024.2-leaderless#L72 -gmann

6 17

[ironic]About the OS supported by Diskimage-builder
by Junya Noguchi (Fujitsu) 03 Feb '25

03 Feb '25

To whom it may concern. My name is Junya Noguchi. I have some questions about the supported OS for Diskimage-builder operation. Please respond. References https://docs.openstack.org/ironic-python-agent-builder/latest/admin/dib.html Does it mean that the OS that says "work but receive only limited testing and support" works but is not supported? References https://docs.openstack.org/diskimage-builder/latest/ This document describes RHEL and other operating systems that are not supported in the previous question. Are these operating systems supported? Also, are there any plans to support more operating systems in the future? Best regards, Junya Noguchi

5 7

[glance][ops] proposal to remove the glance-scrubber
by Brian Rosmaita 19 Jan '25

19 Jan '25

Hello Glance users, The Glance project team wants to determine whether anyone is using the glance-scrubber utility [0]. If not, we are thinking of deprecating it this cycle to be removed in the appropriate future cycle. If you have not previously heard of the glance-scrubber, that's most likely because it's only useful for particular deployment configurations of which yours is not one. See the doc [0] for details if you are curious. Our suspicion is that the glance-scrubber may have outlived its usefulness for modern OpenStack configurations. (The last time I remember it being discussed at a PTG was during the snowstorm in Dublin, which I believe was the Rocky development cycle). On the other hand, the team also has some ideas for improving the scrubber, but there's no point doing that if it's not actually being used in the first place. So if removal of the glance-scrubber would negatively affect you, please respond to this email so we can get some data points. Thanks! [0] https://docs.openstack.org/glance/latest/cli/glancescrubber.html

2 2

[cinder][kubernetes] pvc stuck in reserved state
by Eugen Block 10 Jan '25

10 Jan '25

Hi, we have a couple of k8s clusters in our openstack environment (Victoria) and are facing some problems when pods get moved to different worker nodes. The PVCs get stuck in "reserved" state, and they only get attached after resetting the volumes to "available" state. First I thought it was due to the missing multiattach volume type which seems to work fine in our customer's cluster. I also found the "duplicate connector" error messages in the nova-compute.log: > Unable to update attachment.(Invalid volume: duplicate connectors > detected on volume But after adding the multiattach volume type and retyping the affected volumes, the PVCs still get stuck in "reserved" state while I don't see any error messages on the hypervisors anymore. So it seems that something has changed/improved, but now I don't have a trace where to look next. Could this be a cinder-csi driver issue or something? Our cinder backend is ceph (Pacific). Not sure which information could help so please ask for anything I can provide. Any pointers are highly appreciated! Thanks! Eugen

1 2

ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version
by Swogat Pradhan 23 Dec '24

23 Dec '24

Hi, I am currently trying to deploy openstack wallaby using tripleo arch. I created the network jinja templates, ran the following commands also: #openstack overcloud network provision --stack overcloud --output networks-deployed-environment.yaml custom_network_data.yaml # openstack overcloud network vip provision --stack overcloud --output vip-deployed-environment.yaml custom_vip_data.yaml # openstack overcloud node provision --stack overcloud --overcloud-ssh-key /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml and used the environment files in the openstack overcloud deploy command: (undercloud) [stack@hkg2director ~]$ cat deploy.sh #!/bin/bash THT=/usr/share/openstack-tripleo-heat-templates/ CNF=/home/stack/ openstack overcloud deploy --templates $THT \ -r $CNF/templates/roles_data.yaml \ -n $CNF/workplace/custom_network_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e $CNF/templates/node-info.yaml \ -e $CNF/templates/scheduler-hints.yaml \ -e $CNF/workplace/networks-deployed-environment.yaml \ -e $CNF/workplace/vip-deployed-environment.yaml \ -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ -e $CNF/workplace/custom-net-bond-with-vlans.yaml Now when i run the ./deploy.sh script i encounter an error stating: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command.: tripleoclient.exceptions.InvalidConfiguration: Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command. 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END return value: 1 Can someone tell me where the mistake is? With regards, Swogat Pradhan

4 4

[kolla] [openeuler] CI jobs status
by Michał Nasiadka 26 Nov '24

26 Nov '24

Hello Koalas, OpenEuler jobs in master (to be Antelope) are failing for a long time. If no one is interested in maintaining OpenEuler support in Kolla-Ansible - I will propose dropping it’s support on Wednesday Kolla meeting. Best regards, Michal

4 3

[horizon][ceph] object storage container storage policy missing
by Satish Patel 24 Nov '24

24 Nov '24

Folks, Openstack Release: Zed Recently I have integrated ceph RGW service with openstack/keystone and testing with command line tool "openstack container create foo1" and it works but when I go to Horizon UI I am seeing it doesn't have storage policy dropdown list and because of that I am not able to create container or upload file from Horizon GUI https://ibb.co/KhYW6Rd It works with commands like but not horizon. $ openstack container create foo1 +---------------------------------------+-----------+------------------------------------------------+ | account | container | x-trans-id | +---------------------------------------+-----------+------------------------------------------------+ | AUTH_d4d82849885e4126aef402c408ad2c5c | foo1 | tx00000c057454841b13497-0065b558b4-42ff2eb-ahm | +---------------------------------------+-----------+------------------------------------------------+ $ openstack container list +------+ | Name | +------+ | foo1 | +------+

2 2

[ironic] Having issue with deploying ISO image to node
by James Leong 17 Nov '24

17 Nov '24

Hi all, I am trying to play around with ironic, and I am having an issue with deploying an Ubuntu 20.04 ISO image onto my bare metal node. Since I am quite new to bare metal, I am not entirely sure what went wrong as I couldn't find any error messages shown on my log file (ironic-conductor, ironic-api, ironic_neutron_agent, and nova-compute-ironic). This might be my guess; the ironic inspection is hitting timeout, which is causing the deployment to timeout. However, I have no idea why the inspection is failing at timeout. Any help would be appreciated. Best, James

6 7

[largescale-sig]scaling story
by Alex Song (宋文平) 25 Oct '24

25 Oct '24

Hi everyone, we want to share our scaling story. Please refer the link: https://etherpad.opendev.org/p/large-scale-inspur Thanks, Alex Song

6 8

[nova][libvirtd] internal error: unable to execute QEMU command 'query-named-block-nodes'
by Satish Patel 22 Oct '24

22 Oct '24

Folks, I have a ceph cluster and recently I configured rbd-mirror to replicate all data to remove ceph cluster for disaster recovery. Yesterday for POC we did a hard cutover on ceph and point openstack to the new cluster. All other vms came back up fine but 2 vms stuck in this error state in libvirt logs 2024-01-31 22:44:37.591+0000: 474597: error : qemuMonitorJSONCheckErrorFull:399 : internal error: unable to execute QEMU command 'query-named-block-nodes': cannot read image start for probe: Permission denied If it's a wider issue than it should impact all the vms but why only two VM stuck and not starting and libvirt giving me this error in logs nova-compute logs also showing same error 2024-01-31 22:44:40.925 7 INFO nova.compute.manager [None req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00 ca5c652478c7429e964257990800e9cb - - default default] [instance: 2de0f880-77c7-4d2c-9e01-898c57ad3693] Successfully reverted task state from powering-on on failure for instance. 2024-01-31 22:44:40.944 7 ERROR oslo_messaging.rpc.server [None req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00 ca5c652478c7429e964257990800e9cb - - default default] Exception during message handling: libvirt.libvirtError: internal error: unable to execute QEMU command 'query-named-block-nodes': cannot read image start for probe: Permission denied

3 8

[kuryr][tc] Final call for Kuryr PTL/DPL liaisons
by Ghanshyam Mann 15 Oct '24

15 Oct '24

Hello Everyone, Kuryr project ends up being a leaderless project for the 2024.2 Dalmatian cycle[1]. There is no volunteer to serve as PTL or DPL liaisons yet. As the next step, I have proposed to mark this project Inactive: - https://review.opendev.org/c/openstack/governance/+/916945 If you would like to serve as a leader and help maintain this project, this is the right time to step up by replying to this email or commenting in gerrit change 916945. [1] https://etherpad.opendev.org/p/2024.2-leaderless#L99 Ref: https://governance.openstack.org/tc/reference/emerging-technology-and-inact… -gmann

4 10

Kolla Ansible on Ubuntu 20.04 - cloud-init & other network issues
by Tobias McNulty 10 Oct '24

10 Oct '24

Hi, I'm attempting to use Kolla Ansible 14.6.0 to deploy OpenStack Yoga on a small 3-node Ubuntu 20.04 cluster. The nodes have 128 GB RAM each, dual Xeon processors, and dual 10G Intel NICs. The NICs are connected to access ports on a 10G switch with separate VLANs for the local and external networks. All the playbooks run cleanly, but cloud-init is failing in the Ubuntu 20.04 and 22.04 VMs I attempt to boot. The VM images are unmodified from https://cloud-images.ubuntu.com/, and cloud-init works fine if I mount a second volume with user-data. The error is a timeout attempting to reach 169.254.169.254. This occurs both when booting a VM in an internal routed network and directly in an external network. I tried various neutron plugin agents (ovn, linuxbridge, and openvswitch both with and without firewall_driver = openvswitch <https://docs.openstack.org/kolla-ansible/latest/reference/networking/neutro…>) first with a clean install of the entire OS each time, all with the same result. Running tcpdump looking for 169.254.169.254 shows nothing. As a possible clue, the virtual NICs are unable to pass any traffic (e.g., to reach an external DHCP server) unless I completely disable port security on the interface (even if the associated security group is wide open). But disabling port security does not fix cloud-init (not to mention I don't really want to disable port security). Are there any additional requirements related to deploying OpenStack with Kolla on Ubuntu 20.04? This is a fairly vanilla configuration using the multinode inventory as a starting point. I tried to follow the Quick Start <https://docs.openstack.org/kolla-ansible/yoga/user/quickstart.html> as closely as possible; the only material difference I see is that I'm using the same 3 nodes for control + compute. I am using MAAS so it's easy to get a clean OS install on all three nodes ahead of each attempt. I plan to try again with the standard (non-HWE) kernel just in case, but otherwise I am running out of ideas. In case of any additional clues, here are my globals.yml and inventory file, along with the playbook I'm using to configure the network, images, VMs, etc., after bootstrapping the cluster: https://gist.github.com/tobiasmcnulty/7dbbdbc67abc08cbb013bf5983852ed6 Thank you in advance for any advice! Cheers, Tobias

5 9

[all][tc][ptls] Eventlet: Python 3.13 - Attach your seatbelt
by Herve Beraud 06 Sep '24

06 Sep '24

Hi all, As Python 3.13 is close to become a reality, new concerning incompatibilities between this new CPython version and Eventlet also become a reality: - https://github.com/eventlet/eventlet/issues/964 Fedora Python maintainers proposed a patch, but for now the problem remains unsolved. Also, many people reported bugs related to incompatibility between python3.12/dnspython/eventlet/pip-system-cert. There is an example who summarize well the problem https://github.com/eventlet/eventlet/issues/957 Do you have some visibility concerning the adoption of Python 3.13 into Openstack? Turbulences ahead... -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

4 5

[Kolla] [LetsEncrypt] Internal/External/Backend Certificates
by jjjamesg＠proton.me 05 Sep '24

05 Sep '24

Hi all, another one here that can't seem to get kolla running with letsencrypt certs, just curious as to what settings are required for kolla to pull internal/external/backend certs from letsencrypt? I've looked at the docs and have tried a few settings but keep getting errors, sure i had this working last week before i tore it down ready to deploy for production and now i just cant get it to complete the deployment because of certs, only thing i can think of is i need to run kolla-ansible -i multinode certificates but i thought that was just for self made certs? Also, what are best practices for backend tls certs, self signed or using the internal cert etc?? Is it possible to use letsencrypt at all for backend? Cheers all

3 6

[ops][kolla][kolla-ansible] Lets Encrypt Deployment on 2024.1/v18
by Forrest Fuqua 03 Sep '24

03 Sep '24

The documentation seems a little incomplete when it comes to deploying a stack with let's encrypt The attached config is after trying a few settings, but the lets encrypt container errors out every time during deployment causing haproxy to not get a proper ssl certificate the services.d entries are showing its ignoring kolla_internal_fqdn_cert and still using kolla_internal.pem in most services causing the following deployment error TASK [service-ks-register : keystone | Creating services] **************************************************************************************************************************************************** FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (5 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (4 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (3 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (2 retries left). FAILED - RETRYING: [100.70.0.1]: keystone | Creating services (1 retries left). failed: [100.70.0.1] (item=keystone (identity)) => {"action": "os_keystone_service", "ansible_loop_var": "item", "attempts": 5, "changed": false, "item": {"description": "Openstack Identity Service", "endpoints": [{"interface": "internal", "url": "https://openstack.cyberrange.rit.edu:5000"}, {"interface": "public", "url": "https://openstack.cyberrange.rit.edu:5000"}], "name": "keystone", "type": "identity"}, "module_stderr": "Failed to discover available identity versions when contacting https://openstack.cyberrange.rit.edu:5000. Attempting to parse version from URL.\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 715, in urlopen\n httplib_response = self._make_request(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 404, in _make_request\n self._validate_conn(conn)\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 1058, in _validate_conn\n conn.connect()\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connection.py\", line 419, in connect\n self.sock = ssl_wrap_socket(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/util/ssl_.py\", line 449, in ssl_wrap_socket\n ssl_sock = _ssl_wrap_socket_impl(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/util/ssl_.py\", line 493, in _ssl_wrap_socket_impl\n return ssl_context.wrap_socket(sock, server_hostname=server_hostname)\n File \"/usr/lib/python3.10/ssl.py\", line 513, in wrap_socket\n return self.sslsocket_class._create(\n File \"/usr/lib/python3.10/ssl.py\", line 1100, in _create\n self.do_handshake()\n File \"/usr/lib/python3.10/ssl.py\", line 1371, in do_handshake\n self._sslobj.do_handshake()\nssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/requests/adapters.py\", line 486, in send\n resp = conn.urlopen(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/connectionpool.py\", line 799, in urlopen\n retries = retries.increment(\n File \"/opt/ansible/lib/python3.10/site-packages/urllib3/util/retry.py\", line 592, in increment\n raise MaxRetryError(_pool, url, error or ResponseError(cause))\nurllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1021, in _send_request\n resp = self.session.request(method, url, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/requests/sessions.py\", line 589, in request\n resp = self.send(prep, **send_kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/requests/sessions.py\", line 703, in send\n r = adapter.send(request, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/requests/adapters.py\", line 517, in send\n raise SSLError(e, request=request)\nrequests.exceptions.SSLError: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/generic/base.py\", line 133, in _do_create_plugin\n disc = self.get_discovery(session,\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 605, in get_discovery\n return discover.get_discovery(session=session, url=url,\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/discover.py\", line 1459, in get_discovery\n disc = Discover(session, url, authenticated=authenticated)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/discover.py\", line 539, in __init__\n self._data = get_version_data(session, url,\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/discover.py\", line 106, in get_version_data\n resp = session.get(url, headers=headers, authenticated=authenticated)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1154, in get\n return self.request(url, 'GET', **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 930, in request\n resp = send(**kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1025, in _send_request\n raise exceptions.SSLError(msg)\nkeystoneauth1.exceptions.connection.SSLError: SSL exception connecting to https://openstack.cyberrange.rit.edu:5000: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/tmp/ansible-tmp-1720019606.1431425-10481-181523778594420/AnsiballZ_catalog_service.py\", line 107, in <module>\n _ansiballz_main()\n File \"/tmp/ansible-tmp-1720019606.1431425-10481-181523778594420/AnsiballZ_catalog_service.py\", line 99, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/tmp/ansible-tmp-1720019606.1431425-10481-181523778594420/AnsiballZ_catalog_service.py\", line 47, in invoke_module\n runpy.run_module(mod_name='ansible_collections.openstack.cloud.plugins.modules.catalog_service', init_globals=dict(_module_fqn='ansible_collections.openstack.cloud.plugins.modules.catalog_service', _modlib_path=modlib_path),\n File \"/usr/lib/python3.10/runpy.py\", line 224, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib/python3.10/runpy.py\", line 96, in _run_module_code\n _run_code(code, mod_globals, init_globals,\n File \"/usr/lib/python3.10/runpy.py\", line 86, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 211, in <module>\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 207, in main\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/module_utils/openstack.py\", line 415, in __call__\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 113, in run\n File \"/tmp/ansible_os_keystone_service_payload_m68qs7ok/ansible_os_keystone_service_payload.zip/ansible_collections/openstack/cloud/plugins/modules/catalog_service.py\", line 175, in _find\n File \"/opt/ansible/lib/python3.10/site-packages/openstack/service_description.py\", line 89, in __get__\n proxy = self._make_proxy(instance)\n File \"/opt/ansible/lib/python3.10/site-packages/openstack/service_description.py\", line 289, in _make_proxy\n found_version = temp_adapter.get_api_major_version()\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/adapter.py\", line 352, in get_api_major_version\n return self.session.get_api_major_version(auth or self.auth, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/session.py\", line 1289, in get_api_major_version\n return auth.get_api_major_version(self, **kwargs)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 497, in get_api_major_version\n data = get_endpoint_data(discover_versions=discover_versions)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 268, in get_endpoint_data\n service_catalog = self.get_access(session).service_catalog\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/base.py\", line 131, in get_access\n self.auth_ref = self.get_auth_ref(session)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/generic/base.py\", line 203, in get_auth_ref\n self._plugin = self._do_create_plugin(session)\n File \"/opt/ansible/lib/python3.10/site-packages/keystoneauth1/identity/generic/base.py\", line 155, in _do_create_plugin\n raise exceptions.DiscoveryFailure(\nkeystoneauth1.exceptions.discovery.DiscoveryFailure: Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. SSL exception connecting to https://openstack.cyberrange.rit.edu:5000: HTTPSConnectionPool(host='openstack.cyberrange.rit.edu', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)')))\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1} #config excerpts from globals.yml kolla_base_distro: "ubuntu" kolla_internal_vip_address: "100.70.0.20" # /etc/hosts updated on all hosts to have openstack.cyberrange.rit.edu point to the internal IP kolla_internal_fqdn: "openstack.cyberrange.rit.edu" kolla_external_vip_address: "129.21.246.130" kolla_external_fqdn: "openstack.cyberrange.rit.edu" kolla_enable_tls_internal: "yes" kolla_enable_tls_external: "yes" kolla_certificates_dir: "/etc/kolla/certificates" kolla_external_fqdn_cert: "{{ kolla_certificates_dir }}/haproxy.pem" kolla_internal_fqdn_cert: "{{ kolla_certificates_dir }}/haproxy.pem" #kolla_admin_openrc_cacert: "" kolla_copy_ca_into_containers: "yes" haproxy_backend_cacert: "{{ 'ca-certificates.crt' if kolla_base_distro in ['debian', 'ubuntu'] else 'ca-bundle.trust.crt' }}" haproxy_backend_cacert_dir: "/etc/ssl/certs" letsencrypt_email: "fffics(a)rit.edu" enable_letsencrypt: yes letsencrypt_cert_server: "https://acme-v02.api.letsencrypt.org/directory" # attempt to renew Let's Encrypt certificate every 12 hours letsencrypt_cron_renew_schedule: "0 */12 * * *"

3 2

PTG October 2024 Team Signup
by Kendall Nelson 29 Aug '24

29 Aug '24

Hello Everyone, Hopefully you have seen by now, the next OpenInfra PTG[1] which will take place October 21-25, 2024 virtually! Registration is now open[2]! To sign up your team, you must complete the survey[3] by August 30th at 7:00 UTC. We NEED accurate contact information (email and IRC nickname ideally) for the moderator of your team’s sessions. This is because the survey information will be used to organize the schedule signups which will be done via the PTGBot. If you are not on IRC, please get setup[4] on the OFTC network and join #openinfra-events. You are also encouraged to familiarize yourself with the PTGBot documentation[5] as well. If you have any questions, please reach out to ptg(a)openinfra.dev! Information about signing up for time slots will be sent to moderators after the team signup deadline. Continue to visit openinfra.dev/ptg for updates. -Kendall (diablo_rojo) [1] PTG Site: https://openinfra.dev/ptg/ [2] PTG Registration: http://ptg.openinfra.dev/ [3] Team Survey: https://openinfrafoundation.formstack.com/forms/oct2024_ptg_survey [4] Setup IRC: https://docs.openstack.org/contributors/common/irc.html [5] PTGBot README: https://opendev.org/openstack/ptgbot/src/branch/master/README.rst

1 2

[MAGNUM] Fedora Core OS include NVIDIA GPU drivers
by Oliver Weinmann 19 Aug '24

19 Aug '24

Hi all, I'm running OpenStack 2023.2 (Bobcat) deployed with Kolla-Ansible. Currently I'm struggling to get the NVIDIA drivers to work in a FCOS 35 image. I know FCOS 35 is EOL and outdated, but for a K8s 1.21.11 cluster it is the only version that still works with Magnum. So far I managed to get the GPU Passthrough working for two NVIDIA Tesla T4 GPUs. I have attached the GPUs to an Ubuntu instance and ran geekbench without any issues. Then I tried to install the latest NVIDIA drivers in FCOS 35 by running: sudo rpm-ostree install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$ (rpm -E %fedora).noarch.rpm https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$ (rpm -E %fedora).noarch.rpm systemctl reboot rpm-ostree install --apply-live akmod-nvidia akmods pciutils rpmdevtools xorg-x11-drv-nvidia-cuda It fails on: rpm-ostree install --apply-live akmod-nvidia akmods pciutils rpmdevtools xorg-x11-drv-nvidia-cuda error: Cannot download rpm-build-libs-4.17.0-4.fc35.x86_64.rpm: All mirrors were tried; Last error: Status code: 403 for https://fedoraproject-updates-archive.fedoraproject.org/fedora/35/x86_64/rp… I guess the binaries are no longer hosted? Running the same on FCOS 36 and latest FCOS 40 works fine. Next I have taken new images from the FCOS instances with the NVIDIA driver. openstack server image create --name fedora_coreos_36_nvidia_cuda fedora_coreos_35_latest_no_update --wait This takes ages but anyway I can successfully deploy new instances from the images with my gpu flavor and the GPU is working fine. Again tested with geekbench. But now when adding new K8s worker nodes in a fresh nodegroup on an existing K8s 1.21.11 cluster the nodes are not joining the cluster: openstack coe nodegroup create \ --node-count 1 \ --role worker-gpu \ --flavor gpuflavor \ --image fedora_coreos_36_nvidia_cuda \ $CLUSTER_ID worker-gpu I have tested the latest available images of FCOS36 and they work fine, so something must be wrong with how I created the new image. Is there some magic that needs to be done? I'm also not sure if this is the best way of doing this. Maybe there is already a ready FCOS image with nvidia drivers? I can find lots of outdated information on the internet. Generally I tend to use Vexxhost CAPI from now on, which works great. This is really just a one off attempt to have a working NVIDIA T4 GPU in a K8s 1.21.11 cluster. Best Regards, Oliver

5 16

networking-onos plugin
by Ganesan, Sathya 14 Aug '24

14 Aug '24

Good Day, Is there an alternative integration available for ONOS and Openstack since networking-onos is archived? Sathya Ganesan Geico Tech | Data Center Core Network Engineering - SDN | sganesan(a)geico.com<mailto:sganesan@geico.com> | 240.478.2121 | M-F 9a-5:30p EST Sensitivity: Company-Internal ==================== This email/fax message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution of this email/fax is prohibited. If you are not the intended recipient, please destroy all paper and electronic copies of the original message.

2 2

[rabbitmq] Maximum Number of Sessions (8192) Reached
by Eugen Block 14 Aug '24

14 Aug '24

Hi *, I can't seem to find much about rabbitmq and logind, so I wanted to ask the list if anyone has encountered the same and if so, how they dealt with it. We're supporting a Victoria cluster (installed with our own deployment method) mostly controlled by pacemaker. And on the rabbit master node I see this warning constantly: ---snip--- 2024-07-29T14:09:23.552576+02:00 control01 su: pam_unix(su:session): session opened for user rabbitmq by (uid=0) 2024-07-29T14:09:24.450657+02:00 control01 su: pam_unix(su:session): session closed for user rabbitmq 2024-07-29T14:09:24.500356+02:00 control01 su: (to rabbitmq) root on none 2024-07-29T14:09:24.502370+02:00 control01 su: pam_systemd(su:session): Failed to create session: Maximum number of sessions (8192) reached, refusing further sessions. 2024-07-29T14:09:24.502681+02:00 control01 su: pam_unix(su:session): session opened for user rabbitmq by (uid=0) 2024-07-29T14:09:25.565203+02:00 control01 su: pam_unix(su:session): session closed for user rabbitmq 2024-07-29T14:09:25.609613+02:00 control01 su: (to rabbitmq) root on none ---snip--- Looking into loginctl list-sessions, almost all of them belong to rabbitmq and they have a very old timestamp (2023). I'm aware of older systemd versions which can't handle closing sessions correctly [0], but we can't upgrade at this time. Would enabling "linger" fix this (loginctl enable-linger rabbitmq) after a reboot? During the next maintenance window I would reboot control01 and watch how the other control nodes behave wrt rabbit login-sessions. But I'm wondering if somebody already has dealt with this? Thanks for any pointers! Eugen [0] https://www.suse.com/support/kb/doc/?id=000020549

1 2

Firewall as a Service - kolla ansible 2023.1
by Thamanna Farhath 06 Aug '24

06 Aug '24

Hello OpenStack Community, I hope this message finds you well. My name is Thamanna , and I am currently working with the OpenStack Kolla-Ansible deployment version 2023.1. I am interested in deploying Firewall as a Service (FWaaS) within our OpenStack environment. However, I have noticed that FWaaS is not available in the Kolla-Ansible 2023.1 version. Could anyone please provide guidance on the following points? 1. Is there an official or recommended way to integrate FWaaS with Kolla-Ansible 2023.1? 2. Are there any specific configurations or additional components required to enable FWaaS? 3. Has anyone in the community successfully deployed FWaaS with this version, and if so, could you share your experience and any potential challenges faced? 4. Are there alternative approaches or tools that you would recommend for deploying FWaaS in an OpenStack environment? Any help or pointers to relevant documentation would be greatly appreciated. Thank you in advance for your assistance! Best regards, Thamanna mailto:thamanna.f@zybisys.com Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

3 5

Live migration fails
by Szabo, Istvan (Agoda) 06 Aug '24

06 Aug '24

Hi, I have couple of compute nodes where the live migration fails with existing vms. When I quickly spawn a vm and try live migration it works so I assume shouldn't be a big problem with the compute node. However I have many existing vms where it fails with a servername not found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 ERROR nova.conductor.tasks.migrate [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Unable to find record for source node servername on servername: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Failed to compute_task_migrate_server: Compute host servername could not be found.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Setting instance to ACTIVE state.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.672 227612 ERROR oslo_messaging.rpc.server [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Exception during message handling: ComputeHostNotFound: Compute host am-osfecn-4025 Tried with this command: nova live-migration --block-migrate id. Any idea? Thank you. ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

5 16

Assistance needed - Application Credential
by Naveen Anbarasu 05 Aug '24

05 Aug '24

Hi team, I have created an application credential for the compute service and successfully obtained a token. Access rule: "service": "compute", "method": "GET", "path": "/v2.1/servers" However, when using the token to retrieve the server list, I receive a Error 403 Forbidden error with the message: 'Policy doesn't allow os_compute_api:servers to be performed. But i have a necessary permission within the respective project (member role) How can I solve this problem? Thanks in advance

7 12

[openstack][kolla-ansible]about upgrading openstack
by Nguyễn Hữu Khôi 04 Aug '24

04 Aug '24

Hello. I read this doc about upgrading openstack https://docs.openstack.org/kolla-ansible/latest/user/operating-kolla.html But I would like to know if We install a new kolla-ansible(version to upgrade) on the new deployment host, apply changes with password.yml, global.yml and multinode and keep old /etc/kolla/config. Finally, we use kolla-ansible deploy instead of kolla-ansible upgrade. Are these steps ok? Pls correct me if I am wrong. Thank you. Regards Nguyen Huu Khoi

3 8

[election] [tc] Notification of non-candidacy; how it impacts TC diversity
by Jay Faulkner 01 Aug '24

01 Aug '24

Hi all, The nomination period for TC members / PTLs is coming up soon. I wanted to inform the community that I do not intend on standing for election for another year on the TC. Now is a good time, especially if you work somewhere other than Red Hat, to consider serving as a TC member to ensure our leadership team contains representation for all perspectives of OpenStack use. I'm happy to speak with anyone privately who is interested in running about what it's like to be on the TC, and to share some strategies I exercised to ensure the value of my TC membership was being communicated to leadership at my job. Thanks, Jay Faulkner Technical Committee Member

6 5

oslo.service
by engineer2024 01 Aug '24

01 Aug '24

Can anyone guide me how to configure and use oslo.service for nova-api service for tracing and debugging ? Thanks elinux

3 2

[cinder] Stepping down from Cinder core
by Pete Zaitcev 01 Aug '24

01 Aug '24

Hello: Because of a job change, my time available is becoming quite limited, so I'm not going to be able to contribute at the appropriate level. So, I contacted our PTL to get me removed from the core team. Thanks everyone for the help and the journey in Cinder. Greetings, Pete

3 2

nova deploy issue
by Parsa Aminian 30 Jul '24

30 Jul '24

hello when I try to reconfigure nova with kolla ansible this error will be shown : TASK [nova-cell : Waiting for nova-compute services to register themselves] ******************************************************************************************* task path: /usr/local/share/kolla-ansible/ansible/roles/nova-cell/tasks/wait_discover_computes.yml:8 fatal: [control01 -> {{ groups[nova_cell_conductor_group][0] }}]: FAILED! => { "msg": "The conditional check 'inventory_hostname == all_computes_in_batch[0]' failed. The error was: error while evaluating conditional (inventory_hostname == all_computes_in_batch[0]): 'all_computes_in_batch' is undefined\n\nThe error appears to be in '/usr/local/share/kolla-ansible/ansible/roles/nova-cell/tasks/wait_discover_computes.yml': line 8, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n- block:\n - name: Waiting for nova-compute services to register themselves\n ^ here\n" Does anyone have any idea what I should do ?

1 0

Assistance Needed for Trove Instance Creation
by Thamanna Farhath 30 Jul '24

30 Jul '24

Hello community, I am currently deploying Trove using Kolla Ansible 2023.1, following the guide provided here: https://satishdotpatel.github.io/trove-dbaas-openstack/ . Despite cleaving to the instructions, I am facing an issue where the Trove instance ends up in an error state. trove-guestagent.conf [DEFAULT] transport_url = rabbit://openstack:USI0SmaT7lPF7MJCxHPZ4alm8wiyxoihssD1VlnA@192.168.1.101:5672,openstack:USI0SmaT7lPF7MJCxHPZ4alm8wiyxoihssD1VlnA@192.168.1.102:5672,openstack:USI0SmaT7lPF7MJCxHPZ4alm8wiyxoihssD1VlnA@192.168.1.103:5672// control_exchange = trove root_grant = ALL root_grant_option = True debug = False Docker Status Check: docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f698ee14d928 mysql:5.7.29 "docker-entrypoint.s…" 18 minutes ago Up 18 minutes 0.0.0.0:3306->3306/tcp, :::3306->3306/tcp, 33060/tcp database Telnet Connection: telnet 192.168.50.78 3306 Trying 192.168.50.78... Connected to 192.168.50.78. Escape character is '^]'. J 5.7.29p+C5 }0^k[zH3mysql_native_password Syslog(SSH into the trove instance) ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)\n": Exception: Running command error: b"mysql: [Warning] Using a password on the command line interface can be insecure.ERROR trove.guestagent.datastore.service [None req-a03ff3dd-5a8f-4fd4-80d5-76e31fa16dd0 - - - - - -] Timeout while waiting for database status to change. Expected state healthy, current state is running When looking for the missing directory, I discovered it in /var/lib/mysql/mysqld.sock and /var/lib/mysql/mysqld.pid paths. How can I get an active instance?Kindly help me resolve this. Thank you in advance Disclaimer : The content of this email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error, please notify the sender and remove the messages from your system. If you are not the named addressee, it is strictly forbidden for you to share, circulate, distribute or copy any part of this e-mail to any third party without the written consent of the sender. E-mail transmission cannot be guaranteed to be secured or error free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or may contain viruses. Therefore, we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email."

3 2

[horizon] How to enable MFA using TOTP on Horizon (ZED)
by Shubham Kumar Yadav 30 Jul '24

30 Jul '24

Hi all, I was working on a MFA in openstack (ZED). Is it possible to enable MFA using TOTP on Horizon? I'm using a ZED Openstack with Ubuntu 22.04 , and I know that from the CLI point of view, you can have Multi Factor Authentication, but I don't see that implemented on Horizon. i added MFA using - https://docs.openstack.org/keystone/pike/advanced-topics/auth-totp.html but now the problem is i can use the token issued to run openstack cli but not able to login into Horizon with user and password/totp. I see The Totp support was added to Horizon in the Bobcat (2023.2) release So i would like to ask how it is possible to backport the feature in Horizon in the zed release. I would appreciate any help! Thanks!

2 1

[ironic] ironic-inspector bugfix branches
by Riccardo Pittau 30 Jul '24

30 Jul '24

Hello ironicers! With the deprecation and the migration of the ironic-inspector functionalities to the new agent interface in ironic we have decided to also stop cutting bugfix branches for the project. The last bugfix branch for ironic-inspector is bugfix/12.2 supported until December 2024. We'll continue with the normal release schedule until full retirement of the project. Thanks Ciao! Riccardo

1 0

[neutron] CI Meeting cancelled for today
by Yatin Karel 30 Jul '24

30 Jul '24

Hello Neutron Team !! I will not be able to chair the CI meeting today as I have to attend some internal meetings. Sorry for the late notice. We have some known issues in CI and all are under track with patches in gate or already merged. There are some issues in the periodic line specific to ovs-master jobs and 9-stream so will report issues for those. -- Thanks and Regards Yatin Karel

1 0

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2024.2/R-9)
by Goutham Pacha Ravi 29 Jul '24

29 Jul '24

Hello Stackers, We're counting down 9 weeks to the 2024.2 (Dalmatian) release [1]. OpenStack projects that follow the cycle-with-rc release model [2] will have their code freeze in 4 weeks. Between now and the release, the release team will communicate with us weekly to coordinate deliverables across the community. The elections for OpenStack PTLs and Technical Committee members are around the corner as well, so now is a good time to ensure that you are eligible to vote in these elections! Please navigate to: https://civs1.civs.us/cgi-bin/opt_in.pl Enter your preferred email address on Gerrit and request activation. It only takes a minute! :) The following are updates from the OpenStack Technical Committee during the past week. === Weekly Meeting === The TC had a productive meeting via IRC on OFTC's #openstack-tc channel last week [3]. We discussed follow-ups for the eventlet removal goal proposal [4] and recent updates to inactive and emerging OpenStack projects. We spent some time assessing gate health concerns, including an issue with a change to `setuptools` that requires us to bump the upper constraint on the "packaging" library in stable/2023.1 [5]. This was important to get some CVE fixes landed in various OpenStack code repositories. Finally, we refreshed our understanding of the TC's house rules. To summarize: - Changes to the TC charter require at least 2/3 of the TC voting in favor, while governance motions require at least 1/3 of the TC to vote in favor. - TC members can vote positively, negatively, or abstain (using the “RollCall-Vote” in Gerrit). - Decisions need more positive votes than negative votes (ties mean the proposal is rejected). - Any motion or charter-change is deliberately open at least a week after it was last proposed/updated, allowing the TC and the wider community to participate fully. - Significant governance changes take a while to merge because of the importance of ensuring full TC and community engagement in votes. Our next weekly meeting is tomorrow (2024-07-30) at 1800 UTC via IRC on OFTC's #openstack-tc channel. You can find the agenda on the OpenStack TC wiki page [6]. Please consider joining us and remember, you can edit the wiki and propose new topics! === Governance Proposals === ==== Merged ==== - Add `os-test-images` under glance | https://review.opendev.org/c/openstack/governance/+/925044 ==== Open for review/discussion ==== - [Follow-up] Fix typos in the eventlet proposal | https://review.opendev.org/c/openstack/governance/+/924944 - Update criteria for inactive projects to become active again | https://review.opendev.org/c/openstack/governance/+/921500 - Follow up on the affiliation diversity handling | https://review.opendev.org/c/openstack/governance/+/923876 - Inactive state extensions: Monasca | https://review.opendev.org/c/openstack/governance/+/923919 === Upcoming Events === - 2024-08-12: Call for Proposals due for the OpenInfra Days NA [7] - 2024-08-14: Nominations open for 2025.1 OpenStack PTL+TC elections - 2024-08-24: OpenInfra Days, Vietnam: https://2024.vietopeninfra.org/ - 2024-08-29: Dalmatian-3 milestone - 2024-09-03: OpenInfra Summit Asia, Suwon The OpenStack Technical Committee serves you! We're always willing to chat and would love feedback: === How to contact the TC === You can reach out to the TC in multiple ways: 1. Email: Send an email with the tag [tc] on this email list. 2. Ping us using the 'tc-members' keyword on the #openstack-tc IRC channel. 3. Join us at our weekly meeting: The Technical Committee meets every week on Tuesdays at 1800 UTC [7]. Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] Release Schedule: https://releases.openstack.org/dalmatian/schedule.html [2] OpenStack cycle-with-rc model: https://releases.openstack.org/reference/release_models.html#cycle-with-rc [3] TC Meeting IRC Logs, 2024-07-23: https://meetings.opendev.org/meetings/tc/2024/tc.2024-07-23-18.00.log.html [4] Remove Eventlet From Openstack: https://review.opendev.org/c/openstack/governance/+/902585 [5] Update packaging constraint in stable/2023.1: https://review.opendev.org/c/openstack/requirements/+/924764 [6] TC Meeting, 2024-07-30: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [7] Call for Proposals, OpenInfra Days NA: https://docs.google.com/forms/d/e/1FAIpQLSeFQBL1qGNegDWDHCVTLtu_wLf06w7JJkc…

1 0

Cinder Bug Report 2024-07-29
by Jon Bernard 29 Jul '24

29 Jul '24

Hello Argonauts, Cinder Bug Meeting Etherpad <https://etherpad.opendev.org/p/cinder-bug-squad-meeting> Undecided - When restoring volume using the imported backup, it is always scheduled to the same host. <https://bugs.launchpad.net/cinder/+bug/2074058> - Status: In Progress (assigned) - The cinder-backup swift driver connects to swift in a different region. <https://bugs.launchpad.net/cinder/+bug/2074126> - Status: In Progress (assigned) - allow_qcow2_backing_file flag cannot be passed into some image_utils operations <https://bugs.launchpad.net/cinder/+bug/2074377> - Status: New (unassigned) Thanks, -- Jon

1 0

[nova][ops] Deprecate AMI/direct kernel boot support
by Dan Smith 29 Jul '24

29 Jul '24

Hi all, tl;dr: We really need ops feedback about any direct need to retain this feature. Since the early days, Nova has supported the notion of "direct linux kernel boot" (with libvirt/kvm at least). This means each bootable image in Glance is actually three parts: a disk image, a ramdisk image, and a kernel. Booting the instance involves putting the kernel and ramdisk into memory in a special way that the kernel can immediately start booting instead of doing things like running the bootloader like a normal system does. This is present in Nova because of long-ago Amazon compatibility, as this is how Xen paravirt guests used to work for them. I think that in the early days of Nova, compatibility with the _actual_ images used on Amazon was important, but I think that is no longer something we need to care about today, just as we have deprecated EC2 API compatibility. It should also be noted that some of it is broken, and the recent CVE fixes have broken it like three times in the last 30 days. Continuing to support it makes the code more complex and fragile, and arguably less secure. So, I'm asking for operator feedback on this question: Do you need this feature, and if so, why? Speak now or expect to see it deprecated for removal very soon. Thanks! --Dan

3 5

[neutron] Bug deputy report (week starting on July 22)
by Bernard Cafarelli 29 Jul '24

29 Jul '24

Hey neutrinos, here is the list of bugs reported during 2024 week 30 Most are in progress (or already fixed), except these: https://bugs.launchpad.net/bugs/2074207 (endless loop in DHCP agent), https://bugs.launchpad.net/bugs/2073872 (L3 HA routes issue in Yoga) and https://bugs.launchpad.net/neutron/+bug/2074056 (security groups doc/API) Critical * Couple of jobs broken as rpc-server not enabled since patch 924473 - https://bugs.launchpad.net/bugs/2073844 Mentioning for visibility, this was fixed in devstack with https://review.opendev.org/c/openstack/devstack/+/924693 * compute tests involving shelve/snapshots/rescue fails with Image is unacceptable: Image content does not match disk_format - https://bugs.launchpad.net/bugs/2073944 Also for visibility, fixed in nova with https://review.opendev.org/c/openstack/nova/+/924866 High * Switch from distributed to centralized Floating IPs breaks connectivity to the existing FIPs - https://bugs.launchpad.net/neutron/+bug/2073987 Fix in progress by slaweq - https://review.opendev.org/c/openstack/neutron/+/925007 * DHCP agent makes endless attempts to configure a network with MTU < 1280 - https://bugs.launchpad.net/bugs/2074207 Related/follow-up to similar https://bugs.launchpad.net/neutron/+bug/1988069, unassigned Medium * ``ProcessMonitor._check_child_processes`` cannot release the GIL inside a locked method - https://bugs.launchpad.net/bugs/2073743 Patch by Rodolfo - https://review.opendev.org/c/openstack/neutron/+/924581 * [eventlet-deprecation] Reduce the ``IpConntrackManager`` process pool to a single thread - https://bugs.launchpad.net/neutron/+bug/2073745 Patch by Rodolfo - https://review.opendev.org/c/openstack/neutron/+/924582 * "Tagging" extension does not initialize the policy enforcer - https://bugs.launchpad.net/bugs/2073782 Patch by Rodolfo - https://review.opendev.org/c/openstack/neutron/+/924656 * "Tagging" extension cannot add tags with charater "/" - https://bugs.launchpad.net/bugs/2073836 Patches by Rodolfo - https://review.opendev.org/c/openstack/neutron-lib/+/924700 and https://review.opendev.org/c/openstack/neutron/+/924724 * Neutron L3 agent not create ECMP in HA router but in single mode is ok - https://bugs.launchpad.net/bugs/2073872 L3-HA bug reported on Yoga, needs some eyes * IPv6 dns nameservers described with their scope on the IP are not supported - https://bugs.launchpad.net/bugs/2073894 Elvira working on it (fix in oslo.utils) * Subnet pool attribute updates fail when using address scopes - https://bugs.launchpad.net/neutron/+bug/2074092 Possible regression from https://review.opendev.org/c/openstack/neutron/+/667511, fix discussed https://review.opendev.org/c/openstack/neutron/+/924984 * [OVN] Trunk subports should have the same VIF type as the parent port - https://bugs.launchpad.net/neutron/+bug/2074185 Assigned to Rodolfo * OVN maintenance tasks may be delayed 10 minutes in the podified deployment - https://bugs.launchpad.net/bugs/2074209 Assigned to Slaweq Low * Invalid documented security group rule protocol "any" - https://bugs.launchpad.net/neutron/+bug/2074056 Sounds like low-hanging fruit on documentation and security groups * DB semantic violation seen in neutron-tempest-plugin-openvswitch logs deleting router - https://bugs.launchpad.net/bugs/2074312 Assigned to Brian Wishlist * [RFE] Extend "port list" command to show the trunk subports - https://bugs.launchpad.net/bugs/2074187 -- Bernard Cafarelli

1 0

[release][ptl] 2025.1 Epoxy release schedule - review request from teams
by Elõd Illés 29 Jul '24

29 Jul '24

Hi, Let me share the proposed 2025.1 Epoxy release schedule [1] (see the generated page [2] - for Patch Set 2 !!!). PTLs, teams, please review it and let us know your opinions! [1] https://review.opendev.org/c/openstack/releases/+/924671 [2] https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/… Cheers, Előd Illés irc: elodilles @ #openstack-release

1 0

how upgrade OpenStack version 5.8.0 to 6.1.0
by kjme001＠gmail.com 28 Jul '24

28 Jul '24

Hello I have OpenStack version 5.8.0, how do I upgrade it to a newer version, e.g. to version 6.1.0? thanks for your help regards

5 6

[nova] rescheduler
by Alex Song (宋文平) 26 Jul '24

26 Jul '24

Hi all, Is the reschedule flow only for building instance? Why evacuate and migration hasn’t? Thanks in advance Alex song

3 2

[qa] Core lists cleanup
by Martin Kopec 26 Jul '24

26 Jul '24

Hi everyone, we're planning to clean up the QA core lists, which have grown over time, to ensure they contain only active contributors. This will help anyone seeking a core reviewer for their patch know whom to contact. A core reviewer's activity will be assessed based on a 2-year period of making reviews. In other words, we will remove all users who haven't made any reviews in the respective repositories in the past 2 years. The cleanup will focus on these 3 core lists: * tempest-core * devstack-core * qa-release We're going to assess the activity and make the changes not sooner than August 12. If this cleanup affects you and you still wish to be included in one of the above core lists and work in a core reviewer capacity, please let me know. Thanks, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 0

[neutron] Drivers meeting 26.07.2024 cancelled
by Sławek Kapłoński 26 Jul '24

26 Jul '24

Hi, There is no agenda for today's meeting so I am cancelling it. Have a great Friday and weekend. See You on the meeting next week :) -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

keystone credentials
by engineer2024 25 Jul '24

25 Jul '24

Hi all, We are receiving this message while executing keystone playbook to upgrade from yoga to zed through openstack ansible: ---- FAILED! => {"changed": true, "cmd": ["/openstack/venvs/keystone-26.3.0/bin/keystone-manage", "credential_migrate", "--keystone-user", "keystone", "--keystone-group", "keystone"], "msg": "non-zero return code", "rc": 1,} --- Then I tried credential_setup to setup new fernet keys which succeeded, but when trying to do credential_migrate, it is still failing with the above message. The debug log shows something like "unable to decrypt; use the same primary key used for encryption" I even copied the newly setup keys to the other two containers. Stil it does not work. As a last resort , I have destroyed the three keystone containers and recreated and ran the playbook. Then it worked. But in our scenario, this caused downtime for the apps using this keystone setup which attracted an RCA requirement. So, I want to know how to avoid downtime as in a normal upgrade , when creating/destroying containers and the reason for the above error and the workaround... Appreciate your time. Thanks elinux

1 1

Upcoming Student Opportunities
by Kendall Nelson 25 Jul '24

25 Jul '24

Hello Everyone, I have a number of universities looking for projects and mentors for students in the coming semester. Three universities in particular - DIckinson University, Boston University, and the University of Jos in Nigeria. Dickinson University's course takes place over the whole academic year (two semesters) starting the start of September. Students will work in groups of 2-4 with the actual contribution period being October through May. Most students will be in their last year of a Bachelor's degree in CS. Boston University's course takes place across one semester. Groups of students will be 3-5 people and students will also be in their last year of a Bachelor's degree in CS. Students will make their project selections early September and begin contributing soon after. It's likely this course will also take place next semester. *For both universities, I need project proposals and volunteers (if interested of course), by the 23 of August. * The University of Jos partnership is more internship based and has a more flexible timeline. I am still gathering more details on what this will look like, but I wanted to give you all a heads up. If you have questions about any of these opportunities, please let me know!! Likewise, if you're interested, please let me know and I can support you in getting a project proposal together. -- Kendall Nelson *Senior Upstream Developer Advocate* The OpenInfra Foundation <https://openinfra.dev/>

1 0

update ubuntu and nova does not work
by kjme001＠gmail.com 25 Jul '24

25 Jul '24

update ubuntu and nova does not work During the update of ubuntu I selected option Y, as below and broke one node, the new service starts but does not work from the cluster side, what can I do now to fix it? Configuration file '/etc/nova/nova.conf' ==> Modified (by you or by a script) since installation. ==> Package distributor has shipped an updated version. What would you like to do about it ? Your options are: Y or I : install the package maintainer's version N or O : keep your currently-installed version D : show the differences between the versions Z : start a shell to examine the situation The default action is to keep your current version. *** nova.conf (Y/I/N/O/D/Z) [default=N] ?

2 1

[manila] Proposing Kiran Pawar to manila core
by Carlos Silva 25 Jul '24

25 Jul '24

Hello everyone, I would like to add Kiran Pawar to the OpenStack Manila core team. Kiran has been contributing with reviews [1], new features, bug fixes [2] to the OpenStack Manila components for several cycles and has clearly acquired knowledge on many areas of Manila. He has designed and worked in the implementation of several features in the core of Manila, such as share backups, deferred deletion for shares and snapshots, and most recently has been working on shares encryption and integrating Manila with Barbican. He's clearly demonstrated understanding not only on the manila core, but also in the integrations with share drivers. The participation on PTGs and other events evolving Manila has also been strong, and very impactful. Kiran also brings knowledge on large cloud environments, having worked closely on large deployments and finding solutions for their problems. This addition also means more diversity among the Manila core group, and maintaining a healthy community with different companies sharing the responsibility of maintaining an open source project. Kiran brings a lot of experience with cloud environments to the table and has been helpful on reviews and contributing with a lot of code to Manila, making him a good candidate to join the core team. In case there are no objections, I will add him to the core team in 1 week from now. [1] https://bit.ly/manila-contribution-60-days [2] https://bit.ly/kpdev-code-contributions-to-manila Thank you! Regards, carloss

4 4

[manila] Proposing Ashley Rodrigues to python-manilaclient and manila-ui core
by Carlos Silva 25 Jul '24

25 Jul '24

Hello everyone, I would like to nominate Ashley Rodriguez as python-manilaclient and manila-ui core. Ashley started contributing to OpenStack Manila more than three years ago, and managed to work on a couple of features such as the integration of Manila with OpenStack SDK, metadata for shares and share snapshots, several enhancements to the Ceph drivers testing on CI and several other bug fixes [1], which helped her in acquiring precious knowledge on the code bases, testing and providing feedback. Ashley has also been working as a mentor for many mentorship initiatives we have been promoting in Manila (Outreachy, NDSU, GHC and CMU), where we focused on different repositories, but she always demonstrated the ability to guide the interns providing insightful reviews, debugging and demonstrating good knowledge on python-manilaclient and manila-ui. This addition to the core team is one more step into maintaining a healthy community, as well as a very good way to recognize all the work and dedication Ashley has been putting on these repositories in different ways. In case there are no objections, I will add Ashley to the python-manilaclient and manila-ui core sub groups in 1 week from now. [1] https://review.opendev.org/q/owner:ashrod98@redhat.com [2] https://bit.ly/manila-contribution-60-days Thank you! Regards, carloss

4 4

Updating Instance Flavor Properties in OpenStack Cluster Without Downtime
by Mia smith 25 Jul '24

25 Jul '24

Hello OpenStack Community, In our OpenStack cluster, we recently updated some flavor properties. As a result, we need to apply these changes to all currently active instances that were created with the old flavor properties. We are looking for a solution that allows us to update these instance properties without causing any downtime or needing to resize the instances. Could anyone provide suggestions on how to achieve this? Is there a recommended method or ad-hoc command that can be used to safely update the properties directly in the database? Thank you for your assistance!

3 8