May 2024 - openstack-discuss - lists.openstack.org

[tc][all][monasca ] Retiring the Monasca project
by Ghanshyam Mann 16 Feb '25

16 Feb '25

Hello everyone, As you might know, the Monasca project is not active and was marked 'inactive' since 2024.1 cycle[1]. In this cycle also, this project is leaderless[2] and no maintainer stepped up to maintain it. As discussed in the Leaderless project etherpad[2], the next step is to retire the project. I'm sending this email to propose the Monasca retirement and based on the response, I will proceed next. [1] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [2] https://etherpad.opendev.org/p/2024.2-leaderless#L72 -gmann

6 17

[ironic]About the OS supported by Diskimage-builder
by Junya Noguchi (Fujitsu) 03 Feb '25

03 Feb '25

To whom it may concern. My name is Junya Noguchi. I have some questions about the supported OS for Diskimage-builder operation. Please respond. References https://docs.openstack.org/ironic-python-agent-builder/latest/admin/dib.html Does it mean that the OS that says "work but receive only limited testing and support" works but is not supported? References https://docs.openstack.org/diskimage-builder/latest/ This document describes RHEL and other operating systems that are not supported in the previous question. Are these operating systems supported? Also, are there any plans to support more operating systems in the future? Best regards, Junya Noguchi

5 7

[glance][ops] proposal to remove the glance-scrubber
by Brian Rosmaita 20 Jan '25

20 Jan '25

Hello Glance users, The Glance project team wants to determine whether anyone is using the glance-scrubber utility [0]. If not, we are thinking of deprecating it this cycle to be removed in the appropriate future cycle. If you have not previously heard of the glance-scrubber, that's most likely because it's only useful for particular deployment configurations of which yours is not one. See the doc [0] for details if you are curious. Our suspicion is that the glance-scrubber may have outlived its usefulness for modern OpenStack configurations. (The last time I remember it being discussed at a PTG was during the snowstorm in Dublin, which I believe was the Rocky development cycle). On the other hand, the team also has some ideas for improving the scrubber, but there's no point doing that if it's not actually being used in the first place. So if removal of the glance-scrubber would negatively affect you, please respond to this email so we can get some data points. Thanks! [0] https://docs.openstack.org/glance/latest/cli/glancescrubber.html

2 2

[cinder][kubernetes] pvc stuck in reserved state
by Eugen Block 10 Jan '25

10 Jan '25

Hi, we have a couple of k8s clusters in our openstack environment (Victoria) and are facing some problems when pods get moved to different worker nodes. The PVCs get stuck in "reserved" state, and they only get attached after resetting the volumes to "available" state. First I thought it was due to the missing multiattach volume type which seems to work fine in our customer's cluster. I also found the "duplicate connector" error messages in the nova-compute.log: > Unable to update attachment.(Invalid volume: duplicate connectors > detected on volume But after adding the multiattach volume type and retyping the affected volumes, the PVCs still get stuck in "reserved" state while I don't see any error messages on the hypervisors anymore. So it seems that something has changed/improved, but now I don't have a trace where to look next. Could this be a cinder-csi driver issue or something? Our cinder backend is ceph (Pacific). Not sure which information could help so please ask for anything I can provide. Any pointers are highly appreciated! Thanks! Eugen

1 2

ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version
by Swogat Pradhan 24 Dec '24

24 Dec '24

Hi, I am currently trying to deploy openstack wallaby using tripleo arch. I created the network jinja templates, ran the following commands also: #openstack overcloud network provision --stack overcloud --output networks-deployed-environment.yaml custom_network_data.yaml # openstack overcloud network vip provision --stack overcloud --output vip-deployed-environment.yaml custom_vip_data.yaml # openstack overcloud node provision --stack overcloud --overcloud-ssh-key /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml and used the environment files in the openstack overcloud deploy command: (undercloud) [stack@hkg2director ~]$ cat deploy.sh #!/bin/bash THT=/usr/share/openstack-tripleo-heat-templates/ CNF=/home/stack/ openstack overcloud deploy --templates $THT \ -r $CNF/templates/roles_data.yaml \ -n $CNF/workplace/custom_network_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e $CNF/templates/node-info.yaml \ -e $CNF/templates/scheduler-hints.yaml \ -e $CNF/workplace/networks-deployed-environment.yaml \ -e $CNF/workplace/vip-deployed-environment.yaml \ -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ -e $CNF/workplace/custom-net-bond-with-vlans.yaml Now when i run the ./deploy.sh script i encounter an error stating: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command.: tripleoclient.exceptions.InvalidConfiguration: Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command. 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END return value: 1 Can someone tell me where the mistake is? With regards, Swogat Pradhan

4 4

[kolla] [openeuler] CI jobs status
by Michał Nasiadka 26 Nov '24

26 Nov '24

Hello Koalas, OpenEuler jobs in master (to be Antelope) are failing for a long time. If no one is interested in maintaining OpenEuler support in Kolla-Ansible - I will propose dropping it’s support on Wednesday Kolla meeting. Best regards, Michal

4 3

[horizon][ceph] object storage container storage policy missing
by Satish Patel 25 Nov '24

25 Nov '24

Folks, Openstack Release: Zed Recently I have integrated ceph RGW service with openstack/keystone and testing with command line tool "openstack container create foo1" and it works but when I go to Horizon UI I am seeing it doesn't have storage policy dropdown list and because of that I am not able to create container or upload file from Horizon GUI https://ibb.co/KhYW6Rd It works with commands like but not horizon. $ openstack container create foo1 +---------------------------------------+-----------+------------------------------------------------+ | account | container | x-trans-id | +---------------------------------------+-----------+------------------------------------------------+ | AUTH_d4d82849885e4126aef402c408ad2c5c | foo1 | tx00000c057454841b13497-0065b558b4-42ff2eb-ahm | +---------------------------------------+-----------+------------------------------------------------+ $ openstack container list +------+ | Name | +------+ | foo1 | +------+

2 2

[ironic] Having issue with deploying ISO image to node
by James Leong 18 Nov '24

18 Nov '24

Hi all, I am trying to play around with ironic, and I am having an issue with deploying an Ubuntu 20.04 ISO image onto my bare metal node. Since I am quite new to bare metal, I am not entirely sure what went wrong as I couldn't find any error messages shown on my log file (ironic-conductor, ironic-api, ironic_neutron_agent, and nova-compute-ironic). This might be my guess; the ironic inspection is hitting timeout, which is causing the deployment to timeout. However, I have no idea why the inspection is failing at timeout. Any help would be appreciated. Best, James

6 7

[nova][libvirtd] internal error: unable to execute QEMU command 'query-named-block-nodes'
by Satish Patel 22 Oct '24

22 Oct '24

Folks, I have a ceph cluster and recently I configured rbd-mirror to replicate all data to remove ceph cluster for disaster recovery. Yesterday for POC we did a hard cutover on ceph and point openstack to the new cluster. All other vms came back up fine but 2 vms stuck in this error state in libvirt logs 2024-01-31 22:44:37.591+0000: 474597: error : qemuMonitorJSONCheckErrorFull:399 : internal error: unable to execute QEMU command 'query-named-block-nodes': cannot read image start for probe: Permission denied If it's a wider issue than it should impact all the vms but why only two VM stuck and not starting and libvirt giving me this error in logs nova-compute logs also showing same error 2024-01-31 22:44:40.925 7 INFO nova.compute.manager [None req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00 ca5c652478c7429e964257990800e9cb - - default default] [instance: 2de0f880-77c7-4d2c-9e01-898c57ad3693] Successfully reverted task state from powering-on on failure for instance. 2024-01-31 22:44:40.944 7 ERROR oslo_messaging.rpc.server [None req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00 ca5c652478c7429e964257990800e9cb - - default default] Exception during message handling: libvirt.libvirtError: internal error: unable to execute QEMU command 'query-named-block-nodes': cannot read image start for probe: Permission denied

3 8

[kuryr][tc] Final call for Kuryr PTL/DPL liaisons
by Ghanshyam Mann 15 Oct '24

15 Oct '24

Hello Everyone, Kuryr project ends up being a leaderless project for the 2024.2 Dalmatian cycle[1]. There is no volunteer to serve as PTL or DPL liaisons yet. As the next step, I have proposed to mark this project Inactive: - https://review.opendev.org/c/openstack/governance/+/916945 If you would like to serve as a leader and help maintain this project, this is the right time to step up by replying to this email or commenting in gerrit change 916945. [1] https://etherpad.opendev.org/p/2024.2-leaderless#L99 Ref: https://governance.openstack.org/tc/reference/emerging-technology-and-inact… -gmann

4 10

Kolla Ansible on Ubuntu 20.04 - cloud-init & other network issues
by Tobias McNulty 10 Oct '24

10 Oct '24

Hi, I'm attempting to use Kolla Ansible 14.6.0 to deploy OpenStack Yoga on a small 3-node Ubuntu 20.04 cluster. The nodes have 128 GB RAM each, dual Xeon processors, and dual 10G Intel NICs. The NICs are connected to access ports on a 10G switch with separate VLANs for the local and external networks. All the playbooks run cleanly, but cloud-init is failing in the Ubuntu 20.04 and 22.04 VMs I attempt to boot. The VM images are unmodified from https://cloud-images.ubuntu.com/, and cloud-init works fine if I mount a second volume with user-data. The error is a timeout attempting to reach 169.254.169.254. This occurs both when booting a VM in an internal routed network and directly in an external network. I tried various neutron plugin agents (ovn, linuxbridge, and openvswitch both with and without firewall_driver = openvswitch <https://docs.openstack.org/kolla-ansible/latest/reference/networking/neutro…>) first with a clean install of the entire OS each time, all with the same result. Running tcpdump looking for 169.254.169.254 shows nothing. As a possible clue, the virtual NICs are unable to pass any traffic (e.g., to reach an external DHCP server) unless I completely disable port security on the interface (even if the associated security group is wide open). But disabling port security does not fix cloud-init (not to mention I don't really want to disable port security). Are there any additional requirements related to deploying OpenStack with Kolla on Ubuntu 20.04? This is a fairly vanilla configuration using the multinode inventory as a starting point. I tried to follow the Quick Start <https://docs.openstack.org/kolla-ansible/yoga/user/quickstart.html> as closely as possible; the only material difference I see is that I'm using the same 3 nodes for control + compute. I am using MAAS so it's easy to get a clean OS install on all three nodes ahead of each attempt. I plan to try again with the standard (non-HWE) kernel just in case, but otherwise I am running out of ideas. In case of any additional clues, here are my globals.yml and inventory file, along with the playbook I'm using to configure the network, images, VMs, etc., after bootstrapping the cluster: https://gist.github.com/tobiasmcnulty/7dbbdbc67abc08cbb013bf5983852ed6 Thank you in advance for any advice! Cheers, Tobias

5 9

Live migration fails
by Szabo, Istvan (Agoda) 06 Aug '24

06 Aug '24

Hi, I have couple of compute nodes where the live migration fails with existing vms. When I quickly spawn a vm and try live migration it works so I assume shouldn't be a big problem with the compute node. However I have many existing vms where it fails with a servername not found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 ERROR nova.conductor.tasks.migrate [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Unable to find record for source node servername on servername: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Failed to compute_task_migrate_server: Compute host servername could not be found.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Setting instance to ACTIVE state.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.672 227612 ERROR oslo_messaging.rpc.server [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Exception during message handling: ComputeHostNotFound: Compute host am-osfecn-4025 Tried with this command: nova live-migration --block-migrate id. Any idea? Thank you. ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

5 16

[openstack][kolla-ansible]about upgrading openstack
by Nguyễn Hữu Khôi 04 Aug '24

04 Aug '24

Hello. I read this doc about upgrading openstack https://docs.openstack.org/kolla-ansible/latest/user/operating-kolla.html But I would like to know if We install a new kolla-ansible(version to upgrade) on the new deployment host, apply changes with password.yml, global.yml and multinode and keep old /etc/kolla/config. Finally, we use kolla-ansible deploy instead of kolla-ansible upgrade. Are these steps ok? Pls correct me if I am wrong. Thank you. Regards Nguyen Huu Khoi

3 8

[kolla-ansible][letsencrypt] containers are running but not getting certificates.
by Jonathan Proulx 03 Jul '24

03 Jul '24

Hi All, I'm trying to do a test multinode deploy using 2023.2 I have letsencrypt_webserver and letsencrypt_lego contsainers running and I'm seeing random traffic in the /var/log/kolla/letsencrypt/letsencrypt-webserver-access.log so fairly confident they're plumbed through to the public internet properly, but I don't seem to be getting certificates. how can I trigger a renewal attempt so I can maybe see what I've screwed up? Thanks, -Jon -- Jonathan Proulx (he/him) Sr. Technical Architect The Infrastructure Group MIT CSAIL

3 6

[barbican][openstack-helm][tacker] Request to Include Barbican Installation Guide in OpenStack-Helm Documentation
by JUSO WATARU(十楚　航) 21 Jun '24

21 Jun '24

Hello, We are presently in the process of enhancing the official OpenStack-Helm documentation by including a procedural guide for deploying Tacker using helmchart. Through discussions with members of the openstack-helm community, it has been emphasized that Barbican is an indispensable component for Tacker. Hence, it would be highly beneficial for users to have comprehensive instructions for installing Barbican with helmchart included in the official OpenStack-Helm documentation [1]. Consequently, we suggest that the Barbican community integrate the installation guide for the Barbican component into the official OpenStack-Helm documentation. This addition is poised to significantly enhance the clarity and accessibility of deployment procedures for users. We eagerly welcome your opinions and feedback on this proposal, as we aim to ensure that the documentation for openstack deployment with helmchart meets the needs of our community. [1] https://docs.openstack.org/openstack-helm/latest/install/openstack.html Regards, Wataru

2 2

Issue in Stack.sh
by Lohit S 13 Jun '24

13 Jun '24

How to fix this issue while stack.sh is executed +functions-common:_run_under_systemd:1655 sudo systemctl start devstack(a)q-svc.service +functions-common:run_process:1687 time_stop run_process +functions-common:time_stop:2421 local name +functions-common:time_stop:2422 local end_time +functions-common:time_stop:2423 local elapsed_time +functions-common:time_stop:2424 local total +functions-common:time_stop:2425 local start_time +functions-common:time_stop:2427 name=run_process +functions-common:time_stop:2428 start_time=1716869530330 +functions-common:time_stop:2430 [[ -z 1716869530330 ]] ++functions-common:time_stop:2433 date +%s%3N +functions-common:time_stop:2433 end_time=1716869534546 +functions-common:time_stop:2434 elapsed_time=4216 +functions-common:time_stop:2435 total=16163 +functions-common:time_stop:2437 _TIME_START[$name]= +functions-common:time_stop:2438 _TIME_TOTAL[$name]=20379 +lib/neutron:start_neutron_service_and_check:640 neutron_url= http://192.168.122.150:9696/ +lib/neutron:start_neutron_service_and_check:642 is_service_enabled tls-proxy +functions-common:is_service_enabled:2077 return 1 +lib/neutron:start_neutron_service_and_check:646 '[' '!' -z networking ']' +lib/neutron:start_neutron_service_and_check:647 neutron_url= http://192.168.122.150:9696/networking +lib/neutron:start_neutron_service_and_check:649 echo 'Waiting for Neutron to start...' Waiting for Neutron to start... +lib/neutron:start_neutron_service_and_check:651 local 'testcmd=wget --no-proxy -q -O- http://192.168.122.150:9696/networking' +lib/neutron:start_neutron_service_and_check:652 test_with_retry 'wget --no-proxy -q -O- http://192.168.122.150:9696/networking' 'Neutron did not start' 60 +functions-common:test_with_retry:2359 local 'testcmd=wget --no-proxy -q -O- http://192.168.122.150:9696/networking' +functions-common:test_with_retry:2360 local 'failmsg=Neutron did not start' +functions-common:test_with_retry:2361 local until=60 +functions-common:test_with_retry:2362 local sleep=0.5 +functions-common:test_with_retry:2364 time_start test_with_retry +functions-common:time_start:2407 local name=test_with_retry +functions-common:time_start:2408 local start_time= +functions-common:time_start:2409 [[ -n '' ]] ++functions-common:time_start:2412 date +%s%3N +functions-common:time_start:2412 _TIME_START[$name]=1716869535329 +functions-common:test_with_retry:2365 timeout 60 sh -c 'while ! wget --no-proxy -q -O- http://192.168.122.150:9696/networking; do sleep 0.5; done' +functions-common:test_with_retry:2366 die 2366 'Neutron did not start' +functions-common:die:285 local exitcode=0 [Call Trace] ./stack.sh:1322:start_neutron_service_and_check /home/lohit22/opensource/openstack/devstack/Org/devstack/lib/neutron:652:test_with_retry /home/lohit22/opensource/openstack/devstack/Org/devstack/functions-common:2366:die [ERROR] /home/lohit22/opensource/openstack/devstack/Org/devstack/functions-common:2366 Neutron did not start exit_trap: cleaning up child processes Error on exit

2 2

[openstack charm][tc] openstack-charms still need PTL appointment
by Ghanshyam Mann 11 Jun '24

11 Jun '24

Hello Felipe, openstack-charm team, I tried to reach out to you on IRC but could not connect. This is regarding openstack-charm PTL appointment for the 2024.2 cycle. I am not sure if you noticed or not but there are some comments for you to fix on your PTL appointment. Please check the below Gerrit change and fix the same or let us know if there is a change in the plan. - https://review.opendev.org/c/openstack/governance/+/914254 -gmann

2 3

[swift] clean up orphan segment parts
by Rafael Tavares Oliveira 07 Jun '24

07 Jun '24

Hey there! We've been using swift and we came across this bug https://bugs.launchpad.net/swift/+bug/1813202. We decided to work it out and have an ongoing patch to solve this issue alongside ronaldo.maia(a)luizalabs.com, we would like to share and get some feedback before we submit it. To solve this, we took inspiration in obj/expirer.py: In S3 controllers we register task objects in an internal account, each task object points to an orphaned slo object, which can be created in two situations, when a multipart-upload is not completed or when a multipart object is overwritten. These tasks will be checked by the daemon (a new one we made called "s3-slo-expirer") after some time and deleted from the base accounts. Rafael

2 2

[all][tc] Eventlet migration: Call to action
by Jay Faulkner 07 Jun '24

07 Jun '24

Hi all, As you may have read in Goutham's TC summary, the existing eventlet goal[0] is unlikely to gain consensus to merge. This appears to be due to the difference in needs between OpenStack projects, and belief that it's unlikely a single solution would serve them all. Our current situation is: 1) Eventlet usage must be discontinued. The temporary maintainers are just that -- temporary -- and we should not expect eventlet to continue to be a valid path moving forward. 2) We have basic agreement that shared tooling, such as oslo libraries, should not dictate a specific threading model. In lieu of specific technical leadership and guidance from the TC, this leaves it to individual projects to decide what threading model to use instead of eventlet monkey patching. The unmerged spec[0] may be a good starting point if you're wondering where to start. Importantly: do not let this work wait! It is a significant change and will take a long time to complete. Thanks, Jay Faulkner 0: https://review.opendev.org/c/openstack/governance/+/902585

6 7

Antelope Barbican/Keystone: failure to delete secrets (Keystone TrustNotFound)
by Michel Jouvin 05 Jun '24

05 Jun '24

Hi, I've been validating Yoga -> Antelope on our preprod cloud. It was working successfully but "suddenly" (i.e. without a kwown/identified change!) when deleting Magnum clusters, the deletion of Barbican secrets (after stack deletion) fail. In the Magnum log, I find the following error: ---- Failed to delete trust: keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-a4400250-e8d5-46ec-a28a-dcb97a682512) (traceback skipped for readability) 2024-05-29 10:43:09.750 2078 INFO magnum.common.cert_manager.barbican_cert_manager [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] Recursively deleting certificate container https://os-77023.lal.in2p3.fr:9311/v1/containers/edd8487a-6cf1-4f4c-9f53-17… from Barbican. 2024-05-29 10:43:09.750 2078 INFO barbicanclient.base [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] Calculated Containers uuid ref: containers/edd8487a-6cf1-4f4c-9f53-173dfae7a8b2 2024-05-29 10:43:09.760 2078 ERROR barbicanclient.client [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] 4xx Client error: Not Found: Secrets container not found. 2024-05-29 10:43:09.761 2078 ERROR magnum.common.cert_manager.barbican_cert_manager [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] Error recursively deleting certificate container https://os-77023.lal.in2p3.fr:9311/v1/containers/edd8487a-6cf1-4f4c-9f53-17…: barbicanclient.exceptions.HTTPClientError: Not Found: Secrets container not found. ---- In Keystone log, I find with the same timestamp the following error that seems related to the one above: ---- 2024-05-29 10:43:09.690 1713 WARNING keystone.api._shared.authentication [None req-a4400250-e8d5-46ec-a28a-dcb97a682512 - - - - - -] Could not find trust: cf4b1b6082594025a1ff9e48df383566.: keystone.exception.TrustNotFound: Could not find trust: cf4b1b6082594025a1ff9e48df383566. 2024-05-29 10:43:09.693 1713 WARNING keystone.server.flask.application [None req-a4400250-e8d5-46ec-a28a-dcb97a682512 - - - - - -] Authorization failed. The request you have made requires authentication. from 134.158.77.23: keystone.exception.Unauthorized: The request you have made requires authentication. 2024-05-29 10:43:09.746 1723 ERROR keystone.server.flask.application [None req-8c76f304-c2a3-418c-8372-02c706ed356e 44d8cb3192b3485790ababe29991491a - 5b67a8641dd041089e018ceea9884eac - 5b67a8641dd041089e018ceea9884eac -] Could not find user: 527ec32509d849e8a49b41c3dc4c4c32.: keystone.exception.UserNotFound: Could not find user: 527ec32509d849e8a49b41c3dc4c4c32. ---- It is not clear for me if the ERROR is releated to the 2 warnings as the request is not the same but it seems to happen every time this way. In Barbican logs, there is not much things related except a 401 status for the request corresponding ot the TrustNotFound. I'm looking for advices on how to troubleshoot this. In particular, when the trust should be created and whether I should find somewhere a matching error at the time of the trust creation. It seems the error is only affecting deletion but I may be wrong. for the record, when starting Magnum there is an error complaining about /etc/magnum/keystone_auth_default_policy.json being not found but ignored it so far. And in Keystone an error seems to occur at Magnum cluster creation time: --- 2024-05-29 10:17:17.530 1711 WARNING py.warnings [None req-f484cc35-3e54-4d7f-8688-e6442c90344f a22b1199c74f4a819a49f3ac037a5e44 1dd67113c8bb4ef0a31dfca53335ffbd - - defaul t default] /usr/lib/python3.9/site-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is re quired ---- Thanks in advance for any hint. Best regards, Michel

2 3

Antelope Magnum: creating a new cluster makes the/some others unhealthy
by Michel Jouvin 05 Jun '24

05 Jun '24

Hi, I have been playing with Magnum (Antelope), creating a lot of clusters with the same template and varying the number of masters and/or nodes, having ~10 clusters started in the same project. I observed recently that cluster configuration that used to work well were not working any more (CREATE_FAILED during the master deployment in general). I have not found any evidence of the cause digging in various logs but I observed that with the current list of active clusters, if add a new one (even with a configuration as minimal as 1 master and 1 node), not only its creation fails but also the other (or at least) running clusters become unhealthy. If I delete the failed cluster, the other clusters return to the healthy state. It seems very reproducible (I did it more than 10 times) but I still don't see any message in the logs that could help to identify the cause. I've the feeling (but may be wrong) that it is related to either an insufficient project quota or an insufficient limit on one of the OpenStack service. Any idea on a possible cause or any advice on where to look for some information? Thanks in advance. Michel

2 6

[all] SQLAlchemy minimum requirement
by Pierre Riteau 02 Jun '24

02 Jun '24

Hello everyone, Historically, many projects were using the following requirement for SQLAlchemy: SQLAlchemy!=1.1.5,!=1.1.6,!=1.1.7,!=1.1.8,>=1.0.10 These banned versions were present in global-requirements.txt until earlier this month [1]. The remaining projects with the above requirement must change it to fix their requirements-check job (ERROR: Requirement for package SQLAlchemy excludes a version not excluded in the global list). While fixing the requirement for blazar, I realised that OpenStack projects are using a wide range of minimum requirements. For example, networking-sfc had already bumped the minimum to SQLAlchemy 2.0 [2], although this is being reverted [3]. While all projects must be compatible with the upper constraint (currently 2.0.30), should we agree on a maximum version for the lower bound? The 2024.2 release targets Ubuntu Jammy, which includes python3-sqlalchemy 1.4.31, and Debian bookworm, which includes python3-sqlalchemy 1.4.46. Thanks, Pierre Riteau (priteau) [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://review.opendev.org/c/openstack/networking-sfc/+/920046 [3] https://review.opendev.org/c/openstack/networking-sfc/+/920868

2 1

The outbound unicast traffic of VMS with security group enabled is broadcast
by renwc＠chinatelecom.cn 31 May '24

31 May '24

hello everyone， Here is a description of a bug I encountered using openstack Security Group,If security groups are enabled for virtual nics on cloud hosts, The outgoing traffic of the VMS enabled in the security group should be sent in unicast mode, but it is eventually forwarded to all ports in the vlan on the ovs br-int bridge. Does the openstack open source community have a patch to fix this bug? Can you provide patch? The following is my debugging process in detail. The unicast packet sent by the remote virtual machine 172.83.160.63 to the local virtual machine 172.83.160.74 is passed to the br-ex->br-int-> VM, Because the security group is enabled on the VM NIC, The br-int cannot learn the entry whose port is patch-ex and mac address is vm2. Therefore those packets sent from vm1 to vm2 are broadcast in vlans because they fail to match mac address entries. 172.83.160.74 patch-int 172.83.160.63 vm1 --- ovs(br-int) ------ ovs(br-ex)--- tor(physical switch) ......vm2 patch-ex The following is the ovs kernel flow table. The mac address of vm1 is fa:16:3e:6c:f3:09,The mac address of vm2 is fa:16:3e:d2:14:b5, the action of the flow table is multiple ports. recirc_id(0x314123),in_port(8),ct_state(-new+est-rel-rpl),eth(src=fa:16:3e:d2:14:b5,dst=fa:16:3e:6c:f3:09),eth_type(0x0800),ipv4(proto=6,frag=no),tcp(dst=2048/0xf800), packets:60100, bytes:14257193, used:2.035s, flags:P., actions:push_vlan(vid=5,pcp=0),3,pop_vlan,push_vlan(vid=1732,pcp=0),1,pop_vlan,7,9,10,15,5,18 recirc_id(0x314123),in_port(8),ct_state(-new+est-rel+rpl-inv+trk),ct_zone(0x5),ct_mark(0),eth(src=fa:16:3e:d2:14:b5,dst=fa:16:3e:6c:f3:09),eth_type(0x0800),ipv4(frag=no), packets:1, bytes:54, used:2.037s, flags:., actions:push_vlan(vid=5,pcp=0),3,pop_vlan,push_vlan(vid=1732,pcp=0),1,pop_vlan,7,9,10,15,5,18 After careful debugging analysis, the openflow action in the flow table 82 added by neutron is the VM port instead of normal. As a result, the ovs br-int bridge cannot learn the mac entry of vm2. table=82, priority=77,ct_state=+new-est,tcp,reg5=0x9,tp_dst=0x4/0xfffc actions=ct(commit,zone=NXM_NX_REG6[0..15]),output:9 table=82, priority=77,ct_state=+est-rel-rpl,tcp,reg5=0x9,tp_dst=0x4/0xfffc actions=output:9 Looking forward to reply thanks. weichun ren

2 1

The outgoing unicast traffic from the VMS enabled in the security group is eventually forwarded to all ports.
by renwc＠chinatelecom.cn 31 May '24

31 May '24

Hello， Here is a description of a bug I encountered using openstack Security Group,If security groups are enabled for virtual nics on VMS, The outgoing traffic from the VMS enabled in the security group should be sent in unicast mode, but it is eventually forwarded to all ports in the vlan on the ovs br-int bridge. Does the openstack open source community have a patch to fix this bug? Can you provide patch fixed the issue? The following is my debugging process in detail. The unicast packet sent by the remote virtual machine 172.83.160.63 to the local virtual machine 172.83.160.74 is passed to the br-ex->br-int-> VM, Because the security group is enabled on the VM NIC, The br-int cannot learn the entry whose port is patch-ex and mac address is vm2. Therefore those packets sent from vm1 to vm2 are broadcast in vlans because they fail to match mac address entries. 172.83.160.74 patch-int 172.83.160.63 vm1 --- ovs(br-int) ------ ovs(br-ex)--- tor(physical switch) ......vm2 patch-ex The following is the ovs kernel flow table. The mac address of vm1 is fa:16:3e:6c:f3:09,The mac address of vm2 is fa:16:3e:d2:14:b5, the action of the flow table is multiple ports. recirc_id(0x314123),in_port(8),ct_state(-new+est-rel-rpl),eth(src=fa:16:3e:d2:14:b5,dst=fa:16:3e:6c:f3:09),eth_type(0x0800),ipv4(proto=6,frag=no),tcp(dst=2048/0xf800), packets:60100, bytes:14257193, used:2.035s, flags:P., actions:push_vlan(vid=5,pcp=0),3,pop_vlan,push_vlan(vid=1732,pcp=0),1,pop_vlan,7,9,10,15,5,18 recirc_id(0x314123),in_port(8),ct_state(-new+est-rel+rpl-inv+trk),ct_zone(0x5),ct_mark(0),eth(src=fa:16:3e:d2:14:b5,dst=fa:16:3e:6c:f3:09),eth_type(0x0800),ipv4(frag=no), packets:1, bytes:54, used:2.037s, flags:., actions:push_vlan(vid=5,pcp=0),3,pop_vlan,push_vlan(vid=1732,pcp=0),1,pop_vlan,7,9,10,15,5,18 After careful debugging analysis, the openflow action in the flow table 82 added by neutron is the VM port instead of normal. As a result, the ovs br-int bridge cannot learn the mac entry of vm2. table=82, priority=77,ct_state=+new-est,tcp,reg5=0x9,tp_dst=0x4/0xfffc actions=ct(commit,zone=NXM_NX_REG6[0..15]),output:9 table=82, priority=77,ct_state=+est-rel-rpl,tcp,reg5=0x9,tp_dst=0x4/0xfffc actions=output:9 Was the issue above fixed in the openstack ? Looking forward to reply thanks. Best Regards weichun ren

1 0

Re: [tc][all][senlin] Retiring the Senlin project
by Ghanshyam Mann 30 May '24

30 May '24

---- On Thu, 30 May 2024 11:43:00 -0700 Sang Tran Quoc wrote --- > > Dear Ghanshyam, > > Greeting! I’m Sang from Viet Nam and now I’m working at the FPT Smart Cloud (Silver member of OpenInfra) as the openstack team leader. We have teammate with about 10 members who maintains the senlin project on our production AutoScaling Feature (my presentation about senlin at last VietOpenInfra summithttps://www.youtube.com/watch?v=aYXDI4mwohQ&t=1s) It’s sad to hear about community retired this project from upstream so I would like to nominate our team to continue maintain this project and I will handle the PTL role for further roadmap, bugs fix and feature. > > Hopefully it’s not late, I’m look forward to hearing from you for further discuss about next chance for senlin project. ++ openstack-discuss ML Hi Sang, Thanks for reaching out and its good to hear about Senlin project usage and your interest in maintaining it. > Hopefully it’s not late, I’m look forward to hearing from you for further discuss about next chance for senlin project. Unfortunately, TC has already retired the Senlin project[1]. Since last cycle, we have been waiting for interested maintainers to step up. * No Leader in 2024.1 cycle ** https://etherpad.opendev.org/p/2024.1-leaderless * After TC discussion, Senlin has been marked as an 'inactive' project since '2024.1' cycle ** https://review.opendev.org/c/openstack/governance/+/902626 ** This was the time we were expecting someone to come forward and step up to maintain it. * No leader for Senlin in 2024.2 cycles also ** https://etherpad.opendev.org/p/2024.2-leaderless * Considering no leader for two cycles, no maintainer, and the process, TC initiated the retirement: ** https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… But this is not the end and Senlin can still re-become the official OpenStack project[2]. For that, you need to re-apply Senlin as a new project in OpenStack, please check the process: - https://governance.openstack.org/tc/reference/new-projects-requirements.html [1] - https://review.opendev.org/c/openstack/governance/+/919347 - https://review.opendev.org/q/topic:%22retire-senlin%22+ [2]https://governance.openstack.org/tc/reference/dropping-projects.html#re-b… -gmann > > > Thanks & Best Regards > > > Sang Tran Quoc (Mr.) > FPT Smart Cloud > FPT Tower, 10 Pham Van Bach Str., Cau Giay Dist., Ha Noi, Viet Nam > E: sangtq8(a)fpt.com > T: 0962949843 > W: https://fptcloud.com | https://fpt.ai > > > > > From:Ghanshyam Mann gmann(a)ghanshyammann.com> > Date: Saturday, 11 May 2024 at 09:06 > To: openstack-discuss openstack-discuss(a)lists.openstack.org> > Subject: Re: [tc][all][senlin] Retiring the Senlin project > ---- On Wed, 24 Apr 2024 13:39:33 -0700 Ghanshyam Mann wrote --- > > Hello everyone, > > > > As you might know, the Senlin project is not active and was marked 'inactive' > > since 2024.1 cycle[1]. In this cycle also, this project is leaderless[2] and no maintainer > > stepped up to maintain it. As discussed in the Leaderless project etherpad[2], the next > > step is to retire the project. > > > > I'm sending this email to propose the Senlin retirement and based on the response, > > I will proceed next. > > Seeing no response to this email or no volunteer to maintain this project, I have proposed > the retirement in governance: > > - https://review.opendev.org/c/openstack/governance/+/919347 > > -gmann > > > > > [1] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… > > [2] https://etherpad.opendev.org/p/2024.2-leaderless#L72 > > > > -gmann > > >

1 0