April 2024 - openstack-discuss - lists.openstack.org

ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version
by Swogat Pradhan 24 Dec '24

24 Dec '24

Hi, I am currently trying to deploy openstack wallaby using tripleo arch. I created the network jinja templates, ran the following commands also: #openstack overcloud network provision --stack overcloud --output networks-deployed-environment.yaml custom_network_data.yaml # openstack overcloud network vip provision --stack overcloud --output vip-deployed-environment.yaml custom_vip_data.yaml # openstack overcloud node provision --stack overcloud --overcloud-ssh-key /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml and used the environment files in the openstack overcloud deploy command: (undercloud) [stack@hkg2director ~]$ cat deploy.sh #!/bin/bash THT=/usr/share/openstack-tripleo-heat-templates/ CNF=/home/stack/ openstack overcloud deploy --templates $THT \ -r $CNF/templates/roles_data.yaml \ -n $CNF/workplace/custom_network_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e $CNF/templates/node-info.yaml \ -e $CNF/templates/scheduler-hints.yaml \ -e $CNF/workplace/networks-deployed-environment.yaml \ -e $CNF/workplace/vip-deployed-environment.yaml \ -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ -e $CNF/workplace/custom-net-bond-with-vlans.yaml Now when i run the ./deploy.sh script i encounter an error stating: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command.: tripleoclient.exceptions.InvalidConfiguration: Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command. 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END return value: 1 Can someone tell me where the mistake is? With regards, Swogat Pradhan

4 4

[tc][all][monasca ] Retiring the Monasca project
by Ghanshyam Mann 11 Dec '24

11 Dec '24

Hello everyone, As you might know, the Monasca project is not active and was marked 'inactive' since 2024.1 cycle[1]. In this cycle also, this project is leaderless[2] and no maintainer stepped up to maintain it. As discussed in the Leaderless project etherpad[2], the next step is to retire the project. I'm sending this email to propose the Monasca retirement and based on the response, I will proceed next. [1] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [2] https://etherpad.opendev.org/p/2024.2-leaderless#L72 -gmann

6 15

[kolla] [openeuler] CI jobs status
by Michał Nasiadka 26 Nov '24

26 Nov '24

Hello Koalas, OpenEuler jobs in master (to be Antelope) are failing for a long time. If no one is interested in maintaining OpenEuler support in Kolla-Ansible - I will propose dropping it’s support on Wednesday Kolla meeting. Best regards, Michal

4 3

[horizon][ceph] object storage container storage policy missing
by Satish Patel 25 Nov '24

25 Nov '24

Folks, Openstack Release: Zed Recently I have integrated ceph RGW service with openstack/keystone and testing with command line tool "openstack container create foo1" and it works but when I go to Horizon UI I am seeing it doesn't have storage policy dropdown list and because of that I am not able to create container or upload file from Horizon GUI https://ibb.co/KhYW6Rd It works with commands like but not horizon. $ openstack container create foo1 +---------------------------------------+-----------+------------------------------------------------+ | account | container | x-trans-id | +---------------------------------------+-----------+------------------------------------------------+ | AUTH_d4d82849885e4126aef402c408ad2c5c | foo1 | tx00000c057454841b13497-0065b558b4-42ff2eb-ahm | +---------------------------------------+-----------+------------------------------------------------+ $ openstack container list +------+ | Name | +------+ | foo1 | +------+

2 2

[nova][libvirtd] internal error: unable to execute QEMU command 'query-named-block-nodes'
by Satish Patel 22 Oct '24

22 Oct '24

Folks, I have a ceph cluster and recently I configured rbd-mirror to replicate all data to remove ceph cluster for disaster recovery. Yesterday for POC we did a hard cutover on ceph and point openstack to the new cluster. All other vms came back up fine but 2 vms stuck in this error state in libvirt logs 2024-01-31 22:44:37.591+0000: 474597: error : qemuMonitorJSONCheckErrorFull:399 : internal error: unable to execute QEMU command 'query-named-block-nodes': cannot read image start for probe: Permission denied If it's a wider issue than it should impact all the vms but why only two VM stuck and not starting and libvirt giving me this error in logs nova-compute logs also showing same error 2024-01-31 22:44:40.925 7 INFO nova.compute.manager [None req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00 ca5c652478c7429e964257990800e9cb - - default default] [instance: 2de0f880-77c7-4d2c-9e01-898c57ad3693] Successfully reverted task state from powering-on on failure for instance. 2024-01-31 22:44:40.944 7 ERROR oslo_messaging.rpc.server [None req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00 ca5c652478c7429e964257990800e9cb - - default default] Exception during message handling: libvirt.libvirtError: internal error: unable to execute QEMU command 'query-named-block-nodes': cannot read image start for probe: Permission denied

3 8

[kuryr][tc] Final call for Kuryr PTL/DPL liaisons
by Ghanshyam Mann 16 Oct '24

16 Oct '24

Hello Everyone, Kuryr project ends up being a leaderless project for the 2024.2 Dalmatian cycle[1]. There is no volunteer to serve as PTL or DPL liaisons yet. As the next step, I have proposed to mark this project Inactive: - https://review.opendev.org/c/openstack/governance/+/916945 If you would like to serve as a leader and help maintain this project, this is the right time to step up by replying to this email or commenting in gerrit change 916945. [1] https://etherpad.opendev.org/p/2024.2-leaderless#L99 Ref: https://governance.openstack.org/tc/reference/emerging-technology-and-inact… -gmann

4 10

Kolla Ansible on Ubuntu 20.04 - cloud-init & other network issues
by Tobias McNulty 10 Oct '24

10 Oct '24

Hi, I'm attempting to use Kolla Ansible 14.6.0 to deploy OpenStack Yoga on a small 3-node Ubuntu 20.04 cluster. The nodes have 128 GB RAM each, dual Xeon processors, and dual 10G Intel NICs. The NICs are connected to access ports on a 10G switch with separate VLANs for the local and external networks. All the playbooks run cleanly, but cloud-init is failing in the Ubuntu 20.04 and 22.04 VMs I attempt to boot. The VM images are unmodified from https://cloud-images.ubuntu.com/, and cloud-init works fine if I mount a second volume with user-data. The error is a timeout attempting to reach 169.254.169.254. This occurs both when booting a VM in an internal routed network and directly in an external network. I tried various neutron plugin agents (ovn, linuxbridge, and openvswitch both with and without firewall_driver = openvswitch <https://docs.openstack.org/kolla-ansible/latest/reference/networking/neutro…>) first with a clean install of the entire OS each time, all with the same result. Running tcpdump looking for 169.254.169.254 shows nothing. As a possible clue, the virtual NICs are unable to pass any traffic (e.g., to reach an external DHCP server) unless I completely disable port security on the interface (even if the associated security group is wide open). But disabling port security does not fix cloud-init (not to mention I don't really want to disable port security). Are there any additional requirements related to deploying OpenStack with Kolla on Ubuntu 20.04? This is a fairly vanilla configuration using the multinode inventory as a starting point. I tried to follow the Quick Start <https://docs.openstack.org/kolla-ansible/yoga/user/quickstart.html> as closely as possible; the only material difference I see is that I'm using the same 3 nodes for control + compute. I am using MAAS so it's easy to get a clean OS install on all three nodes ahead of each attempt. I plan to try again with the standard (non-HWE) kernel just in case, but otherwise I am running out of ideas. In case of any additional clues, here are my globals.yml and inventory file, along with the playbook I'm using to configure the network, images, VMs, etc., after bootstrapping the cluster: https://gist.github.com/tobiasmcnulty/7dbbdbc67abc08cbb013bf5983852ed6 Thank you in advance for any advice! Cheers, Tobias

5 9

Live migration fails
by Szabo, Istvan (Agoda) 07 Aug '24

07 Aug '24

Hi, I have couple of compute nodes where the live migration fails with existing vms. When I quickly spawn a vm and try live migration it works so I assume shouldn't be a big problem with the compute node. However I have many existing vms where it fails with a servername not found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 ERROR nova.conductor.tasks.migrate [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Unable to find record for source node servername on servername: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Failed to compute_task_migrate_server: Compute host servername could not be found.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Setting instance to ACTIVE state.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.672 227612 ERROR oslo_messaging.rpc.server [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Exception during message handling: ComputeHostNotFound: Compute host am-osfecn-4025 Tried with this command: nova live-migration --block-migrate id. Any idea? Thank you. ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

5 16

[Tempest] [Blazar] Connection Refused Error 111 on Blazar tempest plugin
by Tran, Kenny 28 May '24

28 May '24

First time asking a question on the mailing list, I am trying to set up tempest against a quick start yoga (kolla-ansible) deployment with Keystone, Zun, Heat, Blazar, and Nova (1 controller, 1 compute). Default tempest tests are behaving as expected but when I try to run the blazar plugin tests they fail with Errno 111 Connection Refused. I ran a tcpdump on the tempest server side and saw no issues. Tcpdump on controller showed a reset when trying to connect to keystone. Is this an issue with blazar not being able to authenticate against the openstack deployment? The default tempest tests succeed so I don't think my admin_password admin_username etc. are incorrect. Kenny Tran Iowa State University Student Computer Engineer 515-441-5237 | trantk(a)iastate.edu

2 1

[trove][tc] Next step on leaderless project Trove
by Ghanshyam Mann 23 May '24

23 May '24

Hi Wu, This is regarding the Trove project leader assignment. We saw your late candidacy for Trove PTL[1]. The TC has listed Trove as a late candidacy project[2] and can consider a PTL appointment. If you are still interested in serving as Trove PTL, please propose the PTL appointment request to TC. Example: https://review.opendev.org/c/openstack/governance/+/915108 [1] https://review.opendev.org/c/openstack/election/+/910488 [2] https://etherpad.opendev.org/p/2024.2-leaderless#L99 -gmann

1 1

[tc][all][sahara] Retiring the Sahara project
by Ghanshyam Mann 22 May '24

22 May '24

Hello everyone, As you might know, the Sahara project is not active and was marked 'inactive' since 2024.1 cycle[1]. In this cycle also, this project is leaderless[2] and no maintainer stepped up to maintain it. As discussed in the Leaderless project etherpad[2], the next step is to retire the project. I'm sending this email to propose the Sahara retirement and based on the response, I will proceed next. [1] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [2] https://etherpad.opendev.org/p/2024.2-leaderless#L72 -gmann

3 9

[ptl][tc] OpenStack packages PyPi additional external maintainers audit & cleanup
by Ghanshyam Mann 17 May '24

17 May '24

Hi PTLs, As you might know or have seen for your project package on PyPi, OpenStack deliverables on PyPi have additional maintainers, For example, https://pypi.org/project/murano/, https://pypi.org/project/glance/ We should keep only 'openstackci' as a maintainer in PyPi so that releases of OpenStack deliverables can be managed in a single place. Otherwise, we might face the two sets of maintainers' places and packages might get released in PyPi by additional maintainers without the OpenStack project team knowing about it. One such case is in Horizon repo 'xstatic-font-awesome' where a new maintainer is added by an existing additional maintainer and this package was released without the Horizon team knowing about the changes and release. - https://github.com/openstack/xstatic-font-awesome/pull/2 To avoid the 'xstatic-font-awesome' case for other packages, TC discussed it in their weekly meetings[1] and agreed to audit all the OpenStack packages and then clean up the additional maintainers in PyPi (keep only 'openstackci' as maintainers). To help in this task, TC requests project PTL to perform the audit for their project's repo and add comments in the below etherpad. - https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup Thanks to knikolla to automate the listing of the OpenStack packages with additional maintainers in PyPi which you can find the result in output.txt at the bottom of this link. I have added the project list of who needs to check their repo in etherpad. - https://gist.github.com/knikolla/7303a65a5ddaa2be553fc6e54619a7a1 Please complete the audit for your project before March 15 so that TC can discuss the next step in vPTG. [1] https://meetings.opendev.org/meetings/tc/2023/tc.2023-01-11-16.00.log.html#… -gmann

18 27

Logscraper host and OpenSearch service upgrade
by Daniel Pawlik 16 May '24

16 May '24

Hello, We would like to notify you that the OpenSearch service [1] would be updated to a newer version on 2024-05-15 at 01:00 PM UTC. This procedure might take a while, depending of the cluster size. During that time, the OpenSearch service would not be available. If anyone has any doubts, please reply to the email. Dan [1] - https://opensearch.logs.openstack.org/_dashboards/app/home

1 1

[kolla-ansible] support for FWaaS/LBaaS?
by garcetto 15 May '24

15 May '24

good morning, does kolla support fwaas or lbaas? thank you

3 4

[tc][all][ec2-api] Retiring the ec2-api project
by Ghanshyam Mann 11 May '24

11 May '24

Hello everyone, As you might know, the ec2-api project is not active and was marked 'inactive' since 2024.1 cycle[1]. In this cycle also, this project is leaderless[2] and no maintainer stepped up to maintain it. As discussed in the Leaderless project etherpad[2], the next step is to retire the project. I'm sending this email to propose the ec2-api retirement and based on the response, I will proceed next. [1] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [2] https://etherpad.opendev.org/p/2024.2-leaderless#L72 -gmann

1 1

[tc][all][senlin] Retiring the Senlin project
by Ghanshyam Mann 11 May '24

11 May '24

Hello everyone, As you might know, the Senlin project is not active and was marked 'inactive' since 2024.1 cycle[1]. In this cycle also, this project is leaderless[2] and no maintainer stepped up to maintain it. As discussed in the Leaderless project etherpad[2], the next step is to retire the project. I'm sending this email to propose the Senlin retirement and based on the response, I will proceed next. [1] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [2] https://etherpad.opendev.org/p/2024.2-leaderless#L72 -gmann

1 1

[tc][all][solum] Retiring the Solum project
by Ghanshyam Mann 11 May '24

11 May '24

Hello everyone, As you might know, the Solum project is not active and was marked 'inactive' since 2024.1 cycle[1]. In this cycle also, this project is leaderless[2] and no maintainer stepped up to maintain it. As discussed in the Leaderless project etherpad[2], the next step is to retire the project. I'm sending this email to propose the Solum retirement and based on the response, I will proceed next. [1] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [2] https://etherpad.opendev.org/p/2024.2-leaderless#L72 -gmann

1 1

[tc][all][murano] Retiring the Murano project
by Ghanshyam Mann 11 May '24

11 May '24

Hello everyone, As you might know, the Murano project is not active and was marked 'inactive' since 2024.1 cycle[1]. Due to a critical bug in Murano, the TC needs to pass a resolution to stop its 2024.1 release[2]. In this cycle also, this project is leaderless[3] and no maintainer stepped up to maintain it. As discussed in the Leaderless project etherpad[3], the next step is to retire the project. I'm sending this email to propose the Murano retirement and based on the response, I will proceed next. [1] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [2] https://governance.openstack.org/tc/resolutions/20240227-remove-murano-from… [3] https://etherpad.opendev.org/p/2024.2-leaderless#L72 -gmann

1 1

neutron's OVS based VXLAN seems to be unable to access the network
by 2292613444＠qq.com 07 May '24

07 May '24

I found that the cloud host I created cannot connect to the network. The network type is VXLAN, and it cannot even access cloud init I used this installation document (partially in Chinese) and it was normal around 2022, but it cannot be used after updating the software source recently. The use of VXLAN in cloud hosts has the following faults (1) Unable to access network resources other than ICMP (ping), and even curl http://169.254.169.254 (This also leads to the inability to use cloud init) (2) Sometimes unable to ping 169.254.169.254 for unknown reasons. (3)Any service other than ICMP service cannot be accessed, All firewalls and security groups are allowed or closed. My network situation is as follows Internal (vxlan) 10.0.0.0/24 Gateway 10.0.0.1 Router gateway 10.0.0.1 → 192.168.10.121 External (flat) 192.168.10.11.0/24 Gateway 192.168.10.1.1 If any other information is needed, I can provide it and hope to fix this error. system version:ubuntu 22.04 last version (online ) openstack Y version I used a command in the openstack routing host to check The specific details are as follows：：（10.0.0.129 is cloud host,vxlan is port qr-4528fcca-21） ###start root@controller:~# ip netns qdhcp-2aa854ba-7fa6-43e4-989e-e97dafd777ee (id: 1) qdhcp-c7d9811d-1fcc-4aee-8f01-6f556fb56044 (id: 2) qrouter-48610214-6030-4e7e-8fe8-c60241e7c89c (id: 0) root@controller:~# ip netns exec qrouter-48610214-6030-4e7e-8fe8-c60241e7c89c bash root@controller:~# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 13: qr-4528fcca-21: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether fa:16:3e:41:c8:47 brd ff:ff:ff:ff:ff:ff inet 10.0.0.1/24 brd 10.0.0.255 scope global qr-4528fcca-21 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fe41:c847/64 scope link valid_lft forever preferred_lft forever 14: qg-622550f9-80: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether fa:16:3e:31:eb:c2 brd ff:ff:ff:ff:ff:ff inet 192.168.101.121/24 brd 192.168.101.255 scope global qg-622550f9-80 valid_lft forever preferred_lft forever inet 192.168.101.177/32 brd 192.168.101.177 scope global qg-622550f9-80 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fe31:ebc2/64 scope link valid_lft forever preferred_lft forever root@controller:~# tcpdump -i qr-4528fcca-21 -vvvvv tcpdump: listening on qr-4528fcca-21, link-type EN10MB (Ethernet), snapshot length 262144 bytes 17:36:02.295718 IP6 (hlim 1, next-header Options (0) payload length: 36) :: > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:02.371746 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328) 0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from fa:16:3e:0f:28:ea (oui Unknown), length 300, xid 0xe07c1d6a, Flags [none] (0x0000) Client-Ethernet-Address fa:16:3e:0f:28:ea (oui Unknown) Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message (53), length 1: Discover Parameter-Request (55), length 13: Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121) Domain-Name (15), Domain-Name-Server (6), Hostname (12), YD (40) YS (41), NTP (42), MTU (26), Unknown (119) Default-Gateway (3) END (255), length 0 PAD (0), length 0, occurs 41 17:36:02.373214 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328) 0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from fa:16:3e:0f:28:ea (oui Unknown), length 300, xid 0xe07c1d6a, Flags [none] (0x0000) Client-Ethernet-Address fa:16:3e:0f:28:ea (oui Unknown) Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message (53), length 1: Request Server-ID (54), length 4: 10.0.0.2 Requested-IP (50), length 4: 10.0.0.129 Parameter-Request (55), length 13: Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121) Domain-Name (15), Domain-Name-Server (6), Hostname (12), YD (40) YS (41), NTP (42), MTU (26), Unknown (119) Default-Gateway (3) END (255), length 0 PAD (0), length 0, occurs 29 17:36:02.402630 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:36:02.402639 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:36:02.403415 IP (tos 0x0, ttl 64, id 47481, offset 0, flags [DF], proto UDP (17), length 72) 10.0.0.129.46568 > 223.5.5.5.domain: [udp sum ok] 57238+ A? does-not-exist.example.com. (44) 17:36:02.403453 IP (tos 0x0, ttl 64, id 47482, offset 0, flags [DF], proto UDP (17), length 72) 10.0.0.129.46568 > 223.5.5.5.domain: [udp sum ok] 33458+ AAAA? does-not-exist.example.com. (44) 17:36:02.404176 IP (tos 0x0, ttl 253, id 44451, offset 0, flags [DF], proto UDP (17), length 104) 223.5.5.5.domain > 10.0.0.129.46568: [udp sum ok] 33458 q: AAAA? does-not-exist.example.com. 0/1/0 ns: does-not-exist.example.com. [5m] SOA <BAD PTR> [|domain] 17:36:02.568387 IP (tos 0x0, ttl 62, id 34373, offset 0, flags [DF], proto UDP (17), length 148) 223.5.5.5.domain > 10.0.0.129.46568: [udp sum ok] 57238 NXDomain- q: A? does-not-exist.example.com. 0/1/0 ns: example.com. [1s] SOA ns.icann.org. noc.dns.icann.org. 2024041801 7200 3600 1209600 3600 (120) 17:36:02.721968 IP6 (hlim 1, next-header Options (0) payload length: 36) :: > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:02.866526 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) :: > ff02::1:ff0f:28ea: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has fe80::f816:3eff:fe0f:28ea unknown option (14), length 8 (1): 0x0000: b4d5 8b30 6d50 17:36:03.868276 IP6 (hlim 1, next-header Options (0) payload length: 36) fe80::f816:3eff:fe0f:28ea > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:03.868993 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe0f:28ea > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:0f:28:ea 0x0000: fa16 3e0f 28ea 17:36:04.038227 IP6 (hlim 1, next-header Options (0) payload length: 36) fe80::f816:3eff:fe0f:28ea > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:07.590618 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:36:07.591603 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:36:07.876243 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe0f:28ea > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:0f:28:ea 0x0000: fa16 3e0f 28ea 17:36:11.884088 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe0f:28ea > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:0f:28:ea 0x0000: fa16 3e0f 28ea 17:36:26.791124 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe37:6938 > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:37:69:38 0x0000: fa16 3e37 6938 17:36:34.348772 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:36:34.348788 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:36:43.071294 IP6 (hlim 1, next-header Options (0) payload length: 36) :: > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:43.139163 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328) 0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from fa:16:3e:0f:28:ea (oui Unknown), length 300, xid 0x29d5e37d, Flags [none] (0x0000) Client-Ethernet-Address fa:16:3e:0f:28:ea (oui Unknown) Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message (53), length 1: Request Requested-IP (50), length 4: 10.0.0.129 Hostname (12), length 15: "host-10-0-0-129" Parameter-Request (55), length 13: Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121) Domain-Name (15), Domain-Name-Server (6), Hostname (12), YD (40) YS (41), NTP (42), MTU (26), Unknown (119) Default-Gateway (3) END (255), length 0 PAD (0), length 0, occurs 18 17:36:43.451136 IP6 (hlim 1, next-header Options (0) payload length: 36) :: > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:43.788779 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) :: > ff02::1:ff0f:28ea: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has fe80::f816:3eff:fe0f:28ea unknown option (14), length 8 (1): 0x0000: eb9e aa08 ffc6 17:36:44.790128 IP6 (hlim 1, next-header Options (0) payload length: 36) fe80::f816:3eff:fe0f:28ea > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:44.791007 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe0f:28ea > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:0f:28:ea 0x0000: fa16 3e0f 28ea 17:36:45.433011 IP6 (hlim 1, next-header Options (0) payload length: 36) fe80::f816:3eff:fe0f:28ea > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:45.876227 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:36:45.876239 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:36:48.796920 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe0f:28ea > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:0f:28:ea 0x0000: fa16 3e0f 28ea 17:36:52.804263 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe0f:28ea > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:0f:28:ea 0x0000: fa16 3e0f 28ea 17:37:20.924762 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:37:20.924779 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:37:51.676660 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:37:51.676672 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:38:25.468636 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:38:25.468653 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:38:31.772908 IP (tos 0x0, ttl 64, id 19606, offset 0, flags [DF], proto ICMP (1), length 84) 10.0.0.129 > 8.8.8.8: ICMP echo request, id 1398, seq 1, length 64 17:38:31.783398 IP (tos 0x0, ttl 53, id 0, offset 0, flags [none], proto ICMP (1), length 84) 8.8.8.8 > 10.0.0.129: ICMP echo reply, id 1398, seq 1, length 64 17:38:32.774655 IP (tos 0x0, ttl 64, id 19790, offset 0, flags [DF], proto ICMP (1), length 84) 10.0.0.129 > 8.8.8.8: ICMP echo request, id 1398, seq 2, length 64 17:38:32.785097 IP (tos 0x0, ttl 53, id 0, offset 0, flags [none], proto ICMP (1), length 84) 8.8.8.8 > 10.0.0.129: ICMP echo reply, id 1398, seq 2, length 64 17:38:36.838640 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:38:36.839533 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:38:59.212868 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:38:59.212885 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:39:18.252924 IP (tos 0x0, ttl 64, id 29804, offset 0, flags [DF], proto TCP (6), length 60) 10.0.0.129.60668 > 169.254.169.254.http: Flags [S], cksum 0xd701 (correct), seq 3028122336, win 28200, options [mss 1410,sackOK,TS val 4294871008 ecr 0,nop,wscale 7], length 0 17:39:18.252998 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 169.254.169.254.http > 10.0.0.129.60668: Flags [S.], cksum 0x5eac (incorrect -> 0xfad3), seq 3154613265, ack 3028122337, win 64308, options [mss 1410,sackOK,TS val 3597215885 ecr 4294871008,nop,wscale 7], length 0 17:39:19.270668 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 169.254.169.254.http > 10.0.0.129.60668: Flags [S.], cksum 0x5eac (incorrect -> 0xf6d9), seq 3154613265, ack 3028122337, win 64308, options [mss 1410,sackOK,TS val 3597216903 ecr 4294871008,nop,wscale 7], length 0 17:39:21.286651 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 169.254.169.254.http > 10.0.0.129.60668: Flags [S.], cksum 0x5eac (incorrect -> 0xeef9), seq 3154613265, ack 3028122337, win 64308, options [mss 1410,sackOK,TS val 3597218919 ecr 4294871008,nop,wscale 7], length 0 17:39:23.430642 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:39:23.432189 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:39:25.478662 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 169.254.169.254.http > 10.0.0.129.60668: Flags [S.], cksum 0x5eac (incorrect -> 0xde99), seq 3154613265, ack 3028122337, win 64308, options [mss 1410,sackOK,TS val 3597223111 ecr 4294871008,nop,wscale 7], length 0 17:39:33.670667 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 169.254.169.254.http > 10.0.0.129.60668: Flags [S.], cksum 0x5eac (incorrect -> 0xbe99), seq 3154613265, ack 3028122337, win 64308, options [mss 1410,sackOK,TS val 3597231303 ecr 4294871008,nop,wscale 7], length 0 17:39:44.044984 IP (tos 0x0, ttl 64, id 29814, offset 0, flags [DF], proto TCP (6), length 131) 10.0.0.129.60668 > 169.254.169.254.http: Flags [P.], cksum 0x2428 (correct), seq 1:80, ack 1, win 221, options [nop,nop,TS val 4294896800 ecr 3597215885], length 79: HTTP, length: 79 GET / HTTP/1.1 User-Agent: curl/7.29.0 Host: 169.254.169.254 Accept: */* 17:39:44.086640 IP (tos 0x0, ttl 64, id 60194, offset 0, flags [DF], proto TCP (6), length 52) 169.254.169.254.http > 10.0.0.129.60668: Flags [.], cksum 0x5ea4 (incorrect -> 0x58b3), seq 1, ack 80, win 502, options [nop,nop,TS val 3597241719 ecr 4294896800], length 0 17:39:44.140919 IP (tos 0x0, ttl 64, id 60195, offset 0, flags [DF], proto TCP (6), length 267) 169.254.169.254.http > 10.0.0.129.60668: Flags [P.], cksum 0x5f7b (incorrect -> 0x1982), seq 1:216, ack 80, win 502, options [nop,nop,TS val 3597241773 ecr 4294896800], length 215: HTTP, length: 215 HTTP/1.1 200 OK content-type: text/plain; charset=UTF-8 content-length: 98 date: Mon, 29 Apr 2024 09:39:44 GMT 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 2009-04-04 latest [|http] 17:39:49.053068 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:39:49.053086 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:39:49.286648 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:39:49.287162 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:40:14.170722 IP (tos 0x0, ttl 64, id 60196, offset 0, flags [DF], proto TCP (6), length 52) 169.254.169.254.http > 10.0.0.129.60668: Flags [F.], cksum 0x5ea4 (incorrect -> 0xe256), seq 216, ack 80, win 502, options [nop,nop,TS val 3597271803 ecr 4294896800], length 0 17:40:14.172246 IP (tos 0x0, ttl 64, id 29815, offset 0, flags [DF], proto TCP (6), length 64) 10.0.0.129.60668 > 169.254.169.254.http: Flags [.], cksum 0x0c22 (correct), seq 80, ack 1, win 221, options [nop,nop,TS val 4294926927 ecr 3597241719,nop,nop,sack 1 {216:217}], length 0 17:40:14.172294 IP (tos 0x0, ttl 64, id 60197, offset 0, flags [DF], proto TCP (6), length 267) 169.254.169.254.http > 10.0.0.129.60668: Flags [P.], cksum 0x5f7b (incorrect -> 0x2e83), seq 1:216, ack 80, win 502, options [nop,nop,TS val 3597271804 ecr 4294926927], length 215: HTTP, length: 215 HTTP/1.1 200 OK content-type: text/plain; charset=UTF-8 content-length: 98 date: Mon, 29 Apr 2024 09:39:44 GMT 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 2009-04-04 latest [|http] 17:40:19.238647 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:40:19.240069 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:40:52.608165 IP (tos 0x0, ttl 64, id 24950, offset 0, flags [DF], proto UDP (17), length 67) 10.0.0.129.57838 > 223.5.5.5.domain: [udp sum ok] 18832+ A? 0.centos.pool.ntp.org. (39) 17:40:52.608222 IP (tos 0x0, ttl 64, id 24951, offset 0, flags [DF], proto UDP (17), length 67) 10.0.0.129.57838 > 223.5.5.5.domain: [udp sum ok] 12709+ AAAA? 0.centos.pool.ntp.org. (39) 17:40:52.608926 IP (tos 0x0, ttl 253, id 63426, offset 0, flags [DF], proto UDP (17), length 99) 223.5.5.5.domain > 10.0.0.129.57838: [udp sum ok] 12709 q: AAAA? 0.centos.pool.ntp.org. 0/1/0 ns: 0.centos.pool.ntp.org. [5m] SOA . ^@. [|domain] 17:40:52.621561 IP (tos 0x0, ttl 62, id 26683, offset 0, flags [DF], proto UDP (17), length 215) 223.5.5.5.domain > 10.0.0.129.57838: [udp sum ok] 18832- q: A? 0.centos.pool.ntp.org. 4/0/0 0.centos.pool.ntp.org. [5m] A 162.159.200.1, 0.centos.pool.ntp.org. [5m] A 193.182.111.14, 0.centos.pool.ntp.org. [5m] A 111.230.189.174, 0.centos.pool.ntp.org. [5m] A 193.182.111.12 (187) 17:40:57.612621 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129^C, length 28 17:40:57.612632 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:41:31.942692 IP (tos 0x0, ttl 64, id 60198, offset 0, flags [DF], proto TCP (6), length 267) 10.0.0.1.9697 > 10.0.0.129.60668: Flags [P.], cksum 0x157f (incorrect -> 0x2322), seq 3154613266:3154613481, ack 3028122416, win 502, options [nop,nop,TS val 3597349575 ecr 4294926927], length 215 17:41:31.944269 IP (tos 0x0, ttl 64, id 21103, offset 0, flags [DF], proto TCP (6), length 40) 10.0.0.129.60668 > 10.0.0.1.9697: Flags [R], cksum 0x58d3 (correct), seq 3028122416, win 0, length 0 17:41:32.668720 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:41:32.668750 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:41:37.062634 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:41:37.063306 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:42:43.622650 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe41:c847 > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:41:c8:47 0x0000: fa16 3e41 c847 17:42:54.604794 IP (tos 0x0, ttl 64, id 29817, offset 0, flags [DF], proto TCP (6), length 64) 10.0.0.129.60668 > 169.254.169.254.http: Flags [.], cksum 0x9970 (correct), seq 79, ack 1, win 221, options [nop,nop,TS val 120064 ecr 3597241719,nop,nop,sack 1 {216:217}], length 0 17:42:54.604842 IP (tos 0x0, ttl 64, id 60199, offset 0, flags [DF], proto TCP (6), length 52) 169.254.169.254.http > 10.0.0.129.60668: Flags [.], cksum 0x5ea4 (incorrect -> 0xf9f2), seq 217, ack 80, win 502, options [nop,nop,TS val 3597432237 ecr 4294926927], length 0 17:42:59.612900 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:42:59.612917 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:42:59.750644 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:42:59.751196 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:43:17.363162 IP (tos 0x0, ttl 64, id 49296, offset 0, flags [DF], proto UDP (17), length 76) 10.0.0.129.40786 > 162.159.200.1.ntp: [udp sum ok] NTPv4, Client, length 48 Leap indicator: (0), Stratum 0 (unspecified), poll 7 (128s), precision 32 Root Delay: 0.000000, Root dispersion: 0.000000, Reference-ID: (unspec) Reference Timestamp: 0.000000000 Originator Timestamp: 0.000000000 Receive Timestamp: 0.000000000 Transmit Timestamp: 3841207700.761775004 (2021-09-21T10:08:20Z) Originator - Receive Timestamp: 0.000000000 Originator - Transmit Timestamp: 3841207700.761775004 (2021-09-21T10:08:20Z) 17:43:32.380593 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:43:32.380610 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:44:07.420619 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:44:07.420635 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:44:54.925246 IP (tos 0x0, ttl 64, id 29819, offset 0, flags [DF], proto TCP (6), length 64) 10.0.0.129.60668 > 169.254.169.254.http: Flags [.], cksum 0xc36e (correct), seq 79, ack 1, win 221, options [nop,nop,TS val 240384 ecr 3597241719,nop,nop,sack 1 {216:217}], length 0 17:44:54.925303 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40) 169.254.169.254.http > 10.0.0.129.60668: Flags [R], cksum 0x13fc (correct), seq 3154613266, win 0, length 0 17:44:59.932983 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:44:59.933001 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:45:00.070625 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:45:00.070993 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:47:38.400269 IP (tos 0x0, ttl 64, id 29403, offset 0, flags [DF], proto UDP (17), length 76) 10.0.0.129.57173 > 162.159.200.1.ntp: [udp sum ok] NTPv4, Client, length 48 Leap indicator: (0), Stratum 0 (unspecified), poll 8 (256s), precision 32 Root Delay: 0.000000, Root dispersion: 0.000000, Reference-ID: (unspec) Reference Timestamp: 0.000000000 Originator Timestamp: 0.000000000 Receive Timestamp: 0.000000000 Transmit Timestamp: 3566054594.035896213 (2013-01-01T18:43:14Z) Originator - Receive Timestamp: 0.000000000 Originator - Transmit Timestamp: 3566054594.035896213 (2013-01-01T18:43:14Z) 17:47:43.405117 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:47:43.405133 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:48:21.491778 IP (tos 0x0, ttl 64, id 21240, offset 0, flags [DF], proto UDP (17), length 67) 10.0.0.129.60700 > 223.5.5.5.domain: [udp sum ok] 37019+ A? 1.centos.pool.ntp.org. (39) 17:48:21.491845 IP (tos 0x0, ttl 64, id 21241, offset 0, flags [DF], proto UDP (17), length 67) 10.0.0.129.60700 > 223.5.5.5.domain: [udp sum ok] 41914+ AAAA? 1.centos.pool.ntp.org. (39) 17:48:21.492638 IP (tos 0x0, ttl 253, id 58295, offset 0, flags [DF], proto UDP (17), length 99) 223.5.5.5.domain > 10.0.0.129.60700: [udp sum ok] 41914 q: AAAA? 1.centos.pool.ntp.org. 0/1/0 ns: 1.centos.pool.ntp.org. [5m] SOA . . [|domain] 17:48:21.504470 IP (tos 0x0, ttl 62, id 62111, offset 0, flags [DF], proto UDP (17), length 215) 223.5.5.5.domain > 10.0.0.129.60700: [udp sum ok] 37019- q: A? 1.centos.pool.ntp.org. 4/0/0 1.centos.pool.ntp.org. [5m] A 111.230.189.174, 1.centos.pool.ntp.org. [5m] A 193.182.111.14, 1.centos.pool.ntp.org. [5m] A 117.80.231.60, 1.centos.pool.ntp.org. [5m] A 110.42.98.138 (187) 17:48:26.508934 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:48:26.508951 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:48:26.662640 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:48:26.663269 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 107 packets captured 107 packets received by filter 0 packets dropped by kernel ###end Firstly, this is the situation of tcpdump from the beginning of creating the cloud host to the end of my following command execution The command is Ping 8.8.8.8 #Success Curl http://169.254.169.254 #Report the following errors Curl: (56) Recv failure: connection reset by peer That's all for these situations. If needed, I can provide the following content The situation and installation documents related to this environment Other documents that may help resolve this situation

2 6

[all][oslo][neutron][glance][ironic] Deprecating eventlet based WSGI server
by Takashi Kajinami 07 May '24

07 May '24

Hello, As you know there has been a long discussion about eventlet usage in OpenStack, and after seeing multiple problems caused by the library and no long term plan in the project, and we are moving toward doping the dependency on eventlet. https://review.opendev.org/c/openstack/governance/+/916546 One of the core implementation which heavily depends on eventlet is WSGI server. Multiple projects have been providing the -api console scripts (eg nova-api) based on the oslo.service's WSGI server implementation which depends on eventlet. On the other hand, nowadays we have multiple methods to launch api services, such as httpd+mod_wsgi, uwsgi and gunicorn. And a few projects in OpenStack don't provide the -api console script (eg. aodh, barbican and placement) but suggest using these external frameworks to run their api WSGI application. I know that multiple deployment tools (for example TripleO in the past) and some distros (Debian) already started using the external framework for consistency, better concurrency and so on. Given the situation, I wonder if we can consider deprecating the eventlet WSGI server in oslo.service in 2024.2/2025.1 and eventually remove all -api script in 2025.2. This was discussed during the previous nova PTG sessions and I think this makes complete sense as a global direction given the variety of alternative approaches currently available. I'm tagging neutron(*1) and glance(*2) because IIRC these had(or even have) some problems with running API in wsgi mode, and also ironic because I've seen some works coming from it, iiuc. Please let me know if you have any concern applying this approach to these (or the other) projects. Thank you, Takashi Kajinami (*1) For neutron, there are a few issues being fixed now https://bugs.launchpad.net/neutron/+bug/2062009 but ml2-ovn may need additional fixes. https://bugs.launchpad.net/neutron/+bug/1912359 (*2) As far as I know the issue with glance was already fixed some time ago[1]. https://bugs.launchpad.net/glance/+bug/1888713 -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

7 9

[all][packaging][release][qa] pep-517 and pip 23
by smooney＠redhat.com 06 May '24

06 May '24

hi folks, TL;DR ----- this is a long email so the tl;dr is we should add pyproject.toml ``` [build-system] requires = ["pbr>=5.7.0", "setuptools>=36.6.0", "wheel"] build-backend = "pbr.build" ``` to all openstack projects to supprot pip 23.1+ Long version ------------ As some of you may know the ptg happend last week an done of hte topic raised there was a know limitation in devstack where GLOBAL_VENV=True does not work on centos/fedora derived distros. while i generally dont use centos/fedora derived distros for my daily work i have been using devstack for over 10 years at this point so when im told it broken i ocationlly will go fix it. In this case the issue is not with devstack (it often isnt) and is actully in our collective reliance on a deprecated functionality and how we package openstack today. Back story, for better or worse when openstack was created the landscape of python packaging was significantly different then it is today. To help corodinate our dependcies the openstack comunity went all in on requirements.txt and developed upper-constraits to manage our requirements and developed PBR (python build Reasonableness). both of those approches have served us well and likely will continue to do so for some time. Since then pyproject.toml has started to gain popularity and pep-517 (A build-system independent format for source trees) has been adotpted but so far openstack as a comunity has not embraced those changes. (although pbr already supprot pep-517 :) ). Until now this has not been a problem however with recent release of pip things are now breaking. Problem Context: - openstack does not provide pyporject.toml files - openstack project configure setuptools to use pbr via setup.py - setuptools support generating scripts form entry points(console_scripts) which we use for project binaries( nova- compute, neutron-server, ...) - openstack api services use a pbr extension know as wsgi_scripts which adds a similar functionality for generating wsgi application mains file - devstack due to coming changes in distro to block "sudo pip install" now installs openstack in a single by default on debian distros. on centos/fedora distros it currently almost functional (but not). - devstack installs in editable mode because that is how most developers use devstack for developemnt What changed: - pip 20.2 deprecated support for falling back to calling setup.py when pyproject.toml is not present - https://pip.pypa.io/en/stable/news/#v20-2 - https://github.com/pypa/pip/issues/8368 - at this point when pyproject.yaml is not called pip calls setup.py install [-e] <path to repo or package name> - setup.py invokes pbr to generate wsgi_scripts and other files - in pip 23.1 the fallback was removed https://pip.pypa.io/en/stable/news/#v23-1 - pip now builds a temp local wheel instead of calling setup.py install when pyproject.toml is not found - in editable mode (-e) the default way the wheel is built does not invoke pbr properly to generate the wsgi files - as a result the wsgi files are not installed to the venv bin dir and this causes devstack to fail to start keystone (or any other wsgi api) The fix, there are a few ways to approch this: - clamp pip (https://review.opendev.org/c/openstack/devstack/+/898845) this is a terible idea and it breaks debian - clamping pip below 23.1 works on centos for now however OVS on debian already requires pep 517 for installing - as such when that version of ovs (or other projects start enforcing that) hits centos it will break - add a minimal pyproject.toml to all openstack project - example for keystone https://review.opendev.org/c/openstack/keystone/+/899519 - this example jsut adds the minium pyproject.toml to enable pbr to be used as the build system - it does not remove supprot for setup.py but it will not be used by default going forward whit new version of pip - all config is still in setup.cfg - add pyproject.toml support but remove setup.py support https://review.opendev.org/c/opendev/bindep/+/816741 - this goes one step futrue and actuly nulls out setup.py - we could take this approach if we want to but its not required. The reason for this email is to get agreement on how to support pip 23.1+ My proposal would be to add pyproject.toml with the minimal config required to enable pip 23.2 but not drop the setup.py support in this cycle. i would suggest that we can deprecate installing via setup.py in caracal if that is desirable but not make that change until the next slurp to give time to projects and packages to catch up. if there are no objections to that i would like to ask what is the preferd way to create and submit the patches for the minimal pyproject.toml ``` [build-system] requires = ["pbr>=5.7.0", "setuptools>=36.6.0", "wheel"] build-backend = "pbr.build" ``` i can propose patches to the main project deployed by devstack by default. i.e. nova/neutron ectra but it might be better if someone form the core teams of each project could take on that task and chasing the reviews. if someone has a script to generate a patch for a set of repos that just add the above pyporject.toml then we can use that to keep a consistent topic/commit message across all patches. my proposal for testing this is two fold. in ci we can test the patches are functional by using a DNM patch to devstack to force enable the gloal venv on centos and depend on the set of patches, i will modify https://review.opendev.org/c/openstack/devstack/+/898845 to do that locally we can test that the patch is functional by doing the following substituting keystone for the relevant project. ``` git clone https://opendev.org/openstack/keystone cd keystone python3 -m venv .venv . .venv/bin/activate python3 -m pip install -U pip python3 -m pip install -e . ls .venv/bin/keystone* ``` the ls will not contain the wsgi script with the addition of the pyproject.toml it should. on debian sid i can reproduce the same issue the problem we are seeing on centos 9 now will start impacting other distos eventually so its good to fix this now. if i start proposing this to glance/nova/neutron/cinder/swift/horizon/placement (to get a default devstack working) can i get cores to review? can cores form each project nominate peopel to review or volenteer to do this work. i have created https://etherpad.opendev.org/p/pep-517-and-pip-23 to track this work i have some other work that i think we can do in parallel like addign pyporject.toml support to the pti requirements if we go this way but that i think we can discuss on a governacne patch to updated it. anyway this is a lot to digest so feedback is welcome. if someone else would like to drive this that is also welcome (i.e. tc/release team) regards sean.

10 27

[nova][cinder] Openstack with FC SAN storage with HA
by Satish Patel 06 May '24

06 May '24

Folks, I have a question, Does openstack support FC SAN storage? I have deployed openstack with ceph but first time got this requirement that a customer has SAN storage because they were using VMware but now move to openstack. How does openstack talk to FC SAN and what about VM migration etc? Do I need to mount the LUN disk on each computer and point nova to the LUN disk? Or I have to create some kind of shared file system on top of the LUN and expose that to nova/cinder. Can someone explain the entire workflow?

5 8

Magnum error after Antelope upgrade
by Michel Jouvin 02 May '24

02 May '24

Hi, In a cloud where all services have been upgraded from Yoga to Antelope, I tried to upgrade Magnum but after the (successful) upgrade, Magnum reports "keystone client errors" when trying to retrieve the status of existing clusters or to create new ones (deletion of existing ones seem to work). The typical error is: ---- 2024-04-29 11:57:39.733 72464 WARNING magnum.service.periodic [-] Skip pulling data from cluster 2e8d6ee8-01cb-42d4-9b6d-0cf242cd4afa due to error: unexpected keystone clie nt error occurred: Forbidden (HTTP 403): magnum.common.exception.AuthorizationFailure: unexpected keystone client error occurred: Forbidden (HTTP 403) 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic Traceback (most recent call last): 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/magnum/common/exception.py", line 57, in wrapped 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic return func(*args, **kw) 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/magnum/common/clients.py", line 166, in barbican 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic self._barbican = barbicanclient.Client(session=session, 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/barbicanclient/client.py", line 186, in Client 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic return client_class(session=session, *args, **kwargs) 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/barbicanclient/v1/client.py", line 47, in __init__ 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic normalized_microversion = self._get_max_supported_version( 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/barbicanclient/v1/client.py", line 88, in _get_max_supported_version 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic min_ver, max_ver = self._get_min_max_server_supported_microversion( 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/barbicanclient/v1/client.py", line 119, in _get_min_max_server_supporte d_microversion 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic return self._get_min_max_version(session, endpoint, '1.1') 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/barbicanclient/v1/client.py", line 125, in _get_min_max_version 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic resp = discover.get_version_data( 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/keystoneauth1/discover.py", line 107, in get_version_data 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic resp = session.get(url, headers=headers, authenticated=authenticated) 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/keystoneauth1/session.py", line 1141, in get 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic return self.request(url, 'GET', **kwargs) 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic File "/usr/lib/python3.9/site-packages/keystoneauth1/session.py", line 986, in request 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic raise exceptions.from_response(resp, method, url) 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic keystoneauth1.exceptions.http.Forbidden: Forbidden (HTTP 403) 2024-04-29 11:57:39.733 72464 ERROR magnum.service.periodic --- Did I miss something in the upgrade procedure? Thanks in advance for any help. Best regards, Michel

4 9

Upgrade Bobcat/Caracal and python3-futurist
by Francesco Di Nucci 02 May '24

02 May '24

Hi all, I'm upgrading from Bobcat to Caracal using RDO repositories. I noticed that the package python3-futurist.noarch was present in centos-openstack-bobcat repo, but it is not in openstack-caracal. So it is not possible to update/reinstall/etc. it as it is "orphaned" once the bobcat repository is disabled. Should it be removed? Is it still a dependency? Best regards Francesco Di Nucci

4 6

[dev][requirements] Potentially ceasing development of Paste
by Jeremy Stanley 01 May '24

01 May '24

Just a heads up that Chris is raising concerns about continued maintenance of the Paste library, which more than a few of our projects rely on: https://github.com/cdent/paste/discussions/91 It's worth considering whether we would offer to adopt/fork it or if there are straightforward migration paths to actively maintained alternatives. -- Jeremy Stanley

3 3

[cinder] this week's meeting canceled
by Jon Bernard 01 May '24

01 May '24

Hello all, we will cancel tomorrow's regularly scheduled meeting in honor of International Workers' Day. See you all next week. -- Jon

1 0

Upgrade Bobcat/Caracal and python3-futurist
by Francesco Di Nucci 30 Apr '24

30 Apr '24

Hi all, I'm upgrading from Bobcat to Caracal using RDO repositories. I noticed that the package python3-futurist.noarch was present in centos-openstack-bobcat repo, but it is not in openstack-caracal. So it is not possible to update/reinstall/etc. it as it is "orphaned" once the bobcat repository is disabled. Should it be removed? Is it still a dependency? Best regards Francesco Di Nucci

1 0

[Octavia] next weekly meetings cancelled
by Gregory Thiemonge 30 Apr '24

30 Apr '24

Hi Folks, I'll be on Holiday for the 2 next weekly meetings (May 1st and 8th), so I'm cancelling them. The next meeting will be May 15th. For any topics (requesting reviews, questions, etc..) we're still reachable on IRC. Thanks Greg

1 0

Nova Antelope / RHEL9 : failure to lauch VM because video model virtio is not supported
by Michel Jouvin 30 Apr '24

30 Apr '24

Hi, After upgrading a compute server from Yoga/EL8 to Antelope/EL9, I cannot start VM on it. nova-compute log says: ---- Failed to build and run instance: libvirt.libvirtError: unsupported configuration: domain configuration does not support video model 'virtio' ---- According to https://bugs.launchpad.net/nova/+bug/2018172, it seems the problem also existed when trying to deploy nova-compute Yoga/Zed on EL9 but what fixed the problem is not clear in the issue... Any idea about what could cause this problem and how to fix it? Best regards, Michel

2 2

[Horizon][Skyline][API] Recommendation for a Web Application Firewall in front of Services
by Kaster, Jörn 30 Apr '24

30 Apr '24

Hello together, are there any recommendations or best practices how to expose the openstack API and/or WebUI's over a WAF (web application firewall) to the internet? We think we would use VMware NSX ALB [1] for this, but with the current broadcom trouble in the VMware sphere, we are uncertain for the future with this product. Are there any alternative software tools to reach the waf functionality to secure the api and http path's to openstack? regards Jörn [1]https://docs.vmware.com/en/VMware-NSX-Advanced-Load-Balancer/index.html [https://docs-prod.vmware.com/en/VMware-NSX-Advanced-Load-Balancer/images/GU…]<https://docs.vmware.com/en/VMware-NSX-Advanced-Load-Balancer/index.html> VMware NSX Advanced Load Balancer Documentation<https://docs.vmware.com/en/VMware-NSX-Advanced-Load-Balancer/index.html> Welcome to the VMware NSX Advanced Load Balancer (formerly known as Avi Networks). The NSX Advanced Load Balancer makes it easy to apply load balancing, web application firewall, and container ingress to any application in any datacenter and cloud. docs.vmware.com

1 0

[neutron] neutron-dynamic-routing advertising IPv4 AND IPv6 routes for true MP-BGP OR running multiple BGP speakers per agent (host)?
by Christian Rohmann 30 Apr '24

30 Apr '24

Hello OpenStack-Discuss, when setting up Neutron Dynamic Routing [1] to advertise IPV4 floating IPs and also IPv6 tenant networks I ran into some questions / issues. I was just about to ask about them here on the ML, when I found the older thread [2] discussing the advertising of IPv6 routes (e.g. from tenant networks). There Sean mentioned [4] that even with MP-BGP support [3] it's required to have two BGP speakers. While [3] means IPv4 or IPv6 can be used to establish the BGP session itself to then advertise routes for either address family, it's not possible to advertise IPv4 AND IPv6 routes via the same session. To cut to the chase of this threat .... It's not uncommon to simply use individual (non Multi-Protocol) BGP sessions to exchange routes for IPv4 and IPv6 individually and to me this poses no issue to me. The problem with Neutron is though, that the os_ken dragent driver only supports one BGP speaker [5] per agent. This leads to only one of the BGP speakers (IPv4 or IPv6) running per individual (control plane, networking) host. Or am I not seeing something or am misreading the capabilities of the components involved? If not, I see two options for improvement here: 1) Enabling a BGP speaker to have more than one address family (dragent driver already has MP-BGP support via [3]), so the query fetching the routes [6] simply returns them both. 2) Somehow get more than one BGP speaker per dragent working by ... a) ... extending the driver to host more than one BGP speaker b) ... extending dragent to have more than one driver instance Would love to hear how others have set up their dynamic routing and what your ideas are! Regards Christian [1] https://github.com/openstack/neutron-dynamic-routing [2] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [3] https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/608302 [4] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [5] https://github.com/openstack/neutron-dynamic-routing/blob/ffaaf5a25d846fead… [6] https://github.com/openstack/neutron-dynamic-routing/blob/ffaaf5a25d846fead…

3 4

[tc][all] OpenStack Technical Committee weekly summary and meeting agenda
by Goutham Pacha Ravi 30 Apr '24

30 Apr '24

Hello Stackers, We're slowly approaching Milestone-1 of the 2024.2 (Dalmatian) release cycle [1]. As you observe, there is a buzz of activity across all OpenStack repositories as we've just gotten out of our design and planning event. I hope you've been enjoying the PTG summaries that have been posted thus far; I would especially recommend reading through the summary JayF put up for the Technical Committee meetings [2]. Allow me to take you through the latest news from the Technical Committee: === Weekly Meeting === Last week's TC meeting was held on IRC on Apr 23rd, 2024. The meeting was logged to eavesdrop [3]. Our main topic of discussion was about the fate of unmaintained branches, and the discussion was timely given that Elod Iles from the OpenStack Release Management SIG brought a discussion to this list about transitioning the "stable/zed" branch of all release-cycle deliverables to "unmaintained/zed" [5]. This process is underway [6]. Concerns were expressed that the unmaintained process may be easy for teams to opt in, and the burden on the "unmaintained-core" team is growing with each branch. Project Teams are expected to actively assist in transitioning branches to end-of-life. Since this isn't happening, and since the TC has noted that a number of projects have CI (Zuul) configuration errors on these "unmaintained" branches, it was proposed that we be aggressive about proposing end-of-life for unmaintained branches. It was also argued that our release model is going too fast for real-world deployments to catch up, and transitioning a lot of branches simultaneously to EOL may impact many consumers. In the interest of time during the meeting, we decided not to interfere with the transition of stable/zed to "unmaintained/zed". We encourage project leaders to: - Get their CI configuration in order for any unmaintained branches - If there is no interest/time to maintain any unmaintained branch, please actively pursue the branch's EOL process [7] In the rest of the meeting, we went through open governance proposals. We have been distracted by the PTG and Action Items across OpenStack, so some of these proposals are lacking review attention. We'll be focusing on getting these adequate review attention in the coming week/s. The next TC meeting is today, Apr 30th, 2024, at 1800 UTC on OFTC's #openstack-tc channel [4]. The agenda feels light, but we'll try to address a number of TC trackers at this meeting. This meeting is enhanced by the community's participation. We'd especially invite project leaders (PTLs, DPLs, Core maintainers) to participate and propose topics. === Goal Proposals === I'd like to request a community review of the following OpenStack community goals and updates: - Migrate eventlet usages to asyncio (https://review.opendev.org/c/openstack/governance/+/902585) - Goal: Remove eventlet requirements from shared libraries (https://review.opendev.org/c/openstack/governance/+/916546) - Add timeline to remove enforce_scope in RBAC goal (https://review.opendev.org/c/openstack/governance/+/915179) === Governance Proposals === The following governance proposals are close to being merged: - Add option to move leaderless project to Inactive status (https://review.opendev.org/c/openstack/governance/+/914726) - Appoint Wenxiang Wu as PTL of Skyline | (https://review.opendev.org/c/openstack/governance/+/915108) - Transition Freezer project to DPL | (https://review.opendev.org/c/openstack/governance/+/914911) The following governance proposals are actively being discussed: - Add DPL model & liaison reset policy (https://review.opendev.org/c/openstack/governance/+/916833) - Update to include docs and miscellaneous repos for AC status (https://review.opendev.org/c/openstack/governance/+/915021) - Clarify the timeline for a project to move to Inactive state (https://review.opendev.org/c/openstack/governance/+/908880) The following governance proposals need some review attention: - Move openstack/kuryr and openstack/kuryr-libnetwork to Zun (https://review.opendev.org/c/openstack/governance/+/917517) - 2023 User Survey TC Question Analysis (https://review.opendev.org/c/openstack/governance/+/900516) And finally, please consider volunteering as an Election official for the next OpenStack elections! [8] Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] https://releases.openstack.org/dalmatian/schedule.html [2] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [3] https://meetings.opendev.org/meetings/tc/2024/tc.2024-04-23-18.00.log.html [4] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [5] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [6] https://review.opendev.org/q/topic:zed-unmaintained [7] https://governance.openstack.org/tc/resolutions/20230724-unmaintained-branc… [8] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack…

1 0

[all][oslo] Removing Python 3.8 support
by Takashi Kajinami 30 Apr '24

30 Apr '24

Hello, I'm sending this to remind you of the fact that Python 3.8 support will no longer be part of the tested python versions for 2024.2 . Based on the plan, I'll drop Python 3.8 support from all oslo libraries and update the minimum python version from 3.8 to 3.9, in the final release of oslo deliverables. Please make sure that none of the master jobs are dependent on Python 3.8 . We faced some problems when we first attempted to remove python 3.8 support early and the removal was eventually reverted because of remaining usage of Focal and remaining lifecycle of Python 3.8. However now Python 3.8 is actually reaching its EOL, we may no longer attempt the same revert. https://governance.openstack.org/tc/reference/runtimes/2024.2.html Thank you, Takashi Kajinami -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

1 0

[tc][election] A call for election officials for 2024.2
by Goutham Pacha Ravi 30 Apr '24

30 Apr '24

Hello Stackers, I'd like to express my gratitude to Ian Y Choi and Tony Breeds for their service as OpenStack's election officials for the 2024.1 term. A hallmark of our community has been its leadership and the guiding principles [1] we use when defining them. In Aug-Sep 2024, we're going to be electing the next set of OpenStack Project Team Leads as well as 4 members of the OpenStack Technical Committee. We've run a combined election for PTLs and TC seats in the past few years. We envision doing the same for this upcoming election. Most of the election process is automated, and election officials help execute and oversee this process [2]. If you're interested in being an election official, please reach out to me. If you've previously served as an election official, please do not hesitate to reach out. If we have enough volunteers, you may do us a great favor by mentoring those doing this for the first time. Thank you, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] https://governance.openstack.org/tc/reference/principles.html [2] https://governance.openstack.org/election/process.html

1 0

Re: [magnum][magnum-capi][cinder] Override csi plugin to override availability zone for cinder
by Satish Patel 30 Apr '24

30 Apr '24

Thanks for the quick response. I would like to have this in the MCAPI driver so we have more control on variables by labels. Because each deployment is different. I have created issue here: https://github.com/vexxhost/magnum-cluster-api/issues/366 On Mon, Apr 29, 2024 at 5:26 PM Mohammed Naser <mnaser(a)vexxhost.com> wrote: > Hi Satish, > > > Replies inline, as I’m trying to “decompose” this. 😊 > > > > *From: *Satish Patel <satish.txt(a)gmail.com> > *Date: *Monday, April 29, 2024 at 5:13 PM > *To: *OpenStack Discuss <openstack-discuss(a)lists.openstack.org> > *Subject: *[magnum][magnum-capi][cinder] Override csi plugin to override > availability zone for cinder > > Folks, > > > > I have deployed a magnum with magnum-capi driver. Now trying to work with > a cinder for PV stuff and found some strange issue related to the > Availability Zone. > > > > My k8s cluster running in AZ: foo > > My Ceph/Cinder running in AZ: nova (default) > > > > I have created k8s cluster using magnum-capi selecting AZ: foo but when I > am trying to create/attach PV then getting an error saying invalid AZ. Ofc > because its in different AZ so I have add following snippet in Storage > class to override AZ of CSI > > > > allowedTopologies: - matchLabelExpressions: - key: > topology.cinder.csi.openstack.org/zone values: - nova > > > > So in this case, you do allow cross az attach inside OpenStack in order to > allow this, is this correct? Since the AZ of your storage != the AZ of > your compute nodes. > > > > PV creation completed successfully after applying the above change. Now > when I try to deploy ngnix and pass that PV then again I get a volume > affinity issue because worker nodes and volume are in different AZ. > > > > I did the following to change CSI plugin az to nova for worker node to > make it fix. > > > > kubectl label nodes kube-acrsf-default-worker-rxmg6-t7wzd > topology.cinder.csi.openstack.org/zone=nova --overwrite > > > > What is the proper workaround here for the above scenario and what would > be the proper way to fix it? > > > > In this case, what you’re doing is telling the node that it is running in > the `nova` AZ (which it is not!) > > > > Does magnum have Cinder CSI AZ label to override availability zone in > magnum template? > > > > > https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/cin… > > > > I believe in this case, you’ll need to set the ignore-volume-az if I > understand correctly, it seems Magnum does not have a default label for > this. If you want to file an issue to the CAPI driver we can add an option > to enable this and make it ignore it. >

1 0

[magnum][magnum-capi][cinder] Override csi plugin to override availability zone for cinder
by Satish Patel 30 Apr '24

30 Apr '24

Folks, I have deployed a magnum with magnum-capi driver. Now trying to work with a cinder for PV stuff and found some strange issue related to the Availability Zone. My k8s cluster running in AZ: foo My Ceph/Cinder running in AZ: nova (default) I have created k8s cluster using magnum-capi selecting AZ: foo but when I am trying to create/attach PV then getting an error saying invalid AZ. Ofc because its in different AZ so I have add following snippet in Storage class to override AZ of CSI allowedTopologies: - matchLabelExpressions: - key: topology.cinder.csi.openstack.org/zone values: - nova PV creation completed successfully after applying the above change. Now when I try to deploy ngnix and pass that PV then again I get a volume affinity issue because worker nodes and volume are in different AZ. I did the following to change CSI plugin az to nova for worker node to make it fix. kubectl label nodes kube-acrsf-default-worker-rxmg6-t7wzd topology.cinder.csi.openstack.org/zone=nova --overwrite What is the proper workaround here for the above scenario and what would be the proper way to fix it? Does magnum have Cinder CSI AZ label to override availability zone in magnum template?

1 0

[tacker] Cancelling IRC meeting
by Yasufumi Ogawa 29 Apr '24

29 Apr '24

Hi team, Due to holidays in Japan, I'd like to skip IRC meeting this week. Thanks, Yasufumi

1 0

[ironic][nova-scheduler] "host_state" in nova filters has stale aggregate information
by Michael Sherman 29 Apr '24

29 Apr '24

Hi all, We’re running into an issue, where for two sites with 150-250 ironic nodes on a single conductor and nova-compute instance, we’ve started to get “no hosts available” errors from nova scheduler. We’re using the blazar-nova filter to match on hosts in specifically tagged aggregates. After adding some debug logs, I found that the “host_state” object passed to the filter seems to have out-of-date aggregate information. Specifically, if I query the system with “openstack aggregate show …” or “openstack allocation candidate list”, I see the correct aggregate for the nodes in question, but the contents of “host_state” reflect a previous state. This “staleness” does not seem to correct itself over time, but is resolved by restarting the nova-scheduler process (actually restarting the kolla docker container, but the same effect). However the issues return over the course of a couple hours. We haven’t increased the number of nodes, or otherwise changed the hardware, so I’m not sure what could have triggered this issue. Any advice on further debugging steps would be greatly appreciated. Thank you! -- Michael Sherman Infrastructure Lead – Chameleon Computer Science, University of Chicago MCS, Argonne National Lab

3 13

[publiccloud-sig] Meeting on May 1st will be skipped
by Felix Kronlage-Dammers 29 Apr '24

29 Apr '24

Hi everyone, This Wednesday (May 1st) the next meeting of the publiccloud sig would be scheduled. However in a lot of the european countries this is a public holiday. As such Tobias and I decided that we will skip this call. A lot of us will be at the OpenInfra Day Berlin on the 15th of May - which would be the date for the next publiccloud sig, as such the one meeting on the 15th will happen during coffee break at the OID berlin :) (More infos on the OID Berlin here: https://oideurope2024.openinfra.dev/germany/#:~:text=15%20May%202024%20%7C%….) regards felix -- Felix Kronlage-Dammers Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[oslo] Rootwrap configuration issues (sudoers entries?)
by Francesco Di Nucci 29 Apr '24

29 Apr '24

Hello, I am configuring oslo rootwrap on my OpenStack instance, but I could only find documentation about nova-rootwrap like here <https://docs.openstack.org/oslo.rootwrap/latest/user/usage.html>. On the compute nodes it is now working, while I'm having troubles with eg. network nodes. I have added a sudoers entry on the like of the nova one: neutron ALL = (root) NOPASSWD: /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf * But I don't know if it is correct, is there any documentation about it? Regards Francesco Di Nucci

2 3

[kolla] Weekly meeting cancelled on 1st of May
by Michał Nasiadka 29 Apr '24

29 Apr '24

Hello Koalas, Due to 1st of May being public holiday in some European countries (including Poland) - Kolla weekly meeting is cancelled. See you next week - 8th May. Michal

1 0

[neutron] Bug deputy - week of Apr 22
by Sławek Kapłoński 29 Apr '24

29 Apr '24

Hi, I was bug deputy last week and below is summary of what new bugs opened for Neutron during that time: ## High - https://bugs.launchpad.net/neutron/+bug/2063043 - octavia/ovn: filedescriptor out of range in select() - assigned to Fernando Royo ## Medium - https://bugs.launchpad.net/neutron/+bug/2063081 - [ovn] connectivity to load balancer with FIP from a server connected to a router with external gateway - assigned to Fernando Royo - https://bugs.launchpad.net/neutron/+bug/2063459 - # DHCP agent might use default gateway of unrouted subnets - assigned to Sebastian Lohff, patch already proposed ## Low - https://bugs.launchpad.net/neutron/+bug/2063669 - DNS extensions reported as available even if not enabled - unassigned, seems like kind of low hanging fruit for someone who wants to start ## Others - https://bugs.launchpad.net/neutron/+bug/2063463 - [ovn-octavia-provider] hairpin_snat_ip not set - unassigned -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

Losing connection when connecting br-ex to bond
by sarasoltanm2000＠gmail.com 28 Apr '24

28 Apr '24

The problem started from where I realized I couldn't ping or ssh the instances in openstack then I realized that it was a networking problem and that br-ex was not connected to my interfaces. for some reason I had to create a bond with a static private ip address of the second interface. (172.16.8.2/24). so to solve this problem I tried to connect the bond to the br-ex but as soon as I connect the bond to br-ex I get disconnected from the server (server cannot ping its gateway) and br-ex state goes down, when I connect it to the bond by the following command: sudo ovs-vsctl add-port br-ex bond0 and when I try to bring the bridge br-ex back up, the bond is automatically deleted from the ports. this is the configuration of bridges before adding the bond port: #ovs-vsctl show Manager "ptcp:6640:127.0.0.1" is_connected: true Bridge br-ex Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port phy-br-ex Interface phy-br-ex type: patch options: {peer=int-br-ex} Port br-ex Interface br-ex type: internal Bridge br-int Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port br-int Interface br-int type: internal Port "tapf556c-c7" tag: 1 Interface "tapf556c-c7" type: internal Port "tapd2980-19" tag: 4095 Interface "tapd2980-19" type: internal Port "qr-8ea26-da" tag: 4095 Interface "qr-8ea26-da" type: internal Port "tap89cc3-c0" tag: 2 Interface "tap89cc3-c0" type: internal Port int-br-ex Interface int-br-ex type: patch options: {peer=phy-br-ex} ovs_version: "2.12.0" any idea why this happens and what is the solution? btw the instance is not connected to a router and it is basically connected to 2 networks (172.16.8.32/28 and 172.16.8.48/28)

2 1

[Neutron] Openvswitch forcing MTU 1450
by Karl Kloppenborg 28 Apr '24

28 Apr '24

Hey Neutron team. I have an issue where within the VXLAN network of tenancies the expected 1500 MTU is honoured but despite all MTU in the underlay being 9k and all networks and paths set to 1500, Openvswitch caps the MTU transiting north/south to 1450. This is causing considerable pain to some of our customers and despite trying to force the openvswitche bridges request_mtu to 1500, and all interfaces look right, it’s still caping at 1450. Note, TCP dumps show the pings set to 1500 getting to the network namespace on the openvswitch agent, so it appears something between the nat/snat process. VM1 private -> VM2 private (mtu 1500) = works VM1 private -> VM2 public (mtu 1500) = fails VM -> External (mtu 1500) = fails External Host -> VM (mtu 1500) = fails Adusting to 1422 on ping sizes works. Anyone experienced this? I cannot seem to make heads or tails of it. Thanks, Karl.

1 1

[nova] allow only specific instances on host aggregate
by Eugen Block 26 Apr '24

26 Apr '24

Hi, I'm playing around with host aggregates and the aggregate_instance_extra_specs for flavors. I successfully created a host aggregate, it allows me to schedule instances with a specific flavor on that aggregate only. But is there a way to prevent other VMs from starting on that aggregate (which were launched with different flavors)? The only thing coming to mind is to put that compute node(s) into a different availbility zone. Since the default is "nova", this would prevent anything else to start on that hypervisor. Is there anything I missed with aggregates? Any pointers are appreciated! Thanks! Eugen

3 7

how to set up configuration parameter for a datastore
by 龚永生 26 Apr '24

26 Apr '24

hi, chunyang I installed torve 2023.2, but failed to list the configuration parameter list: root@compute2:~# openstack datastore list +--------------------------------------+----------+ | ID | Name | +--------------------------------------+----------+ | ee74cbff-92d1-4f83-a06e-cd40fb45bac6 | dt_mysql | +--------------------------------------+----------+ root@compute2:~# openstack datastore version list dt_mysql +--------------------------------------+------------+---------+ | ID | Name | Version | +--------------------------------------+------------+---------+ | 82e71712-4e4d-4ff6-947f-8de4c1535d15 | ver_5.7.29 | 5.7.29 | | 8ba2ab5f-6dd1-4f25-b571-4031361a91cf | ver_8.3 | 8.3 | +--------------------------------------+------------+---------+ root@compute2:~# openstack database configuration parameter list 82e71712-4e4d-4ff6-947f-8de4c1535d15 and failed to create configruantion group as well: root@compute2:~# openstack database configuration create group1 '{"sync_binlog" : 1}' --datastore dt_mysql Configuration groups are not supported for this datastore: dt_mysql ver_8.3 (HTTP 422) could you tell me how to enable the configuration parameter list? regards. 龚永生浙江九州未来信息科技有限公司 99CLOUD Co. Ltd. 邮箱（Email）：gong.yongsheng(a)99cloud.net 地址：海淀区上地六街研华大厦5层南侧手机（Mobile)：+86-18618199879 公司网址（WebSite）：http://99cloud.net

1 0

Instance stuck in BUILD status
by 西岭 26 Apr '24

26 Apr '24

Dear openstack trove module developers, I was having trouble creating a trove instance and kept showing build status. My create command is: trove create mysql_zh-11 1 --size 5 --nic net-id=5f161d13-0cfb-4707-9bd6-f23850d86fbf --databases dbtest --users user:password --datastore mysql --datastore_version mysql-5.6 After entering the trove list command, the final state is: | bdcb298c-86d3-42c5-b7af-1cc0c7f93904 | mysql_zh-11 | mysql | mysql-5.6 | BUILD | 1 | 5 | RegionOne | The image I used was downloaded from the openstack website: trove-train-mysql-ubuntu-xenial Enter the openstack server list command and it will display: 2bf580eb-3c93-4689-9b53-e76307beb241 | mysql_zh-11 | ACTIVE | external-net=192.168.0.56 | trove-train-mysql-ubuntu-xenial | mysql-mini | I don't know the username and password of the image, so I can't see the specific information inside the cloud host. In addition, when I used the tool to change the login username and password of the above image, and created the trove instance with the image, after waiting 3600 seconds, it showed: trove.taskmanager.models [-] Service status: ERROR, service error description: guestagent error: TroveError: Failed to get server 0908ee4e-9f1d-444a-9c69-c8b8cc9d8938 for instance 03c2a10c-87f1-4169-9cf7-13735ff8756c, error: Failed to validate token (HTTP 404) (Request-ID: req-9604fb4c-4aae-4351-8996-5ee4dfbd9ad3) This time, I was able to log in to the cloud host generated during the creation of trove. After entering, I found that the guest-agent service failed to start. The reasons were: sudo chown -R ubuntu:root /etc/trove/conf.d cannot be executed. I wonder if it is because the password of the image has been changed? Then, I used the same image to create the trove instance again, and during the instance creation process, I went to the cloud host and started the guest-agent service manually. The result was the same. What else did I miss when creating the instance? How to solve this problem? See attachment for details of trove log. Thank you very much for your help. ----------------------------------------------------- Xi Ling, from Qingdao, China April 25, 2024

2 1

[openstack][nova] Resize instance cause instance error
by Nguyễn Hữu Khôi 26 Apr '24

26 Apr '24

Hello everyone. Sometimes, I resize instance and this instance cannot be resized then it will be error state. I use Kolla-Ansbile Yoga with Dell Powestore5000T as backend with iscsi protocol. I report in this: https://bugs.launchpad.net/nova/+bug/2063345 Pls give some ideas on it? Thank you much Nguyen Huu Khoi

1 1

red hat integration with ceph error
by Adivya Singh 25 Apr '24

25 Apr '24

hi, anybody see this error before, while integrating red hat open stack with ubuntu ceph overcloud-computeids-2 | error={"ansible_job_id": "34 6042025410.662459", "attempts": 2, "changed": false, "finished": 1, "msg": "Paunch failed with config_id tripleo_step4", "rc": 1, "stderr": "Error executing ['podman', 'container', 'exists' , 'nova_libvirt_init_secret']: returned 1\nDid not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_libvirt_init_secret', '--filter', 'label=config_id=tri pleo_step4', '--format', '{{.Names}}']\" - retrying without config_id\nDid not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_libvirt_init_secret', '--f ormat', '{{.Names}}']\"\nError executing ['podman', 'run', '--name', 'nova_libvirt_init_secret', '--label', 'config_id=tripleo_step4', '--label', 'container_name=nova_libvirt_init_secret', '--label', 'managed_by=tripleo-Computeids', '--label', 'config_data={\"command\": [\"/bin/bash\", \"-c\", \"/usr/bin/virsh secret-define --file /etc/nova/secret.xml && SECRET_UUID=$(fgrep \ \\\\"fsid = \\\\\" /etc/ceph/ceph.conf | cut -d \\\\\" \\\\\" -f 3) && test -n \\\\\"$SECRET_UUID\\\\\" && SECRET_KEY=$(fgrep \\\\\"key = \\\\\" /etc/ceph/ceph.client.openstack.keyring | cu t -d \\\\\" \\\\\" -f 3) && test -n \\\\\"$SECRET_KEY\\\\\" && /usr/bin/virsh secret-set-value --secret $SECRET_UUID --base64 $SECRET_KEY\\\\n\"], \"detach\": false, \"image\": \"192.168.0. 1:8787/rhosp-rhel8/openstack-nova-libvirt:16.2\", \"net\": \"host\", \"privileged\": false, \"security_opt\": \"label=disable\", \"user\": \"root\", \"volumes\": [\"/etc/hosts:/etc/hosts:ro \", \"/etc/localtime:/etc/localtime:ro\", \"/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro\", \"/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro\", \"/etc/pk i/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro\", \"/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro\", \"/etc/pki/tls/cert.pem:/etc/pki/tls/c ert.pem:ro\", \"/dev/log:/dev/log\", \"/etc/puppet:/etc/puppet:ro\", \"/var/lib/config-data/puppet-generated/nova_libvirt/etc/nova:/etc/nova:ro\", \"/etc/ceph:/etc/ceph:ro\", \"/etc/libvirt :/etc/libvirt\", \"/var/run/libvirt:/var/run/libvirt:shared,z\", \"/var/lib/libvirt:/var/lib/libvirt:shared\"], \"environment\": {\"TRIPLEO_CONFIG_HASH\": \"f780b2585d2cb2edc5954608158cf938 \"}}', '--conmon-pidfile=/var/run/nova_libvirt_init_secret.pid', '--log-driver', 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/nova_libvirt_init_secret.log', '--env=TRIPLEO_CON regards Adivya Sungh

2 1

[I18n][PTG] 2024.2 Dalmatian PTG - I18n SIG summary
by Ian Y. Choi 25 Apr '24

25 Apr '24

Hi all, As I18n SIG lead, I would like to share with you a summary of the recent discussions held during the 2024.2 Dalmatian PTG regarding the Internationalization (I18n) Special Interest Group (SIG). Firstly, I would like to express my sincere appreciation to the entire community for their commitment to make OpenStack more internationalized. The efforts put forth in translating and making OpenStack accessible globally are truly commendable, thanks to the contributions of numerous individuals, although we are under unprecedented situations and even Zanata will sunset soon [1]. During the PTG, I18n SIG covered several important topics with active and invaluable contributors including Seongsoo (a special thanks for his leadership both I18n team and within the Korea User group): 1. Progress on open reviews for I18n: We discussed pending items in the OpenStack/I18n repository and the status of various reviews. 2. Discussion on the OpenStack/I18n repository structure: This included plans to embrace the use of Weblate - wlc library and add tox environment and corresponding requirements.txt. 3. Pilot work progress on governance translation support: A demonstration was provided on setting up a new project with API and CLI. 4. Feasibility discussion on migrating from translate.openstack.org earlier than [1]. Deliberation on target languages and next steps for user survey translation. 5. User survey translation discussion: Target languages and next steps were deliberated upon. In sum, I18n will continue our effort to finalize the migration from Zanata to Weblate to emphasize the importance of I18n as a fundamental component for global OpenStack accessibility. Thank you for your continuation of the I18n SIG. If you have any questions or would like further information on any of the discussed topics, please don't hesitate to reach out. Best regards, /Ian [1] https://lists.osci.io/hyperkitty/list/zanata-sunset@lists.osci.io/thread/6F… [2] I18n PTG Etherpad: https://etherpad.opendev.org/p/apr2024-ptg-i18n [3] I18n PTG Team Photo 1: https://1drv.ms/i/s!AgSCQctkWshsgb8ZvM-l7Ol1UOxNjA?e=5tdLpb [3] I18n PTG Team Photo 2: https://cdn.discordapp.com/attachments/1192271703057911888/1228373792162840…

1 0

Compute Resources
by Gk Gk 25 Apr '24

25 Apr '24

Hi, Is there a way to find out the available ram, disk and vcpus on a compute node for vm creation using any placement APIs or any other way ?

3 3

[cinder][dev][ptl] NVMe/FC
by yian.zong＠dell.com 25 Apr '24

25 Apr '24

Hi cinder team, I'd like to know if cinder supports NVMe/FC. From nvmeof connector [1], it seems only RoCE(RDMA) and TCP. [1] https://opendev.org/openstack/os-brick/src/branch/master/os_brick/initiator… Thank you! Yian

5 5

[cinder][PTG] 2024.2 Dalmatian PTG summary
by Jon Bernard 25 Apr '24

25 Apr '24

Hello everyone, thanks for a very productive PTG for the Dalmatian cycle. Below is a brief summary of the highlights. The raw etherpad notes are here[1] and both the summary and transcribed etherpad notes are captured in our wiki here[2]. # Improve driver documentation When proposing new or updating existing drivers, our interface can be confusing. We agreed on an effort to migrate class documentation to the interface definitions, make clear the mandatory and optional methods for drivers, and maintain this documentation going forward. # User visible information in volume types Volume type metadata and extra specs are not visible to users, making it difficult to ascertain whether a volume type will lead to encryption or replication. Agreement was reached on metadata fields similar to Glance's metadefs, allowing drivers to report capabilities in a standard way and allowing that metadata to become visible to admins and users. # Optional backup driver dependencies Although volume driver dependencies are optional, those of our backup drivers are listed in requirements and are therefore always installed irrespective of deployment configuration. Agreement was reached to move these dependencies to driver-requirements. This should simplify efforts of both deployers and packagers. # Simplify deployments A few ideas were proposed to improve our deployment: * When cinder-volume is deployed in an active-active configuration, our `backend_host` parameter must be updated on every backend to have the same value, we would like to remove this additional step. * Use of predictable names to define tenants, instead of unique IDs. * Have dedicated quote section in our configuration for quote-related options. # Response schema validation A spec to add response schema validation in addition to our existing request validation was proposed. With adequate coverage, clients in various languages could be autogenerated (in theory). No significant objects were raised, agreement to review related patches this cycle. # Documentation of Ceph auth requirements We do not provide comprehensive and easy-to-find documentation on exactly what authentication/permission expectations there are between different services, leading deployers troubleshoot themselves and not be able to rely on upstream best practices. Agreement to improve this situation in the current cycle. # Cinder backup improvements An accumulation of bug fixes and performance improvements have stalled in the review queue. We went through the major ones as a team to attempt to bring awareness and unblock the remaining review requirements. See WIKI notes for specific details. # Migrating backups between multiple backends There is desire to support multiple backup backends where new backups go to a new backend while backups in an old backend remain readable. We want to avoid needing to create a full backup in the new backend to support incremental snapshots (and the associated charge). A spec will be proposed for review and work towards this goal will proceed during this cycle. # Cross-project with Glance In a cross-project collaboration with the Glance team, an improved method for image migration was proposed. Agreement to introduce a new migration operation in Glance. Cinder context was provided and a consensus on path forward was reached. # Cross-project with Nova In a cross-project collaboration with both Nova and Glance, the topic of image encryption was discussed. Dan Smith from the Nova team provided input from the Nova side. Cinder and Nova expect LUKS formatted images, but Glance currently supports GPG encrypted images - requiring re-encryption prior to use. It was noted that LUKS encrypted images can be created without root permission and the Glance team is now looking to drop GPG support and consolidate around LUKS as our unified format. # Active-Active support for NetApp A NetApp engineer raised questions about adding active-active support to the driver. Questions were answered and that work should proceed in this cycle. # Performance of parallel clone operations For the clone operation in cinder we're using a single distributed lock to prevent the source volume from being deleted mid-operation. This causes multiple concurrent clone operations to block. Under the right conditions, this can cause a significant performance degradation. Multiple possible solutions were discussed (read-write locks) and a consensus to use the DB was reached. Some details are unclear, awaiting a spec before moving forward. # Volume encryption with user defined keys Cinder does not currently support encryption with key provided by the user. Users could both manage their own keys and data could be recovered even if keys are lost at the deployment. There are several technical challenges to support this. Several of these hurdles were raised, more thought and research is needed before we have a spec that could be reviewed. [1] https://etherpad.opendev.org/p/dalmatian-ptg-cinder [2] https://wiki.openstack.org/wiki/CinderDalmatianPTGSummary -- Jon

2 1

[ironic][neutron] Another networking path for Ironic?
by Julia Kreger 25 Apr '24

25 Apr '24

Greetings folks, During the Ironic PTG last week, Ironic had our busiest PTG session on the topic of networking, and what to do in order to move things forward. I think the best way to frame how we reached the discussion is to keep in mind that projects in our community have been built upon boundaries of delineation. For example, Nova for is virtualization. Ironic’s area has been bare metal machine deployment and lifecycle management. Neutron’s area has been networking. Ironic has always fit a little more awkwardly into this model due to the nature of bare metal – as evidenced by the networking related (networking-generic-switch, networking-baremetal) projects living under Ironic’s governance driven by the need to be able to toggle physical switch ports to ensure a secure provisioning process. Now it's getting even more awkward as the lines between a server and a switch start to blur. Ironic is one of the few OpenStack projects that has significant use cases outside of a fully integrated OpenStack cluster. Additionally, even in the fully integrated case, bare metal is often not the primary use case considered -- quite understandable since virtualized networking is not a trivial problem. This combination of two factors: physical networking being a secondary use case for Neutron, and operators wanting to get physical network automation even without a fully integrated OpenStack, leads to difficulties both on the development side and the operational side. This creates a situation where Ironic’s answer to just basic network automation of “use Neutron and an ML2 plugin” is really not tenable for many infrastructure operators. The underlying challenge is that infrastructure operators need something for just basic management actions like attaching or detaching hosts from physical networks which also works in a model of respecting separation of duties. This is further complicated with DPUs and the changing landscape DPUs are creating by forcing a much more complex interaction when it comes to just basic networking needs for supporting the deployment of a bare metal host. To attach a network to a host, it is no longer just toggle a switchport configuration, it is potentially toggle a switchport *and* toggle configuration on a DPU device. And in this weeks’ Ironic PTG session, “Future of networking”[0], discussion of DPUs helped us reach a tipping point for the discussion. The idea of a separate network interface (for Ironic to use) which was not neutron, reached a point of being a chorus of contributors. So to put it in different words, Ironic is considering implementing functionality for Switch and DPU device network configuration management in terms of device attachment to networks which would likely leverage the existing ML2 plugin model such that Ironic can trigger port attachment and detachment directly. We’re not talking about doing anything beyond attachment and detachment. This means aspects like IPAM, Security Groups, Routing, and so on and so forth are entirely off the table for us. Our answer for such will continue to be “use neutron.” The discussion thus far has focused on not replacing existing plugins, but to enable their use. To use the plugins today, Neutron and the involvement of other OpenStack services are required – which is not viable for approximately half of Ironic’s usage base. For example, a Metal3 user is only using Ironic as an embedded tool, and if their environment requires VXLAN networks to be terminated on a DPU then they would need to create/invent tooling to facilitate that, or we can solve that with Ironic and account for the overall physical machine lifecycle at the same time. We also see a path where we can better integrate with Neutron and help operator’s meet more restrictive security requirements by delineating part of the functionality into Ironic’s scope when leveraging upstream solutions. We realize there are many unknowns, and welcome Neutron contributors to collaborate[1] with us, as this could ultimately impact project scope. Thoughts? Thanks, Julia [0]: https://etherpad.opendev.org/p/ironic-ptg-april-2024#L609 [1]: https://review.opendev.org/c/openstack/ironic-specs/+/916126/

2 2

Instance stuck in BUILD status
by 西岭 25 Apr '24

25 Apr '24

Dear openstack trove module developers, I was having trouble creating a trove instance and kept showing build status. My create command is: trove create mysql_zh-11 1 --size 5 --nic net-id=5f161d13-0cfb-4707-9bd6-f23850d86fbf --databases dbtest --users user:password --datastore mysql --datastore_version mysql-5.6 After entering the trove list command, the final state is: | bdcb298c-86d3-42c5-b7af-1cc0c7f93904 | mysql_zh-11 | mysql | mysql-5.6 | BUILD | 1 | 5 | RegionOne | The image I used was downloaded from the openstack website: trove-train-mysql-ubuntu-xenial Enter the openstack server list command and it will display: 2bf580eb-3c93-4689-9b53-e76307beb241 | mysql_zh-11 | ACTIVE | external-net=192.168.0.56 | trove-train-mysql-ubuntu-xenial | mysql-mini | I don't know the username and password of the image, so I can't see the specific information inside the cloud host. In addition, when I used the tool to change the login username and password of the above image, and created the trove instance with the image, after waiting 3600 seconds, it showed: trove.taskmanager.models [-] Service status: ERROR, service error description: guestagent error: TroveError: Failed to get server 0908ee4e-9f1d-444a-9c69-c8b8cc9d8938 for instance 03c2a10c-87f1-4169-9cf7-13735ff8756c, error: Failed to validate token (HTTP 404) (Request-ID: req-9604fb4c-4aae-4351-8996-5ee4dfbd9ad3) This time, I was able to log in to the cloud host generated during the creation of trove. After entering, I found that the guest-agent service failed to start. The reasons were: sudo chown -R ubuntu:root /etc/trove/conf.d cannot be executed. I wonder if it is because the password of the image has been changed? Then, I used the same image to create the trove instance again, and during the instance creation process, I went to the cloud host and started the guest-agent service manually. The result was the same. What else did I miss when creating the instance? How to solve this problem? See attachment for details of trove log. Thank you very much for your help. ----------------------------------------------------- Xi Ling, from Qingdao, China April 25, 2024

1 0

Secure RBAC manager role
by Sławek Kapłoński 25 Apr '24

25 Apr '24

Hi, I started looking at the S-RBAC today and for the phase 3 [1] especially. My question is - do we have agreement how this MANAGER will look like? In the linked document there is only info that keystone's spec [2] will have to be changed but I'm not sure if this is final now and if we can/should start thinking and implementing policies for the MANAGER role or not yet. [1] https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rb… [2] https://review.opendev.org/c/openstack/keystone-specs/+/818603 -- Slawek Kaplonski Principal Software Engineer Red Hat

3 5

[Glance] Deprecate metadata_encryption_key
by Pranali Deore 25 Apr '24

25 Apr '24

Hello Everyone, Glance core team have discussed and decided during Dalmatian PTG[1] about depreciation of `metadata_encryption_key` config option. We decided to deprecate it in this cycle and remove it in `E` (2025.1) cycle[2]. Problems with `metadata_encryption_key` config option and its related functionality: 1. It was added quite a long time ago and even though as per the description it should encrypt the location metadata but it actually encrypts the location url. 2. It encrypts the location url only for image upload/import/download/show APIs and doesn't encrypt url on location APIs. 3. If it's enabled during upgrade, it will break the existing deployment since the existing image url is not encrypted. 4. It doesn't even work for location url encryption as expected since it does not encrypt the legacy images url on start up, download of that image fails with InvalidLocation error. If there are any objections/suggestions on the above plan kindly please let us know before the spec freeze date of glance which is July 05,2024. If there will be no response till 5th July, we will assume that it's a go forward signal and can merge the spec lite[2] for the same. [1]: https://etherpad.opendev.org/p/apr2024-ptg-glance#L339 [2]: https://review.opendev.org/c/openstack/glance-specs/+/916178 Thanks & Regards, Pranali Deore

2 1

[magnum] Policy doesn't allow certificate:get
by Satish Patel 25 Apr '24

25 Apr '24

Folks, I have recently deployed openstack 2023.1 using kolla-ansible and after setting up magnum I noticed the following policy error when obtaining cards. After reading [1] the policy file looks like it needs a reader role to obtain a certificate. I have assigned the "reader" role to the user but still getting the same error message and no luck. $ openstack role add --user user1 --user-domain mydomain1 --project myproject1 reader # Reload User Creds RC file. $ openstack coe cluster config dev2 Policy doesn't allow certificate:get to be performed (HTTP 403) (Request-ID: req-7445ef3c-52a3-4911-97f6-1fb25d9fac1f) What else could be wrong here? 1. https://docs.openstack.org/magnum/latest/configuration/sample-policy.html

2 5

[tc][ptl][dpl] Policy to reset the DPL model every cycle
by Ghanshyam Mann 25 Apr '24

25 Apr '24

Hello Everyone. In vPTG, TC discussed improving to monitor the DPL model & its liaison's activeness. As per current policy[1], we rely on the liaisons to make sure they update the governance if they are not active in assigned roles or TC members to regularly check the DPL projects. Due to various reasons, this did not help and we saw many projects or liaisons going inactive and are detected very late. To improve the DPL model monitoring, I proposed resetting the DPL model to the PTL model every cycle and project teams need to opt-in DPL model every cycle. I would like to highlight this change to the projects currently on the DPL model or planning to adopt the same in future. Feel free to review and provide your feedback on the gerrit change mentioned below: - https://review.opendev.org/c/openstack/governance/+/916833 [1] https://governance.openstack.org/tc/resolutions/20200803-distributed-projec… -gmann

1 0

[largescale-sig] Next meeting: March 27, 9utc
by Thierry Carrez 24 Apr '24

24 Apr '24

Hi everyone, The Large Scale SIG will be meeting this Wednesday in #openstack-operators on OFTC IRC, at 9UTC. Arnaud Morin will be chairing. You can doublecheck how that UTC time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20240327T09 Feel free to add topics to the agenda: https://etherpad.opendev.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez

2 3

openstack python3-keystone package
by Sari Zidan 24 Apr '24

24 Apr '24

Dear OpenStack Support Team, I'm a DEVOPS engineer who works for Zadara LTD. I hope this email finds you well. I am reaching out to inquire about the python-keystone package provided by OpenStack. Currently, our company utilizes Ubuntu Bionic (18.04.6) on our machines, and we have the python-keystone package installed and in use. However, we are now looking to upgrade to Python 3 and, consequently, need to transition from python-keystone to python3-keystone. In our codebase, we've integrated interfaces and APIs that rely on functions from the python-keystone Python library. However, as we transition to python3-keystone, I've identified several deprecated functions and modules which are no longer supported within the new package. To address this, I'm reaching out for assistance in finding suitable replacements within the new package. The deprecated functions/modules that require replacement (no longer supported by the new package) are: 1. keystone.common.dependency.resolve_future_dependencies() 2. keystone.auth.controllers.Auth.validate_token() 3. keystone.contrib.ec2.controllers.Ec2ControllerV3 - specifically: create_credential(), delete_credential() and get_credential() 4. keystone.resource.controllers.ProjectV3 - specifically: get_project, update_project, delete_project, create_project. 5. keystone.assignment.controllers 6. keystone.token.persistence.PersistenceManager Any guidance or assistance you can provide on this matter would be greatly appreciated. Thank you for your attention to this inquiry, and I look forward to hearing from you soon.

1 0

RDO Caracal 2024.1 Released
by Amy Marrich 24 Apr '24

24 Apr '24

RDO Caracal 2024.1 Released The RDO community is pleased to announce the general availability of the RDO build for OpenStack 2024.1 Caracal for RPM-based distributions, CentOS Stream and Red Hat Enterprise Linux. RDO is suitable for building private, public, and hybrid clouds. Caracal is the 29th release from the OpenStack project, which is the work of more than 1,000 contributors from around the world. The release is already available for CentOS Stream 9 on the CentOS mirror network in: http://mirror.stream.centos.org/SIGs/9-stream/cloud/x86_64/openstack-caraca… The RDO community project curates, packages, builds, tests and maintains a complete OpenStack component set for RHEL and CentOS Stream and is a member of the CentOS Cloud SIG. The Cloud SIG focuses on delivering a great user experience for CentOS users looking to build and maintain their own on-premise, public or hybrid clouds. All work on RDO and on the downstream release, Red Hat OpenStack Platform, is 100% open source, with all code changes going upstream first. The highlights of the broader upstream OpenStack project may be read via https://releases.openstack.org/caracal/highlights.html but here are some highlights: - Drivers with inactive CI were marked unsupported including Windows iSCSI Driver, Windows SMB Driver, Dell SC Series Storage Driver (iSCSI, FC), Dell VNX Storage Driver (FC, iSCSI) and Dell XtremeIO Storage Driver (iSCSI, FC). - New driver features were added, notably, Fujitsu ETERNUS DX extend volume on RAID group, Pure Storage synchronous replication, NetApp iSCSI LUN space allocation, Dell PowerFlex Active-Active support, Dell PowerMax configurable SRDF snapshots. - Designate now supports Catalog Zones (RFC 9432). This can improve the scalability of Designate pools managing a large number of zones and significantly reduce the provisioning time when adding additional DNS servers to a Designate pool. - Horizon now uses Django 4.2 as default and dropped Django 3.2 support. - Ironic has enabled RBAC support by default by changing the default values of [oslo_policy]enforce_scope and [oslo_policy]enforce_new_defaults to True. Additionally, we added [DEFAULT]rbac_service_project_name to define a project where users in that project are treated as having a service role. Please see Ironic release notes for full details. - Ironic has added the ability to drain active tasks from a conductor before shutdown. Sending a SIGUSR2 signal to an ironic-conductor will now attempt to complete running tasks with a timeout of [DEFAULT]drain_shutdown_timeout. No new tasks will be started on the conductor while it’s draining. - Support was added for the external-gateway-multihoming API extension. The L3 service plugins supporting it can now create multiple gateway ports per router. It is currently limited to the L3 OVN plugin. OpenStack Caracal is marked as Skip Level Upgrade Release Process or SLURP. According to this model ( https://governance.openstack.org/tc/resolutions/20220210-release-cadence-ad…) this means that upgrades will only be supported from the Antelope 2023.1 release. RDO Caracal 2024.1 has been built and tested with the recently released Ceph 18.2.0 Reef version (https://docs.ceph.com/en/latest/releases/reef/) which has been published by the CentOS Storage SIG in the official CentOS repositories. *Note:* Follow the instructions in [RDO documentation]( https://www.rdoproject.org/install/install-with-ceph/) to install OpenStack and Ceph services in the same host. During Caracal cycle, some projects have been retired or declared inactive upstream. As such, the following packages for some projects are not present in the RDO Caracal 2024.1 release: - cinderlib - dib-utils - ec2api-tempest-plugin - ec2api - murano - murano-agent - muranoclient - murano-dashboard - murano-tempest-plugin - sahara - sahara-dashboard - sahara-image-elements - sahara-plugin-ambari - sahara-plugin-cdh - sahara-plugin-mapr - sahara-plugin-spark - sahara-plugin-storm - sahara-plugin-vanilla - sahara-tests - senlin - senlinclient - puppet-ec2api - puppet-etcd - puppet-haproxy - puppet-module-data - puppet-murano - puppet-qdr - puppet-rsyslog - puppet-sahara During the next release we will continue working on retiring inactive packages in order to ensure RDO content quality and security. Contributors: - During the Caracal cycle, we saw the following new RDO contributors: - Marihan Girgis - Balazs Gibizer - Fiorella Yanac Welcome to all of you and Thank You So Much for participating! But we wouldn’t want to overlook anyone. A super massive Thank You to all 47 contributors who participated in producing this release. This list includes commits to rdo-packages, rdo-infra, and rdo-website repositories: - Ade Lee - Alfredo Moralejo Alonso - Amy Marrich (spotz) - Arx Cruz - Balazs Gibizer - Bernard Cafarelli - Bohdan Dobrelia - Chandan Kumar - Cyril Roelandt - Daniel Pawlik - Douglas Viroel - Emma Foley - Fabricio Aguiar - Fiorella Yanac - Gregory Thiemonge - Grzegorz Grasza - Harald Jensås - Jiří Podivín - Joan Francesc Gilabert - Joel Capitao - Karolina Kula - Lewis Denny - Lon Hohberger - Luigi Toscano - Luis Tomas Bolivar - Lukáš Piwowarski - Maor Blaustein - Marihan Girgis - Marios Andreou - Martin Kopec - Martin Magr - Mathieu Bultel - Michael Johnson - Nate Johnston - Pablo Rodríguez Nava - Priscila Gutierres - Rabi Mishra - Roberto Alfieri - Ronelle Landy - Shreshtha Joshi - Soniya Vyas - Szymon Datko - Takashi Kajinami - Tobias Urdin - Tristan De Cacqueray - Yatin Karel - Zane Bitter The Next Release Cycle At the end of one release, focus shifts immediately to the next release i.e Dalmatian. Get Started To spin up a proof of concept cloud, quickly, and on limited hardware, try an All-In-One Packstack installation. You can run RDO on a single node to get a feel for how it works. For those that do not have any hardware or physical resources, there is the OpenStack Global Passport Program. This is a collaborative effort between OpenStack public cloud providers to let you experience the freedom, performance and interoperability of open source infrastructure. You can quickly and easily gain access to OpenStack infrastructure via trial programs from participating OpenStack public cloud providers around the world. Get Help The RDO Project has our users(a)lists.rdoproject.org for RDO-specific users and operators. For more developer-oriented content we recommend joining the dev(a)lists.rdoproject.org mailing list. Remember to post a brief introduction about yourself and your RDO story. The mailing lists archives are all available at https://mail.rdoproject.org. You can also find extensive documentation on RDOproject.org. The #rdo channel on OFTC IRC is also an excellent place to find and give help. We also welcome comments and requests on the CentOS devel mailing list and the CentOS IRC channels (#centos, #centos-cloud, #centos-devel in Libera.Chat network), however we have a more focused audience within the RDO venues. Get Involved To get involved in the OpenStack RPM packaging effort, check out the RDO contribute pages, peruse the CentOS Cloud SIG page, and inhale the RDO packaging documentation. Join us in #rdo and on the OFTC IRC network and follow us on Twitter @RDOCommunity. You can also find us on Facebook and YouTube.

1 0

[horizon] Weekly meeting cancellation
by Tatiana Ovchinnikova 24 Apr '24

24 Apr '24

Hello team! Apologies for the short notice. I am not feeling well, so I have to cancel today's meeting. See you next week! Tatiana

1 0

[kolla-ansible] redeploy service to another host
by Andrej Moravčík 24 Apr '24

24 Apr '24

Hi. What is the correct procedure for redeploying (generic) service to another host group in Kolla-ansible multinode environment? I did not find such information in documentation. I have mistakenly deployed Opensearch to [control] host group instead of dedicated monitoring host. First, I tried to “rollback” deployment by disabling Opensearch in globals.yml (enable_central_logging: “no”) then performed "kolla-ansible -i multinode.yml reconfigure” but Docker containers “opensearch” and "opensearch-dashboards” are still running on all control nodes. Any advice would be greatly appreciated. Thank you. Regards — Andrej Moravčík ________________________________ Táto správa a jej prílohy sú dôverné. Prosím, narábajte s týmito informáciami ako s dôvernými a nezverejňujte, nekopírujte ani neposielajte tieto informácie bez predchádzajúceho súhlasu spoločnosti Aliter Technologies, a.s. Ak nie ste adresátom indikovaným v záhlaví tejto správy alebo želaný príjemca, upovedomte prosím spoločnosť Aliter Technologies, a.s. odpoveďou na tento e-mail a pôvodnú správu vymažte zo svojho systému. Aliter Technologies - registrovaná ochranná známka pre EU, USA This message and any attachments are confidential. Please treat the information as confidential, and do not disclose, copy or deliver this message to anyone without Aliter Technologies' approval. If you are not the addressee indicated in this message or an intended recipient please notify Aliter Technologies a.s. by return e-mail and delete the message from your system. Aliter Technologies - Registered in WIPO & U.S. Patent and Trademark Office

4 4

[Kolla-ansible] is there a way to swicth from docker to podman
by wodel youchi 24 Apr '24

24 Apr '24

Hi, I am preparing to upgrade a platform from yoga to the latest version of kolla-ansible 2023.2 I am wondering if there is a way to switch from docker to podman while upgrading? Regards. <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> Virus-free.www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai…> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

2 1

[ptg][openstack-ansible] OpenStack-Ansible PTG results for Dalmatian
by Dmitriy Rabotyagov 24 Apr '24

24 Apr '24

Hi everyone! Sorry for the late PTG report. But better late than never I guess :) This time we did not have a lot of discussion points, and mainly re-scoped not delivered topics from previous PTG, including: * Improvements to PKI role for integration with Hashi Vault and ability to handle encrypted keys with ansible-vault * ProxySQL implementation - it's long overdue, actually, but still seems beneficial * Incus support * Usage of deb822_repository module instead of apt_repository. This is going to be breaking change as does not assume backwards compatibility in a way of defining repositories when they're overridden (especially for deployments with limited connectivity) * Improving our documentation state. We are still lagging behind adopting all our docs to OVN driver being a default. We also need to document a way of defining pretty endpoint names (serving all endpoints from port 80/443 rather then unique ports per service). With that being said, we also reviewed what is feasible to finish for Caracal release. We still need to get some progress on following topics in upcoming weeks: * ovn-bgp-agent support: https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/909780 ** We have acquired the FRRouting role from Vexxhost (thanks for that!), but some changes to it are needed, ie EL9 support: https://review.opendev.org/q/project:openstack/ansible-role-frrouting+statu… * OVN driver support for Octavia: https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/868462 ** I need to look at testing of that driver and add corresponsive job to the role * Dealing with inactive projects and prevent new deployments during Caracal release: sahara, murano, senlin * Finalize Skyline support: https://review.opendev.org/q/topic:%22osa/skyline%22 * Enable Quorum Queues by default along with other oslo.messaging improvements: https://review.opendev.org/q/topic:%22osa/messaging_improvements%22 So thanks everyone for participation and let's keep the ball rolling!

2 1

[magnum] k8s cluster deploy stuck timeout
by Jaime Ibar 24 Apr '24

24 Apr '24

Hi all, I have installed magnum with no issues. After the installation, I tried to deploy a kubernetes cluster but after some time, the creation process times out and this always happens during the kube_cluster_deploy stage. I'm running 17.0.0 (Bobcat) so I tried with core os fedora-coreos-38.20230806.3.0 and kube_tag v1.26.8-rancher1 as stated in the documentation but no joy. I also tried different combinations of fedora core and kube_tag but same, no joy. I can ssh into the vms, podman exec into the containers but if I run kubectl version I get the following error The connection to the server localhost:8080 was refused - did you specify the right host or port? and tailing /var/log/heat-config/heat-config-script/ I get the following message ++ kubectl get --raw=/healthz The connection to the server localhost:8080 was refused - did you specify the right host or port? Any idea what might be the problem? TIA Jaime -- salu2 Jaime

4 5

[magnum] Weekly meeting cancelled
by Jake Yip 24 Apr '24

24 Apr '24

Hi all, I'll need to cancel this week's meeting as I am unwell. - Jake

1 0

[nova][ptg] 2024.2 Dalmatian PTG summary
by Sylvain Bauza 24 Apr '24

24 Apr '24

Hey folks, Sorry for being a bit late to provide this summary, but let me do it :-) So, we had in between 10 to 20 contributors during our 4 days. Wow, it was nice ! Thanks all of you that were there, and I was happy to see you even if it was the first time you were in. As a reminder, please don't open the main etherpad link [1] if you use a Web browser with an automatic translation modification feature. If you do it, then it would internally translate all the phrases directly in the etherpad as if you were directly modifying the etherpad by yourself. Please also make sure you don't accidentally remove or modify lines. In order to prevent any misusage, please rather look at this read-only etherpad link https://etherpad.opendev.org/p/r.3d37f484b24bb0415983f345582508f7 Now, let me explain the summary now (first, take a coffee if you want). First, we only had one cross-project meeting, with the Neutron team. I won't provide the summary for this as it was only a neutron discussion (this was about multiple port bindings support with only some neutron backends). Please rather look at the neutron etherpad or wait for its summary. ### Caracal retrospective and Dalmatian planning ### 19 blueprints were approved, 10 of them landed. We also had more contributors (38) than last cycle. We now unfortunately have >60 untriaged bugs so we discussed that. We agreed then that rather than triaging the bugs every week, we should propose a bug triage day on a periodic time and welcome any person who would want to help about it. We will also ask reporters for each bug report that is older than 5 years if they can try to reproduce the issue with a newer/supported release. We also discussed how to make sure we could review the feature implementations. Eventually, we agreed on the fact we should ask spec owners to discuss with us for having feature liaisons for their specs so we could have something like 'subteams' per feature implementation. For that, I'll explain it separately by a new email in the next few weeks. About the proposed Dalmatian planning, we'll have two spec review days (R-20 and R-13) but also a spec approval freeze day (a deadline, if your prefer) by R-11 (mid-July). We'll also have an implementation review day by R-10. ### Enhancing support for running nova services in k8s ### Gibi agreed on providing a backlog spec for explaining what we should modify in Nova. For graceful shutdown support, gibi will also create another spec to make oslo.messaging only supporting some RPC topics usage. ### Unified limits default behaviour with no registered limits ### We discussed some specific case you have today with unified limits : you need to register all the limits for all the resource classes you use for flavors, because if not, you wouldn't have quotas for them. We then agreed on the fact that all the 'core' resource classes (meaning the non-custom ones) should be registered. melwitt will provide a document for explaining it. For custom resource classes, we agreed on providing a relax mode (in the nova configuration) for accepting unregistered limits only for custom resource classes. That option would also have a strict mode for asking to register even the custom RCs. ### Missing instance action events ### We have a bug report about missing instance action events in our API [2]. We agreed on fixing this without requiring a new API microversion. For each of the missing events, ratailor will provide individual bugfixes and will also try to assert the events in Tempest scenarios. ### Ephemeral encryption feature ### As we started to merge some changes for this feature on Caracal, we also found some concerns that needed to be discussed during this PTG. Melwitt will provide a new spec for adding a new API microversion that will add new request parameters for snapshotting an instance. If an instance that has an encrypted disk needs to be rescued, either the rescue disk will also be encrypted or it will use the same key as the rescue image. Resize won't yet be accepted but melwitt will try to look how it could be unencrypted. Eventually, we also discussed the fact that compute services should check when moving an instance whether the target supports the encryption options. ### Nova Metadata optimisation ### In case the metadata API is called quite frequently, there is some read amplification. In order to fix this, fwiesel will create a blueprint for lazy loading some returned parameters and telling which metadata paths are the most called. For the moment, we won't do any cache invalidation feature as we first need to understand whether the lazy-loading helps. ### Extending memory encryption support ### For the moment, Nova only supports AMD-SEV for encrypting the instance memory. We discussed about two other hardware supports : AMD-SEV-ES and SGX. For AMD SEV-ES (new AMD SEV generation), we agreed on documenting some not configuration support by BIOS, modifying the inventories by using nested resource providers with the same resource class and maybe needing some reshape. Takashi will work on it for this cycle (live-migration wouldn't be done by this cycle) For SGX, we said we need a new blueprint and a spec. We are OK to create a new nested resource provider for the resource class, but we would like to know the current move operation limitations. ### Stateless firmware support ### Even if the libvirt driver recreates a new firmware when we're moving/resizing an instance, we discussed whether we should support a specific stateless firmware. We agreed on accepting it, so takashi will work on it, but will also fix the related bug reports [3]. ### Automatically detecting vTPM support ### Long story short, we accepted [4] as a specless blueprint. takashi will work on the implementation this cycle. ### vTPM migrations ### Even if Nova supports vTPM related instances, we don't support yet rebuild, rescue, shelve, live migration and evacuation. We agreed on fixing this cycle live-migration/evacuate as a priority by persisting the vTPM memory somewhere. We need to find a contributor that said, hopefully those weeks. A bit related, we discussed some live-migration other topic that was related to our SSH usage later, and we said in that topic that some kind of small object-store in nova (or elsewhere) could be useful for transferring instance-related local datablocks (like console logs, NVRAM or config drives), including vTPM memory state. ### Live migration options difference ### So, you probably know that we have a lot of live migration configuration options now, as least the fact tht the live_migration_uri option was deprecated in favor of live_migration_scheme and live_migration_inbound_addr. Unfortunately, those new options miss some way to provide some elements like user, port, path and query string. Eventually, we agreed on deprecating this cycle live_migration_scheme while *un*deprecating live_migration_uri. That won't mean that we'll remove live_migration_scheme quickly, at least until 2025.2 F (based on the SLURP policy) so distros shouldn't be afraid. ### Allow Userdata to be updated ### Well, that one is a long overdue feature request. crohmann kindly updated the spec [5] for this cycle based on our feedback from the last reviews, we agreed on reviewing that spec but a quick glance showed up it was going into the right direction. ### Clean up old compute service versions ### That one has been a long discussed topic, even if just tech debt related. The case is that we have old cruft of upgrade checks in our code that we could remove, but we were afraid of removing some unsupported yet useful 'upgrade enveloppe' between very old computes and new controllers. After lots of back and forths, we eventually agreed on having some kind of upcall check from compute to conductor that would signal this compute that it's too old and needs to self-disable, yet having some 'escape valve' (straighly quoting) in case it incorrectly disables things. Dansmith agreed to start scheming it. ### OpenAPI-compatible API reference documentation ### Stephenfin instructed us on the cross-project effort aiming to have project's api-refs automatically generated thru some tooling. In order to achieve that, there are a few gaps in our current code that need to be fixed, mostly about adding some response schema validation. stephenfin already proposed a spec [6]. We agreed on reviewing it. We also wanted the implementation to be iterative per API resource, to leave to the deprecated APIs untouched and to keep the existing api-ref until the new documentation is on par. Stephefin agreed to update his spec based on that feedback. ### SPICE-direct console support ### A new nova spec [7] was created this cycle by mikal in order to provide some hypervisor details for some external tool that could then pass a SPICE-direct console from a specific instance to some user. We discussed this feature and we said that the usecase was reasonable. That said, we had some concerns about the technical modifications but we'll discuss them in the spec review. ### Remote consoles API ### When an user asks Nova to create a new remote console for an instance, the user doesn't tell which kind of console they want to get, only the operator can set which console kind to use per compute. We agreed on providing a new API microversion for helping users which kind of console they want (html5 or tcp-direct for example) by a new request parameter. Fwiesel accepted to create a new spec for it this cycle. ### Resurrected computes should doublecheck the RPC messages they get ### Sorry for the terrible section name above, I tried to make it short. Basically, when a compute service disappears for some time and then returns, it can process any RPC request which was in queue. If some instance was evacuated in between, then this compute will dumbly try to process what was called earlier (like a move operation). We agreed on the fact this is a bug that needs to be fixed by adding some conditional against any move operation RPC targeted method, we don't know yet who could work on it, maybe tobias-urbin since he reported it. ### How can we get rid of eventlet in Nova ? #### I won't go into the details on what eventlet is but please consider that all nova services are hardly dependent on that python library for concurrency reasons, mostly the compute service (besides the API and other services for the workers usage). Sean Mooney proposed us a plan for using direct python threading (instead of asyncio) in order to replace our workers model and to create specific distinct threadpools for compute-related operations. We agreed on that approach so we'll let sean to create a blueprint (which will be approved as specless) and rebase his current series so we could review it. ### EOF ### That's a wrap now, my fingers are now quite bloody (they're not used to write that much). Sorry for the long summary, I just hope your coffee (or tea) was good. -Sylvain (who could have written his last PTG summary email if planets align) [1] https://etherpad.opendev.org/p/nova-dalmatian-pt g (please remove the empty char between 'pt' and 'g') [2] https://bugs.launchpad.net/nova/+bug/2058928 [3] https://bugs.launchpad.net/nova/+bug/1785123 and https://bugs.launchpad.net/nova/+bug/1633447 [4] https://blueprints.launchpad.net/nova/+spec/libvirt-detect-vtpm-support [5] https://review.opendev.org/c/openstack/nova-specs/+/863884 [6] https://review.opendev.org/c/openstack/nova-specs/+/909448 [7] https://review.opendev.org/c/openstack/nova-specs/+/915190

5 5

[kolla] opensearch build issue
by Andrej Moravčík 23 Apr '24

23 Apr '24

Hello, I am having trouble building opensearch image based on Ubuntu distro using Kolla 16.3.1 / OpenStack 2023.1. Has anyone run into the same problem? Any help appreciated. INFO:kolla.common.utils.opensearch:Preparing to unpack .../opensearch_2.13.0_amd64.deb ... INFO:kolla.common.utils.opensearch:Running OpenSearch Pre-Installation Script INFO:kolla.common.utils.opensearch:Unpacking opensearch (2.13.0) ... INFO:kolla.common.utils.opensearch:Setting up opensearch (2.13.0) ... INFO:kolla.common.utils.opensearch:Running OpenSearch Post-Installation Script INFO:kolla.common.utils.opensearch:ERROR: Something went wrong during demo configuration installation. Please see the logs in /var/log/opensearch/install_demo_configuration.log INFO:kolla.common.utils.opensearch:dpkg: error processing package opensearch (--configure): INFO:kolla.common.utils.opensearch: installed opensearch package post-installation script subprocess returned error exit status 1 INFO:kolla.common.utils.opensearch:Processing triggers for libc-bin (2.35-0ubuntu3.6) ... INFO:kolla.common.utils.opensearch:Errors were encountered while processing: INFO:kolla.common.utils.opensearch: opensearch INFO:kolla.common.utils.opensearch:E INFO:kolla.common.utils.opensearch:: INFO:kolla.common.utils.opensearch:Sub-process /usr/bin/dpkg returned an error code (1) INFO:kolla.common.utils.opensearch: INFO:kolla.common.utils.opensearch: INFO:kolla.common.utils.opensearch:Removing intermediate container 72435c692eff ERROR:kolla.common.utils.opensearch:Error'd with the following message ERROR:kolla.common.utils.opensearch:The command '/bin/sh -c apt-get --error-on=any update && apt-get -y install --no-install-recommends opensearch && apt-get clean && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100 Regards — Andrej Moravčík ________________________________ Táto správa a jej prílohy sú dôverné. Prosím, narábajte s týmito informáciami ako s dôvernými a nezverejňujte, nekopírujte ani neposielajte tieto informácie bez predchádzajúceho súhlasu spoločnosti Aliter Technologies, a.s. Ak nie ste adresátom indikovaným v záhlaví tejto správy alebo želaný príjemca, upovedomte prosím spoločnosť Aliter Technologies, a.s. odpoveďou na tento e-mail a pôvodnú správu vymažte zo svojho systému. Aliter Technologies - registrovaná ochranná známka pre EU, USA This message and any attachments are confidential. Please treat the information as confidential, and do not disclose, copy or deliver this message to anyone without Aliter Technologies' approval. If you are not the addressee indicated in this message or an intended recipient please notify Aliter Technologies a.s. by return e-mail and delete the message from your system. Aliter Technologies - Registered in WIPO & U.S. Patent and Trademark Office

2 4

[tc] How to indicate that a project will transition straight to EOL
by Stephen Finucane 23 Apr '24

23 Apr '24

o/ I've noticed that a variety of SDK-related projects have had 'Update .gitreview for unmaintained/wallaby' patches proposed against them, e.g [1]. I'm not sure this makes much sense. Both SDK and OSC are designed to work against any version of OpenStack so there's hardly a reason to keep stable branches around, let alone the unmaintained branches. I took a look through the Project Team Guide docs [2] but while I can find info on opting-in to keep older unmaintained branches around, I can't find any info on opting-out of their creation altogether. That brings me to my question: does such a thing exist, and if not, can it? How can we ensure we go straight from stable to EOL for these projects? As a separate point, we may wish to explore removing stable branches entirely for these projects. Open to input on whether this is even possible, or if their removal would have serious issues for folks. Cheers, Stephen [1] https://review.opendev.org/c/openstack/openstacksdk/+/911660 [2] https://docs.openstack.org/project-team-guide/stable-branches.html#unmainta…

5 7

[Neutron] [VPNaaS] VMs unable to hit tunnel remote cidr
by Karl Kloppenborg 23 Apr '24

23 Apr '24

Hi Neutron/VPNaaS teams. We run an openstack which neutron runs on the following VPNaaS setup: service_provider: VPN:strongswan:neutron_vpnaas.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default L3-agent vpn_device_driver: neutron_vpnaas.services.vpn.device_drivers.strongswan_ipsec.StrongSwanDriver With Neutron running the following service plugins: service_plugins: neutron_dynamic_routing.services.bgp.bgp_plugin.BgpPlugin,neutron.services.l3_router.l3_router_plugin.L3RouterPlugin,router,vpnaas,trunk,segments,bgp And Ml2 config Openvswitch_agent -> Security Group -> firewall driver: openvswitch We have an IPSec tunnel up between a remote site (Fortinet device) and us. Computers within the remote site can reach the VM’s internal IP’s on Openstack across the tunnel, however VM’s from openstack cannot reach the internal network on the remote side. A traceroute from the VM to a known IP on the remote side shows the IP instead transiting out the router gateway. Has anyone seen this before? It looks like something isn’t being picked up/a route not being set. Any assistance would be greatly appreciated. Thanks, Karl.

3 4

[nova][kolla-ansible] having issue running nova API bootstrap container
by James Leong 23 Apr '24

23 Apr '24

Hi all, I am currently trying to deploy a yoga version of OpenStack, However, I am having the following issues when doing a deployment using kolla-ansible. The error occurs on task "[nova : Running Nova API bootstrap container]." "os_traits.COMPUTE_ADDRESS_SPACE_PASSTHROUGH,", "AttributeError: module 'os_traits' has no attribute 'COMPUTE_ADDRESS_SPACE_PASSTHROUGH' Best, James

2 2

[tc] Weekly TC meeting agenda for Apr 23rd 2024
by Goutham Pacha Ravi 23 Apr '24

23 Apr '24

Hello Stackers, We're four weeks [1] into the 2024.2 release of OpenStack and there's a lot of exciting governance proposals at the table already. Please join us to discuss these at the weekly meeting of the OpenStack Technical Committee tomorrow, Tuesday April 23rd, at 1800 UTC in the #openstack-tc channel on OFTC. If you haven't already done so, JayF wrote up a detailed summary of our recently concluded PTG for this list [2]. We'll be using this to drive some of the discussions tomorrow. Thank you, Jay! The agenda for tomorrow's meeting is: * Roll call * AIs from the TC vPTG 2024.2 #link https://etherpad.opendev.org/p/apr2024-ptg-os-tc (Technical Committee vPTG notes) * 2024.2 TC Tracker #link https://etherpad.opendev.org/p/tc-2024.2-tracker (Technical Committee activity tracker) * Ongoing business ** Clarify the timeline for project to move to Inactive state (https://review.opendev.org/c/openstack/governance/+/908880) ** Appoint Wenxiang Wu as PTL of Skyline (https://review.opendev.org/c/openstack/governance/+/915108) ** Transition Freezer project to DPL (https://review.opendev.org/c/openstack/governance/+/914911) ** Assign Dmitriy Rabotyagov as Freezer PTL (https://review.opendev.org/c/openstack/governance/+/915727) ** Move docs tools into TC governance (https://review.opendev.org/c/openstack/governance/+/914950) * Other governance proposals that will transition to "formal-vote" * 2025.1 Elections * TC Summary Email * Open Discussion and Reviews See you there! Thank you, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] https://releases.openstack.org/dalmatian/schedule.html [2] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack…

1 0

[neutron] Bug deputy report (week starting on April 15)
by Elvira Garcia Ruiz 22 Apr '24

22 Apr '24

Hi folks, This is the Neutron deputy report for the week of April 15 - 21. Thanks a lot to froyo for taking a look at a wave of ovn-octavia-provider bugs that appeared on our Launchpad during these last couple of hours! High ------ - https://bugs.launchpad.net/neutron/+bug/2062385 - [ovn-octavia-provider] Member with FIP not reachable Assigned to Fernando Royo - Might actually just be a known limitation of OVN. - https://bugs.launchpad.net/neutron/+bug/2061883 - [fwaas] Duplicate entry <uuid> for key default_firewall_groups.PRIMARY' Unassigned Medium ----------- - https://bugs.launchpad.net/neutron/+bug/2062536 - [ovn] agent_health_check does not work for ovn agents Assigned to Liu Xie. In progress. - https://bugs.launchpad.net/neutron/+bug/2062511 - [ovn] ovn metadata agent occasionally down if SB is snapshoting Fix proposed by Liu Xie https://review.opendev.org/c/openstack/neutron/+/916387 - https://bugs.launchpad.net/neutron/+bug/2062009 - Neutron-server + uwsgi deadlocks whenr unning rpc workers Fix proposed by sebageek https://review.opendev.org/c/openstack/neutron/+/916123 Incomplete --------------- - https://bugs.launchpad.net/neutron/+bug/2061350 - [ovn] The 'ovn_client' of the L3 plugin is occasionally NoneType Undecided -------------- These have been recently moved to neutron launchpad from the octavia one because they belong to the ovn-octavia-provider: - https://bugs.launchpad.net/neutron/+bug/2062965 - octavia/ovn: filedescriptor out of range in select() Unassigned - https://bugs.launchpad.net/neutron/+bug/2063043 - octavia/ovn: missed healthmon port cleanup Unassigned And this was just filed about an hour ago, also ovn-octavia-provider related: - https://bugs.launchpad.net/neutron/+bug/2063081 - [ovn] connectivity to load balancer with FIP from a server connected to a router with external gateway Unassigned Currently Fernando Royo is taking a look at the three of them for further triage. Best regards, Elvira

1 0

[tc] Congratulations, Goutham Pacha Ravi is new TC Chair
by Jay Faulkner 20 Apr '24

20 Apr '24

Hi all, I'd like to congratulate Goutham Pacha Ravi on becoming the TC Chair for 2024.2 cycle [1]. Thanks for the opportunity to serve as chair for the last six months; please treat Goutham kindly during his time in the role as you did me. Thanks, Jay Faulkner OpenStack TC 1: https://review.opendev.org/c/openstack/governance/+/916024

9 8

[openstack-helm] Cleanup OSH ingress chart
by Vladimir Kozhukalov 20 Apr '24

20 Apr '24

Hi dear colleagues, Some time ago we switched all our test deployment jobs to using the upstream Ingress-nginx chart. So it looks like we can recommend using one of the third party ingress controller charts to all our users. The deployment requirements for third party ingress controllers like ingress class name and ingress pod labels are described in the documentation [1] and this script [2] can be used as an example. I suggest removing the OSH ingress chart to reduce the code base that we need to support. Opinions are very welcome. [1] https://docs.openstack.org/openstack-helm/latest/install/deploy_ingress_con… [2] https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deploy… -- Best regards, Kozhukalov Vladimir

4 4

[kolla][kolla-ansible] Promethus libvirt_export with project_name label
by Satish Patel 20 Apr '24

20 Apr '24

Folks, I am collecting metrics of libvirt using prometheus_libvirt_exporter and it works but I am trying to add project_name tag in each libvirt label so I can shorten them by project_name. It Looks like kolla default image of prometheus libvirt version doesn't have that option to send project name. Based on this repo - https://github.com/kumina/libvirt_exporter?tab=readme-ov-file We can add --libvirt.export-nova-metadata flag to send project_name label in each job. Do I need different versions of libvirt_exporter to make it work or am I missing something? # /opt/libvirt-exporter --help usage: libvirt_exporter [<flags>] Prometheus metrics exporter for libvirt Flags: --help Show context-sensitive help (also try --help-long and --help-man). --web.listen-address=":9177" Address to listen on for web interface and telemetry. --web.telemetry-path="/metrics" Path under which to expose metrics. --libvirt.uri="qemu:///system" Libvirt URI from which to extract metrics. --version Show application version.

1 0

[tc] TC vPTG 2024.2 Summary / Notes
by Jay Faulkner 20 Apr '24

20 Apr '24

Hi all, We had TC PTG sessions during the vPTG last week. I'm going to give an extremely high level summary of topics/action items here, as always for full context please see the etherpad: https://etherpad.opendev.org/p/apr2024-ptg-os-tc -- and, as always, remember TC decisions are made via gerrit patches; PTG conversations just help us rapidly spin up context for input to those decisions. # TC <> Community Interaction session Job failures were a hot topic during this session, with TC and community members discussing strategies for helping newer contributors troubleshoot and understand the importance of troubleshooting intermittent gate failures. Several suggestions were made, but no concrete actions were identified to take. Getting things done in Openstack / 'modernization' was the other topic discussed in this session. Some contributors identified that changes such as more strict typing may not be considered a straightforward benefit over duck-typed python. Generally there was no conclusion here either for the whole of OpenStack, however, individual projects should not feel like they need to wait for "permission" to perform these kinds of modernization activities. # OpenStack - where are we, where are we going ## User Survey Analysis Slaweq summarized the user survey in a governance patch, https://review.opendev.org/c/openstack/governance/+/900516, as usual (thank you!) and the TC members and community discussed it. It was noted that as he was summarizing the survey results, Slaweq indicated he found some comments which went against community spirit, attributing a lack of contribution to "not helping competition". Some of the other comments were around data and the validity thereof; such as the 24% of users who responded they don't upgrade OpenStack potentially answering that because they replace the cluster in-place. ACTION: Goutham volunteered to help reword the questions to avoid pitfalls mentioned during the session. ## Re-Visioning OpenStack aka "What is OpenStack Now" JayF, in proposing this topic, suggested that the mission of OpenStack had gotten so large, and the projects so varied, that it's difficult to know what OpenStack itself means -- much less manage it as a whole. This reality was potentially proven via the unfixed Murano CVE during the last cycle. We had some discussion around what being in OpenStack means, and it was reiterated that OpenStack projects are not required to have security issues managed by the VMT or their process. It was pointed out that one of the side effects is that it's difficult to determine what common OpenStack components to teach given the number on the periphery. There was a suggestion that the TC should be doing project reviews to address this concern, but no concrete action was taken. # TC - Specific new governance issues ## Affiliation Diversity Handling in OpenStack TC As most reading this likely already know, during the last TC election there was an issue where affiliation diversity requirements combined with the elected TC members came into conflict. While Bauzas resolved this conflict by stepping aside, we want to prevent this kind of situation from deadlocking in the future. There was a large amount of async discussion and comments on this topic, so I hesitate to summarize them here. Please review the etherpad directly for the myriad of ideas and issues presented by the large amount of engagement in this issue. ACTION: gmann offered to propose guidelines for resolving this in the TC charter. ## Raising the bar for Activity This topic was proposed in response to the Murano CVE and the PTL not participating in a security fix. This raises the question as to how we assessed Murano was an active project, and how to prevent this kind of issue in the future. No consensus was reached on any of the larger-scoped items, however, it was indicated by Julia that the new Cyber Resiliancy Act guidelines, when they are released, may require us to take some kind of action on this front. There were some small items which garnered consensus, such as marking all leaderless projects as inactive by default -- as originally discussed in the 2024.1 PTG. This was another very active discussion with lots of both sync and async engagement -- I strongly suggest you read the direct comments for this topic to get a full feeling of the breadth of this issue. ACTION: https://review.opendev.org/c/openstack/governance/+/914726 was proposed by gmann, which implements the consensus idea to mark leaderless projects as inactive by default. ## Eventlet/AsyncIO migration goal This issue was raised to give a venue for https://review.opendev.org/c/openstack/governance/+/902585 to get synchronous comments/discussion. Dan indicated he had not voted in favor of the change, and didn't plan to until he felt it had a reasonable likelihood to succeed in OpenStack -- early tests of this functionality failed. Jay indicated that his primary concern was elimination of eventlet; the new maintainer group does not plan on keeping eventlet alive forever. Bauzas pointed out that this is more than a goal -- we need dedicated effort in order to execute on this idea. Sean Mooney indicated that we need a non-wsgi service to migrate as an example. Along these lines, we had agreement to reduce scope of the goal to remove eventlet dependencies/requirements from shared OpenStack libraries to enable individual projects to have an easier time with proof of concepts, and revisit the larger goal after such a proof of concept exists. ACTION: JayF to propose a new, smaller scoped goal around removing eventlet dependencies. # Follow-ups from Ongoing TC Activities ## Unmaintained Migration pain points There have been some lingering pain points while migrating from the Extended Maintenance model to Unmaintained. Firstly, despite these branches being "unmaintained", we've set a minimum requirement of green CI with no Zuul config errors in order for a project to continue having their branch open for submission. This does not dictate a minimum level of testing; only that such tests are running and passing. These issues must be repaired in order for an unmaintained branch to remain open, but there are significant numbers that are failing. Please see Elod's recent email to the list for more information and deadlines on these cases. Secondly, tooling for unmaintained branch EOL still needs to have automation completed -- the etherpad contains a full list of the items needed. This goes along with the overall sentiment of this session: the unmaintained branch migration has created an enormous amount of extra work. ACTION: Please go look at Elod's email! This email being sent was a primary action related to this PTG session. ## PyPI Maintainer Cleanup The PyPi maintainer cleanup effort, detailed here: https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup has been ongoing for over a year. As of March 15, 2024, we still have 232 project/maintainer pairs to cleanup and 110 separate humans who have access to at least one openstack pypi deliverable. The TC discussed next steps; consensus was that in cases where openstackci holds owner permissions, we are in favor of using those permissions to cleanup where possible. However, in cases where openstackci does not hold owner permissions, it gets a bit more difficult and we have fewer options: either engage with the PSF's process to recover the pypi account, which could take months; or migrate to a different pypi deliverable name. Both are difficult and time-consuming. ACTION: TC to encourage PTLs to once again reach out to any pypi owners/maintainers who have not yet acted to attempt to cleanup their access. ## Revise the project inactive state process This was some specific, targeted discussion around if we needed to revise inactive policy to include the case which happened to Murano -- because our process dictates that inactive projects are only removed from the release prior to milestone-2, finding out Murano was inactive and removing it from the release after M-2 was a special case. This discussion centered primarily around whether any revision to the policy was needed. In the end, no specific action items were created. ## Leaderless projects Details on this item are in https://etherpad.opendev.org/p/2024.2-leaderless and have been sent to the list separately. # Allowing other languages for client facing tools (rust) Notes in the etherpad for this topic was minimal. Essentially, there is no restriction on what languages can be used -- the runtimes document is meant to be specific information about those languages, not restricting other languages that could be used. There were two primary concerns around this item, JayF asked why we were looking into new rust-based tooling instead of integrating with the existing openstack-rust community. Other contributors expressed concerns around being able to understand or debug code in different languages. No action items came out of this discussion. # OpenAPI in OpenStack There was minimal discussion around this item. There appeared to be clear consensus with moving forward with OpenAPI on a project-by-project basis, with any efforts towards creating a goal being pushed until after there is more adoption. # Deprecate all native clients in favor of OpenStackSDK/openstackclient Again, minimal discussion on this item. There seemed to be little objection to just getting it done, but there were no immediate volunteers to do the work. See the etherpad for some specific examples of gaps that need to be resolved. Thank you all for participating in the TC PTG. Please do not let these items fall by the wayside; if you had an idea that didn't reach consensus in the PTG, one possible way to move it forward is a concrete proposal where folks can discuss -- either to the governance repo, or to the mailing list. -- Jay Faulkner OpenStack TC

1 0

[kolla-ansible][Antelope][Heat] Heat UpdateWaitConditionHandle Problems
by Devin Patterson 19 Apr '24

19 Apr '24

Hello, We are attempting to use Heat stacks with wait conditions with nested stacks. We have a test stack similar to this one https://github.com/openstack/heat-templates/tree/master/hot/autohealing where we are scaling a template. We are testing with the proposed fix in the link below. https://review.opendev.org/c/openstack/heat/+/915209/2/heat/engine/resource… Problem being faced is when updating the heat stack, all of the resources are replaced. This is not ideal as we would like the server resources to update with rolling updates. The rolling updates do take place as described in the auto-scaling group, but only after all resources are replaced. Seems the _needs_update call for the wait handle is triggering a replaced for all resources in the nested stack which is not the desired outcome. 2024-04-19 18:12:03Z [stack_1-j190L]: UPDATE_IN_PROGRESS Stack UPDATE started 2024-04-19 18:12:03Z [stack_1-j190L.asg]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:04Z [stack_1-j190L.asg]: UPDATE_IN_PROGRESS Stack UPDATE started 2024-04-19 18:12:04Z [stack_1-j190L.asg.4s52xugnxsrl]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:05Z [stack_1-j190L.asg.fpe6jz2dccpg]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:05Z [stack_1-j190L.asg.4s52xugnxsrl]: UPDATE_IN_PROGRESS Stack UPDATE started 2024-04-19 18:12:06Z [stack_1-j190L.asg.4s52xugnxsrl.wait_handle]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:06Z [stack_1-j190L.asg.o7dhu7neswv4]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:06Z [stack_1-j190L.asg.fpe6jz2dccpg]: UPDATE_IN_PROGRESS Stack UPDATE started 2024-04-19 18:12:07Z [stack_1-j190L.asg.4s52xugnxsrl.wait_handle]: CREATE_COMPLETE state changed 2024-04-19 18:12:07Z [stack_1-j190L.asg.fpe6jz2dccpg.wait_handle]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:07Z [stack_1-j190L.asg.4s52xugnxsrl.instance1]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:07Z [stack_1-j190L.asg.4s52xugnxsrl.wait_condition]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:07Z [stack_1-j190L.asg.4s52xugnxsrl.wait_condition]: UPDATE_COMPLETE The Resource wait_condition requires replacement. 2024-04-19 18:12:07Z [stack_1-j190L.asg.4s52xugnxsrl.wait_condition]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:07Z [stack_1-j190L.asg.o7dhu7neswv4]: UPDATE_IN_PROGRESS Stack UPDATE started 2024-04-19 18:12:07Z [stack_1-j190L.asg.fpe6jz2dccpg.wait_handle]: CREATE_COMPLETE state changed 2024-04-19 18:12:08Z [stack_1-j190L.asg.o7dhu7neswv4.wait_handle]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:08Z [stack_1-j190L.asg.fpe6jz2dccpg.instance1]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:08Z [stack_1-j190L.asg.fpe6jz2dccpg.wait_condition]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:08Z [stack_1-j190L.asg.fpe6jz2dccpg.wait_condition]: UPDATE_COMPLETE The Resource wait_condition requires replacement. 2024-04-19 18:12:08Z [stack_1-j190L.asg.fpe6jz2dccpg.wait_condition]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:08Z [stack_1-j190L.asg.o7dhu7neswv4.wait_handle]: CREATE_COMPLETE state changed 2024-04-19 18:12:09Z [stack_1-j190L.asg.o7dhu7neswv4.instance1]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:09Z [stack_1-j190L.asg.o7dhu7neswv4.wait_condition]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:09Z [stack_1-j190L.asg.o7dhu7neswv4.wait_condition]: UPDATE_COMPLETE The Resource wait_condition requires replacement. 2024-04-19 18:12:09Z [stack_1-j190L.asg.o7dhu7neswv4.wait_condition]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:11Z [stack_1-j190L.asg.4s52xugnxsrl.instance1]: UPDATE_COMPLETE The Resource Server "instance1" [27bbecab-bca3-48f0-9b1b-75d240d741a4] Stack "stack_1-j190L-asg-fmddtpch4ekg-4s52xugnxsrl-n56r3c5cxvwm" [8c4b9dcb-d4e3-4ff5-a442-c2b4f6302c38] requires replacement. 2024-04-19 18:12:11Z [stack_1-j190L.asg.4s52xugnxsrl.instance1]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:11Z [stack_1-j190L.asg.fpe6jz2dccpg.instance1]: UPDATE_COMPLETE The Resource Server "instance1" [58cffc7c-9278-43eb-bc36-fbbf9537a580] Stack "stack_1-j190L-asg-fmddtpch4ekg-fpe6jz2dccpg-7ipvlcpw237k" [1fd36182-6ac6-4f7b-aa3c-8ddaac1d933c] requires replacement. 2024-04-19 18:12:11Z [stack_1-j190L.asg.fpe6jz2dccpg.instance1]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:12Z [stack_1-j190L.asg.o7dhu7neswv4.instance1]: UPDATE_COMPLETE The Resource Server "instance1" [61dbda35-f20e-4018-85ad-66ba2e03c506] Stack "stack_1-j190L-asg-fmddtpch4ekg-o7dhu7neswv4-bqwgkjmslxkq" [c4b73bac-fdc8-4a78-8c18-1963054bc2fe] requires replacement. Regards, Devin

1 0

[openstack-ansible][magnum-cluster-api]
by pahrialtkj＠gmail.com 19 Apr '24

19 Apr '24

I tried magnum-cluster-api on openstack ansible with Antelope version, I have finished creating the k8s control plane containers and then Run the magnum-cluster-api deployment, but at the time of running it an error was found, where the role openstack.osa.lxc_container_setup is not available, anyone knows how to solve it? root@aio1:/opt/openstack-ansible# openstack-ansible osa_ops.mcapi_vexxhost.k8s_install Variable files: "-e @/etc/openstack_deploy/user_secrets.yml -e @/etc/openstack_deploy/user_variables.yml -e @/etc/openstack_deploy/user_variables_octavia.yml " [WARNING]: Unable to parse /etc/openstack_deploy/inventory.ini as an inventory source [WARNING]: running playbook inside collection osa_ops.mcapi_vexxhost ERROR! the role 'openstack.osa.lxc_container_setup' was not found in osa_ops.mcapi_vexxhost:ansible.legacy:/etc/ansible/ansible_collections/osa_ops/mcapi_vexxhost/playbooks/roles:/etc/ansible/roles:/etc/ansible/roles/ceph-ansible/roles:/etc/ansible/ansible_collections/osa_ops/mcapi_vexxhost/playbooks The error appears to be in '/etc/ansible/ansible_collections/osa_ops/mcapi_vexxhost/playbooks/k8s_install.yml': line 36, column 15, but may be elsewhere in the file depending on the exact syntax problem. The offending line appears to be: - import_role: name: openstack.osa.lxc_container_setup ^ here EXIT NOTICE [Playbook execution failure] ************************************** ===============================================================================

2 5

[PTL][release][stable] Transition Zed to Unmaintained
by Elõd Illés 19 Apr '24

19 Apr '24

Hi teams, As 2024.1 Caracal was released recently and we are in a less busy period, now is the good time to call your attention to the following: In less than 2 weeks Zed is planned to transition to Unmaintained [1] state as planned date is May 2nd, 2024. (For details about Unmaintained state please see: [2][3]). I have generated the transition patches [4] for Zed's every deliverables. After the transition it will not be possible anymore to release off of Zed branch. So it is strongly recommended to do a *final* Zed release if there are any merged or open patches that would fix a significant bug. On the other hand, please be careful to not to release any breaking changes as there won't be possible to do another release! Please mark the transition patch with a -1 if your team plans to do such final release, or +1 if the team is ready for the transition. The schedule from now on is as follows: * patches with +1 from PTL / release liaison will be merged, thus those repositories will transition to Unmaintained * at the planned deadline (May 2nd) the Release Team will merge all transition patches (even the ones without any response!) * unmaintained/zed branch will be cut from HEAD of stable/zed * after the transition, stable/zed will be *deleted*! * there won't be official releases anymore for Zed Don't forget, that: * backports can be pushed to unmaintained/zed branch * openstack-unmaintained-core (or <project>-unmaintained-core groups if such exists) have the rights to merge patches (please see my former mail about Unmaintained gates [5]) *NOTE*: teams, please focus on wrapping up your libraries first if there is any concern about the changes, in order to avoid broken (final!!) releases! [1] https://releases.openstack.org [2] https://docs.openstack.org/project-team-guide/stable-branches.html#unmainta… [3] https://governance.openstack.org/tc/resolutions/20230724-unmaintained-branc… [4] https://review.opendev.org/q/topic:zed-unmaintained [5] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… Thanks for reading this, Előd irc: elodilles @ #openstack-release

1 0

[openstack-ansible][magnum-cluster-api] magnum-cluster-api deployment role 'openstack.osa.lxc_container_setup' not found
by pahrialtkj＠gmail.com 19 Apr '24

19 Apr '24

I tried magnum-cluster-api on openstack ansible with Antelope version, I have finished creating the k8s control plane containers and then Run the magnum-cluster-api deployment, but at the time of running it an error was found, where the role openstack.osa.lxc_container_setup is not available, anyone knows how to solve it? root@aio1:/opt/openstack-ansible# openstack-ansible osa_ops.mcapi_vexxhost.k8s_install Variable files: "-e @/etc/openstack_deploy/user_secrets.yml -e @/etc/openstack_deploy/user_variables.yml -e @/etc/openstack_deploy/user_variables_octavia.yml " [WARNING]: Unable to parse /etc/openstack_deploy/inventory.ini as an inventory source [WARNING]: running playbook inside collection osa_ops.mcapi_vexxhost ERROR! the role 'openstack.osa.lxc_container_setup' was not found in osa_ops.mcapi_vexxhost:ansible.legacy:/etc/ansible/ansible_collections/osa_ops/mcapi_vexxhost/playbooks/roles:/etc/ansible/roles:/etc/ansible/roles/ceph-ansible/roles:/etc/ansible/ansible_collections/osa_ops/mcapi_vexxhost/playbooks The error appears to be in '/etc/ansible/ansible_collections/osa_ops/mcapi_vexxhost/playbooks/k8s_install.yml': line 36, column 15, but may be elsewhere in the file depending on the exact syntax problem. The offending line appears to be: - import_role: name: openstack.osa.lxc_container_setup ^ here EXIT NOTICE [Playbook execution failure] ************************************** ===============================================================================

1 0

[oslo][release][requirements] broken watch_log_file depends on unmaintained pyinotify
by Takashi Kajinami 19 Apr '24

19 Apr '24

Hello, # This is mainly a oslo topic but I'm adding release team and requirements # team because we likely need new stable/2024.1 release of oslo.log. I looked into an old oslo.log bug[1] during the weekend and noticed that the watch_log_file option in oslo.log has been broken for some time. Looking into the problem further, I leaned that pyinotify[1] library, which is used in this feature, hasn't been updated for 9 years (!!). What is worse, pyinotify dependes on asynccore which has been removed from Python 3.12 and I'm skeptical that we can fix pyinotify to make it work with Python 3.12. There was an old discussion to replace pyinotify by inorify[3] but itnofiy hasn't been updated for 4 years and looks also unmaintained. I think the only option we have here is deprecating the functionality in 2024.1 so that we can remove it in early phase of 2024.2. Because we are quite close to 2024.1 GA, we have to make the decision early. I've already proposed the change for deprecation[4]. If you have any concerns then please reply to this email or leave your comments in the review. Thank you, Takashi [1] https://bugs.launchpad.net/oslo.log/+bug/1740111 [2] https://github.com/seb-m/pyinotify [3] https://github.com/dsoprea/PyInotify [4] https://review.opendev.org/c/openstack/oslo.log/+/914788 -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

3 5

[policy][ptg] RBAC 2024.1 Dalmatian cycle vPTG discussions summary
by Ghanshyam Mann 19 Apr '24

19 Apr '24

Hello Everyone, I am summarizing the RBAC discussion that happened on Wednesday. Complete notes can be found @ https://etherpad.opendev.org/p/rbac-2024.2-ptg Goal document: https://governance.openstack.org/tc/goals/selected/consistent-and-secure-rb… Tracking: https://etherpad.opendev.org/p/rbac-goal-tracking Total Attendees: 10-15 Current progress: --------------------- There are a few projects that progress in various phases, one of them is Tacker completed the phase-1. Current progress can be found @ https://etherpad.opendev.org/p/rbac-goal-tracking Phase-2 (service role): -------------------------- This has two parts, 1. Change APIs policy default which is meant for service-to-service usage only to allow for service roles. 2. Use service tokens to interact with other services. In 2nd case, we need to use both the user token and as well as service token. Service token will be used for authentication purposes only and user token to do all other operations. Slaweq updated that Neutron changed the policy but needs to change the token to be used by interaction with other service. Action item: - gmann to add the job to make service user token (have service role only) to validate/capture any failure while we do service token/policy changes. Enable 'enforce_scope' by default & drop this config option: ---------------------------------------------------------------------- We discussed the plan to drop the enforce_scope config option. Scope enforcement needs to be enabled by default and should not be configurable. I have proposed the plan in the below review, please review and level your feedback. - https://review.opendev.org/c/openstack/governance/+/915179 Before doing that we have one dependency[1] and Takashi needs to test the heat jobs to make sure enabling the enforce_scope by default does not break anything. AGREE: - Takashi to add a heat job to find if enforce_scope=True works perfectly and after that only we can enable the flag in oslo policy. - Based on the above testing results, change the default value to True to be done in oslo policy and release oslo policy so that CI adopts the new default value. Domain Manager Persona ------------------------------- Details are in the keystone spec[2], there are no objections to this use case. Request to review the spec and provide your feedback. Global reader ---------------- We discussed it at the Vancouver Summit also. There is no objection to adding it but we need spec to be up with details including the new role name. This new global reader can be added like 'admin OR global-reader OR project-reader' AGREE: - 'admin OR global-reader OR project-reader' approach sounds good, especially not to split the admin role and avoid breaking existing use cases. - johnthetubaguy will propose the keystone spec with the details. - gmann to help in implementation. I host the RBAC biweekly meeting[3], feel free to add any topic to the meeting etherpad ('Biweekly meeting' section) - https://etherpad.opendev.org/p/rbac-goal-tracking#L169 [1] https://bugs.launchpad.net/keystone/+bug/2059780 [2] https://review.opendev.org/c/openstack/keystone-specs/+/903172 [3] https://wiki.openstack.org/wiki/Consistent_and_Secure_Default_Policies_Popu… -gmann

2 2

Whitepaper Collaboration - Linux, OpenStack & Kubernetes
by Kendall Nelson 18 Apr '24

18 Apr '24

Hello Folks, As you hopefully know, the OpenInfra Foundation has a history of helping to define patterns and new usage patterns in our ecosystem through its collaboration on whitepapers. With the user survey, we have seen a distinct pattern of users leveraging an entirely open source stack to meet their needs - Kubernetes, OpenStack and Linux together. So, we are planning the compilation of a whitepaper further explaining this integration, planning to publish in September of this year. Continuing on the theme of collaboration, I would love help with both the writing and the reviewing of two sections in particular - the introduction and integration sections of the whitepaper. Other sections that will be included are model architectures and user case studies from OpenInfra member companies making use of this infrastructure pattern. If you are interested in collaborating on this whitepaper, please contact me directly and I will get you in the loop! Drop me an email to knelson(a)openinfra.dev -Kendall

1 0

[designate] Support for DNSSEC // Bump in the wire signer via BIND9 or Knot
by Christian Rohmann 18 Apr '24

18 Apr '24

Hello, I was wondering is anybody knew about the current state of DNSSEC signing for Designate-managed zones. Since Designate MDNS serves as primary they actually should do the signing / provide already signed zones via zone transfers. Adding support for DNSSEC this was last discussed for Kilo [1], but that spec was never finished, DNSSEC support never implemented. One approach to do this is using a bump in the wire signer [2][3][4] and have an intermediate BIND9 or Knot server doing the signing. Has anybody implemented something of this kind? If so, how do your users receive their initial DS / DNSKEY for the parent zone? Regards Christian [1] https://review.opendev.org/c/openstack/designate-specs/+/132571 [2] https://bind9.readthedocs.io/en/latest/dnssec-guide.html#bump-in-the-wire-s… [3] https://jpmens.net/2023/07/22/adieu-opendnssec-bienvenido-knot-dns/ [4] https://labs.ripe.net/author/anandb/dnssec-signer-migration/

4 7

[kolla] openstack centralized logging
by Eugen Block 18 Apr '24

18 Apr '24

Hi, I'm trying to find out how the "Venus" project could work. Unfortunately, there's not much to find yet. There's no installation guide as for other projects although it already exists since Zed release. I tried enabling it in my kolla all-in-one test deployment (2023.1), the api and manager container are running, but I have no idea how to get some data into or out of it. My conclusion is that it's not a mature project yet, could anyone comment on that? As an alternative, I enabled centralized logging in the same deployment as well, but unfortunately I can't use the dashboard because of [1]. So I don't really know what I can get out of opensearch in order to have a centralized logging. If possible, I would like to adapt that to a non-containerized openstack deployment. How can I change the opensearch version in my kolla deployment to be able to login and inspect what I got? Any pointers and comments are highly appreciated! Thanks! Eugen [1] https://bugs.launchpad.net/kolla-ansible/+bug/2060306

4 9

[tacker][ptg] Dalmatian PTG Summary
by Yasufumi Ogawa 17 Apr '24

17 Apr '24

Hi all, Here is a summary of Tacker team's two days discussion during vPTG 2024.2[1]. # Revise FTs We've tried to revise functional tests on zuul for the latest cycles to reduce the burden of rechecks for unexpected failures which are sometimes happened especially at the end of the cycles. We have several targets for the revise, such as clean dir structure, reduce the amount of test scenarios or so, and share the status of each tasks. # Progress report for Legacy API deprecating/obsoleting Done dropping all the regacy codes from tacker and its client, but still remained in tacker-horizon. We're going to complete this task in this release. # Opentelemetry plugin Support for opentelemetry's observability was proposed in Caracal, but couldn't be finished because of some more higher priority tasks. We've agreed to change the plan to release the feature is this release. # Support devstack install for Cilium as a CNI of Kubernetes VIM We released a sample VNF package using Cilium CNI for demonstrating VNF applications can communicate via Cilium's high performance connectivity for considering vRAN usecases. # Demonstration for tacker-horizon NFV Orchestration API v2.0 Shared the latest GUI support of tacker-horizon for managing NFV orchestration via ETSI SOL v2.0 APIs. # TOSCA 1.3 support Tacker supports TOSCA as a VNF descriptor defined in OASIS standards[2], but the latest version in the support is TOSCA 1.2. Although there are two choices for us for newer ones, 1.3 or 2.x, we've agree to introduce 1.3 which is considered as the most active version currently and shared the implementation plan in this release. # Proposal for adding Tacker installation with openstack-helm in offical OpenStack-helm document We added tacker support in openstack-helm without documentation in the previous cycle. So, we'll complete the remained task in Dalmatian. # Fix schema definition error of TST 2.6.1 repository TST is an ETSI compliance test suites and each of its versions is compatible with SOL specification's as a maintenance policy in the community. We're going to complete to implement TST 2.6.1. The problem is that there is unexpected data field `virtualLinkDescId` in TST is incompatible with SOL standards. It should be in TST while keeping the versions. We'll propose the issue and fix to the standards. # Performance improvement of "list" APIs There is a performance issue in APIs for retrieving entries from a user feedback and it can be a problem especially in commercial usecases managing large amount of objects. There are several possibility of the performance issue such as lack of database optimization or implementation design is not still well considered. For the issue, we've started to figure out the cause of the issue from some more detailed experimentation. [1] https://etherpad.opendev.org/p/apr2024-ptg-tacker [2] https://groups.oasis-open.org/communities/tc-community-home2?CommunityKey=f…

1 0

[all][unmaintained] Gate status of Unmaintained branches
by Elõd Illés 17 Apr '24

17 Apr '24

Hi, Recently, based on the 'Unmaintained replaces Extended Maintenance' resolution [1], old branches (Xena, Wallaby, and Victoria), originally in Extended Maintenance, moved to Unmaintained, together with Yoga branch that transitioned directly from maintained to Unmaintained as a first such branch. From now on, there is the possibility for folks who are interested in further maintaining these old branches (in the so called 'Unmaintained' state :)). For that, maintainers need to sign up to be a member of openstack-unmaintained-core group (or the project specific group, if such exists). There is no requirement to be part of that group, but we would like to ask 'unmaintainers' to follow the stable policy [2], if possible, for unmaintained/* branches as well (*if the given project follows it on maintained branches). To signal your interest, please write a mail to this mailing list, or ping openstack-unmaintained-core group members (for example me :)) on IRC. As a first important thing, i would like to call attention to the following part of the resolution: 'The CI for all branches must be in good standing at the time of opt-in. At a minimum this needs to contain all integrated jobs, unit tests, pep8, and functional testing. However, as this is a best-effort CI and to preserve resources, Unmaintained branches will include periodic jobs of no higher than monthly frequency.' [1] However, while many gates look healthy [3] (i.e. patches have merged), there are still a significant number of failing CI gates [4]. So, to these projects and their unmaintained/.* branches, to be eligible to kept open for further maintenance and be able to opt-in in the next cycle for maintenance, these gates have to be fixed as soon as possible. Otherwise project branches need to be deleted and transitioned to End of Life. As an example for a simple gate fix, many project/branch need to drop their failing 'lower-constraints' job (or fix them). Also a good starter is to fix zuul config errors [5] if there exists any. Final note from me: as the above quote shows, the concept of good standing CI is quite minimal. Any further CI jobs (devstack based tempest jobs) add value to these branches for those who consume them. So any such work is really appreciated. If you have any question, feel free to ask, either here, on the mailing list, or ping me or the TC on IRC. Thanks for reading this, Előd Illés irc: elodilles @ #openstack-release #openstack-stable #openstack-unmaintained [1] https://governance.openstack.org/tc/resolutions/20230724-unmaintained-branc… [2] https://docs.openstack.org/project-team-guide/stable-branches.html [3] https://review.opendev.org/q/branch:%5Eunmaintained/.*+label:Verified%252B2… [4] https://review.opendev.org/q/branch:%5Eunmaintained/.*+label:Verified-1,225 [5] https://zuul.opendev.org/t/openstack/config-errors?branch=unmaintained%2Fyo…

1 0

[heat] Parametrization of Neutron trunk subports scaling down
by Lajos Katona 17 Apr '24

17 Apr '24

Hi, With Hot templates we run into a scaling issue with Neutron trunk subports. The task is to create Neutron trunks with "vlan_count" number of subports, and with stack update be able to add or remove subports of the created trunk. We tried with resourceGroup, and that works well if we increase the number of subports. The problem comes with decrease, as with resourceGroups I can't (my lack of knowledge) tell heat_engine that first update the trunk and remove the subport and after that delete the port. And Neutron API gives back conflict as the port which heat_engine tries to delete is still used by the trunk. Do you know a way to accomplish such an operation with Hot templates? Thanks in advance for the help. Best wishes Lajos Katona (lajoskatona)

2 4

[magnum] Dalmatian PTG summary
by Jake Yip 17 Apr '24

17 Apr '24

Hi all, Magnum held our PTG on 2024-04-10 0600-0800 UTC. I have summarised some points that may be of interest. 1. Magnum team is small, we welcome help with reviews, documentation, deprecation. 2. In Caracal, we have done a lot of work improving the CI. This allowed us to confidently remove old code that has been dragging us down. A big shout out to mnasiadka for this effort. For Dalmatian, we are improving this further with sonobuoy tests, etc. 3. In Caracal, we enhanced the driver code to support multiple drivers. Historically, Magnum drivers were in tree. It was difficult to support out-of-tree drivers without clashing. With Caracal, now an operator can have multiple active drivers in their installation, and can also move to new drivers in the future. 4. ClusterAPI driver There are two ClusterAPI drivers developed by different organisations (outside Magnum). - magnum-capi-helm - out-of-tree, in governance - https://github.com/openstack/magnum-capi-helm - magnum-cluster-api - out-of-tree, outside goverance - https://github.com/vexxhost/magnum-cluster-api StackHPC as kindly offered to move their magnum-capi-helm driver to OpenStack governance, and will work with Magnum team to get it working in CI in Dalmatian. Thanks mnasiadka, mkjpryor, johnthetubaguy, travisholton, for this work. A number of users are also reporting success with magnum-cluster-api driver. Feel free to try this out and ask questions on ML. Thanks mnaser who have been active on the ML answering questions. 3. Heat driver Currently the Heat driver supports Kubernetes v1.27 in Caracal. We aim to get v1.28 supported in Dalmatian. v1.29 is already supported in both ClusterAPI drivers. Heat driver still needs to be supported for possibly one/two cycles once ClusterAPI driver is available, so we target F for removal of Heat driver. For more information, please see the etherpad [1], or join the Magnum meeting if you have any questions. Regards, Jake

1 0

[qa][ptg] QA Dalmatian vPTG Summary
by Martin Kopec 17 Apr '24

17 Apr '24

Hi everyone, here is a summary of the discussions. The full agenda can be found here [1]. == Caracal Retrospective == We still manage making releases, doing code reviews and bug triaging even with only a few active contributors. Kudos to them. == Simplify grenade skip level jobs == It was agreed that we want to run grenade-skip-level job always as voting by default in template and project can opt-out for non-SLURP release. == Replace paramiko with libssh2 == It was agreed that the priority of getting rid of paramiko should take place at services' side first. After all services are done with this change, then we can proceed with this on tempest's side. In the meantime, we at least know that tempest works with the FIPS environment [2]. == Triage previous and write new priority items == The list of the priority items planned for this cycle as well as the ones which were carried over from the previous cycles, can be found here [3]. [1] https://etherpad.opendev.org/p/apr2024-ptg-qa [2] https://zuul.openstack.org/builds?job_name=tempest-centos9-stream-fips&skip… [3] https://etherpad.opendev.org/p/qa-caracal-priority Thank you all for attending the PTG and contributing to the discussion, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 0

volume in use
by kjme001＠gmail.com 17 Apr '24

17 Apr '24

helllo, I have the following problem that after deleting the instance, the volume remains in use and is connected to an instance that does not exist. I have to execute the command: cinder reset-state --state available --attach-status detached name_volume then the volume becomes available and is not connected to the instance. How to solve this problem? Regards

2 1

[manila] 2024.2 Dalmatian PTG summary
by Carlos Silva 16 Apr '24

16 Apr '24

Hello everyone, Thank you for participating in the PTG last week! You can check the notes in our PTG etherpad [0], and the recording for all sessions is available in the OpenStack Manila youtube channel [1]. Here is a brief summary of what we discussed and the action items: *# Caracal Retrospective:* *Highlights:* Great internship rounds (manila-ui, docs, SDK and bandit testing). SQLAlchemy 2.0 migration is close to completion and we are planning to backport it. Doc bug squash was a success. We managed to close almost 70% of the bugs we signed up for. FFEs are allowing little time for testing and we are willing to take some different actions. *AIs:* - Schedule a mid cycle check-in with our PTG etherpad to ensure all the topics we brought up are being addressed. - Collaborative review sessions now should be proposed alongside the feature when submitting the features to gerrit. - Bug squashes and hackathons will be announced earlier, so we can try to have more traction. *# Enhance PoolWeigher to respect current share network* The PoolWeigher doesn't take into consideration the share server per share network allocation. As a result, the weigher will take one of these approaches: - Stack: create all shares in a pool even though a newly added pool is available until it is full - Spread: create shares in the pool with fewer shares Instead of looking if there is already any share server on the pool, weigh according to if there is already any share server of the same share network on the pool *AIs:* - carthaca will create a launchpad blueprint with these notes for permanence. - Enhance the poolweigher to consider the share network in the request. *# NetApp Snaplock/WORM feature* An RFE has been filed for supporting WORM shares [2] Retention time should be configurable. We can implement such things through extra specs or new APIs for such shares. *AIs*: - NetApp/SAP will ask around if there are other vendors that have the same feature. - If they don't, we should take the extra spec route and add the NetApp specific metadata keys. *# Cross-service-request-id + Enhancements to logging* We need to enable x-openstack-request-id, and use it when making cross service requests (to cinder/glance/neutron) for better tracking in the cross-service logs. We could have an implementation in Manila similar to what is available in Nova and Cinder [3]. This could become an internship theme. *AIs:* - carthaca will report a bug to document this request *# Backend driver operations based on resource metadata update API* Metadata currently is being used as tags to refer to db resources and when it is updated, we don't check if the share back end needs to update anything in the share. The backends should be notified of metadata modifications in case there is need to update something in the shares. Where metadata changes should be reflected to Backend drivers: snapshot policy, show-mount option *AIs:* - kpdev will document the proposed new behavior into a specification *# Technical Debt - SRBAC* gouthamr talked us through all of the work that is complete and what are the next steps in the next few cycles. *AIs:* - Increase test coverage. - Set "[oslo_policy]/enforce_new_defaults=True" as a default and see if it works. - Introduce the manager role for reset-state, promote out of sync replicas, force-delete resources. *# Retrospective on internship projects* We discussed what we have been working on with the ~10 interns we had in Manila at the end of the Caracal cycle, as well as our plans for Dalmatian There are also some mentoring opportunities in case people are interested. Please reach out :) *# All things CephFS* ashrodri talked us through the updates from the previous cycle and all the testing she has been doing with CI and enhancing the CI jobs, as well as introducing the ingress daemon service and so on. We are hitting some package issues for testing, and we are trying to figure those out. On new features we are planning: *- Manage/unmanage implementation in the CephFS driver* - This is already available to other drivers but we need to implement this to the CephFS drivers. We expect to be able to start on this work during the Dalmatian cycle*.* *- **Impact of the new ensure shares API in the CephFS driver* - Ensure shares comes from a request from CERN, while they had their Ceph Mon IPs changing but they needed to restart the services to reconstruct the export locations. - It ended up becoming a feature that will be used by many people and we are trying to make this as generic as possible, as it can also be useful for backend migrations. - A spec is being worked on by carloss [4] and we expect to complete this work during the Dalmatian cycle. *- DHSS=True implementation with the CephFS driver*. - We received several requests to have DHSS=True available for the CephFS driver and we are willing to start on this work after another request showed up [5]. - *AI*: gouthamr will propose a spec to document how it will work. *- NFSv3 known issue* - NFSv3 has a large number of deficiencies that NFS-Ganesha can't fill - there are external services like rpcbind, statd that handle portions of the NFS protocol that are natively supported in NFSv4.1+ - However, Microsoft Windows doesn't have a native NFSv4.1+ client; which means Windows guests can't mount CephFS/NFS shares - until now. - With Ceph Reef, the NFS service enables v4 and v3 export rules; which means a share can be mounted with NFS v3 or NFS v4.1+ - It isn't advised to mount with both protocols at the same time, nor is it advised to use NFSv3 outside of windows - because there's no recovery in case there's a failover in the NFS server *- Async mirroring* - This feature will use cephfs mirroring [6], which sounds like share replication but it has some differences. - It works similarly to the way it was designed in Cinder [7]. - *AI*: ashrodri will work on a blueprint to track this effort. *# Backup APIs* Backups are available for some releases, and NetApp + SAP are planning some enhancements and are willing to include a new backup type entity in Manila, similar to share types We agreed that the backup type properties should reuse the metadata mechanism to reuse code and UX, so we will end up also having backup type metadata. Migrations can be a concern, as the backup types were configuration blocks in the manila.conf file, so we will rely on administrators to properly migrate that, or even provide a script to do such migration. *AIs*: - NetApp will work on a specification for the backup updates *# Backup driver implementation with CBACK* CERN is working on adding a driver that integrates with their backup solution (CBACK) to manila, which uses Ceph storages as the backups backend. Zach showed us some findings during the implementation and talked about his plans and some challenges. The plan is to introduce the new backup driver during the Dalmatian cycle and also start testing on CI with it. *# Bug management stats and cycle reports* vhari shared the bug stats after the previous cycle and how encouraging they are. We reflected on the data and had a suggestion to expand our bugsquash events to two in a cycle, so that can help us reduce the bug backlog quicker. We had great stats on reducing doc bugs backlog with the help of a bugsquash late in the previous cycle. We also talked about possible role rotation and discussed what it means to be the Manila bug representative. *# Provide support for efficiency policy on NetApp shares* NetApp is figuring out a way to determine efficiency policies during the creation of shares individually. They want to implement this by introducing new share type extra specs to configure such things. DHSS=True is still being discussed, and this feature will be worked on for DHSS=False first. NetApp intends to implement this feature during the Dalmatian cycle. *# Technical Debt - sqlalchemy 2.0* Thanks to stephenfin and zzzek, all changes for sqlalchemy 2.0 were merged in the beginning of the Dalmatian release, as some issues showed up in the process and we couldn't figure them out. We will be backporting the changes that didn't make it to Caracal, so sqlalchemy 2.0 will also be supported by Manila. There are no current plans for backporting this further. We also need to evaluate removing the backref to back_populates. *AIs:* - Complete the backport of the changes to 2024.1 caracal *# Add response schema validation* The SDK team would like to generate OpenAPI schemas stored in-tree, to ensure things are complete and up-to-date, and to avoid landing another large deliverable on the SDK team, and to allow service teams to fix their own issues. OpenAPI 3.1 is a superset of JSONSchema, which means we can use the same tooling we currently use for this. *AIs*: - stephenfin will work on a spec and on bootstraping it, and more people would be able to pick it up. *# Shares encryption* We had a brief discussion about the design of this feature for Manila, which the spec is already merged. kpdev has already proposed two patches for this change [8] *AIs:* schedule a collaborative review early in the cycle so we can start reviewing it as early as possible. *# DB migration script squash * tkajinam brough to our attention during the previous cycle that there was room for enhancement in our migration scripts, as there are 70+ of them now and we could squash them. We are also considering automated ways to do this with sqlalchemy. *AIs*: - gouthamr/tkajinam will reach out to zzzek and stephenfin and ask if there is an automated way to make this process easier. [0] https://etherpad.opendev.org/p/dalmatian-ptg-manila [1] https://www.youtube.com/@openstackmanila/ [2] https://bugs.launchpad.net/manila/+bug/2060027 [3] https://blueprints.launchpad.net/nova/+spec/cross-service-request-id [4] https://review.opendev.org/c/openstack/manila-specs/+/915480 [5] https://github.com/nfs-ganesha/nfs-ganesha/issues/1087 [6] https://docs.ceph.com/en/latest/dev/cephfs-mirroring/ [7] https://docs.openstack.org/cinder/latest/contributor/replication.html [8] https://review.opendev.org/q/topic:%22bp/share-encryption%22 If you have questions, please feel free to reach out. Looking forward to a very productive release cycle with you! Regards, carloss

1 0

[kolla-ansible][Antelope][Heat] Heat UpdateWaitConditionHandle Issues
by Devin Patterson 16 Apr '24

16 Apr '24

Hello, We are creating a workflow to use Heat auto-scaling groups with wait conditions. The ideal goal would be to have the wait condition clear and require another signal when the Heat stack is updated. Trying to use the resource OS::Heat::UpdateWaitConditionHandle instead of OS::Heat::WaitConditionHandle. This works during initial creation, but when I udpdate the stack with a new configuration, I get the following error. UpdateWaitConditionHandle.update() got an unexpected keyword argument 'new_template_id' Any ideas on how to use wait conditions during stack updates or to fix the above error. Thanks in advance. Devin

2 2

[Glance] Dalmatian PTG Summary
by Pranali Deore 16 Apr '24

16 Apr '24

Hello Everyone, We had our Dalmatian virtual PTG between 9th APRIL to 12th APRIL 2024. Thanks to everyone who joined the virtual PTG sessions. Using Opendev Meetpad we had all the discussion around different topics for glance. Here is the short summary of the discussions. Tuesday , APRIL 9th 2024 # Caracal Retrospective On a positive note we have completed few important things like adding support of new centralized_db cache driver which will be enabled default for caching related operations and stop using sqlite database from caracal release onwards and deprecated Sqlite cache driver, we also have deprecated glance-cache-manage command in favor of the new Cache API, glance scrubber utility and its associated configuration options, location_strategy option since in Bobcat, we added new weighing mechanism which made the location strategy no longer required and continued working on adding new location API support which will replace the image-update mechanism for consumers like cinder and nova for OSSN-0090 and OSSN-0065. On the other side we decided to speed up the work we've had in flight and speed up the reviews. # Add missing CLI support for some Glance API in Openstack Client The migration work for adding missing CLI was initiated in the antelope cycle. We have added the SDK support for cache API , images API and metadef API There are image-create-via-import, image-upload, member commands & metadef tags patches are under review. This cycle we are hoping to get all the above mentioned patches merged and mark this work complete. # Introduce new sub-store NFS for filesystem store Currently in glance when we configure nfs store, we do not store any information about what server nfs is configured, what mount point is configured, if it is available or not. From the beginning nfs is treated as a local FS store in glance. So if the NFS server is down , we don't have any idea about that, we write the image to the local FS store and it doesn't get reflected to the nfs store if nfs is down. We decided to have this support similar to cinder, when it configures nfs backend it checks at the initial level whether mount point is available or not and again checks during volume upload whether connected to nfs mount point We will enhance file system store by introducing some config parameters to store this information by creating separate ini file which can have information related to nfs server store with which we can check nfs server is reachable or not with the use of os-brick library. # Driver test coverage Currently glance has CI coverage for 'cinder', 'rbd', 'swift' stores and functional tests for FS store. Vmware driver is deprecated in 2024.1 but we decided to keep it and revise the status after updating the existing thread with the outcome in manila ptg. For S3 driver, we decided to ask for any interest in maintaining s3 driver (with adding CI, fixing bugs, etc) and depending on the response we will go ahead and add the coverage. # DB Migration script cleanup Glance has been using alembic migrations on the upgrading database schema from old version to the new version, currently glance maintains migration scripts from liberty and upgraded as we moved ahead cycle wise. So we decided to squash some of these[1] until the oldest release not EOLed and which is wallaby currently and decided to document this and from now onwards we will support migration scripts post EOL cycle and scrub the migration once the cycle is EOL. Wednesday April 10th , 2024 # Migrate existing images to new store at once Glance currently supports copying existing images to another available store via import workflow but one image at a time. If the deployer wants to desommision the existing store and add a new one, then all the images need to be copied to the new store and delete those one by one from the existing one. In onder to solve this problem we decided to provide glance-manage command line utility by introducing new operation copy-images to copy all or selective images in newly added store and if we add a flag 'delete_from_source' and set it to true we can delete all the images from the source store Example: glance-manage copy_images --source-store store_1 --destination-store store_2 --delete_from_source After all images are copied we will print the stat on the console - copied images, deleted images, ignored- images ( if user deletes any image in between the copy operation then, we will raise and catch the NotFound exception and add it to the ignored stores. Admin can check the ignored images issue and incase it's present then the process can be reinitiated) # Introduce common configuration option(s) at base store level which can be used/accessible as a common for all available stores? Since glance supports separate configuration options for each store incase of multi store enabled with the same store type. So we decided to introduce common configuration options at base store level which can be used/accessible as a common for all available stores similar to cinder [2] # Generic Image migration + Optimization for same store migration Currently the preferred way of migration in glance is two step 1. Copy the image from source store to destination store 2. delete the image from source store With the above approach there are 2 problems: 1. requires manual intervention from operators after waiting for image copy to finish and then delete the image from source store 2. No way for stores to optimize the operation We decided to address it by introducing a migration operation in glance which will have two features [3] 1. A generic migration workflow where we will perform the image copy and delete in the same API operation 2. Allow an interface for glance store methods to optimize the migration if possible else it will fall back to the generic workflow # New Location APIs We have decided to have support of new location apis which will replace the image-update mechanism for consumers like cinder and nova for OSSN-0090 and OSSN-0065. The implementation patches are up for review along with the unit/functional tests, documentation, api version bump, client & sdk side support and tempest coverage[2] The Poc patches of nova and cinder are also there to test this new location api functionality. During last cycle we have noticed few issues with cinder store while testing the cinder side POC patches, so the glance_store & cinder side fix is up for review, and once those patches are merged and glance_store is released, we can merged the New Location API patches after getting approval from Nova & Cinder side since both are the consumers of these 2 new APIs. We also decided to have a new CI job where all newly added tempest tests for these new APIs are passed and add command line utility to re-calculate the hash of the active images which are without hash if hash calculation fails during location-import. Thursday April 11th , 2024 # Add support to test whether swift is deployed or not We have noticed that when glance is configured to use swift as a glance store and swift service is not enabled/deployed/down then glance api service starts but image creation fails and image will remain in importing state forever. In case of import we upload the data at staging store first and then we try to import it to swift store, at that time We decided to add the support at the store side to check at the startup if swift is up and running and if it's not running then log a warning and disabled the store for adding & delete opearetion. # Deprecate and remove 'metadata_encryption_key' and related functionality 'metadata_encryption_key' config option and its related functionality was added quite a long back ago and even though as per the description it encrypts the location metadata, it actually encrypts location url. It encrypts the location url only for image upload/import/download/show APIs, doesn't encrypt url on location APIs. If it gets enabled during upgrade then it will break the existing deployment since existing image urls are not being encrypted. Hence we decided to deprecate it in this cycle and if there is no objection seen then we will remove it in the next cycle or may this the current cycle as well. # Shelved instance image protection This is an Optional feature proposal to prevent deletion of shelved instance's image. Currently when an instance is shelved its image can be removed and afterwards an instance cannot be booted again. Glance team has suggested checking the implementation of 'cinder_encryption_key_deletion_policy' and check whether it is possible to implement on a similar count and update the existing spec[4] accordingly. Apart from above topics you can find other miscellaneous topics discussed in PTG etherpad [5] and the recordings of all the sessions and milestone wise priorities at the end. If you have any questions/suggestions then please join us in our weekly meeting(each Thursday on #openstack-meeting irc channel at 1400 UTC) [1]: https://review.opendev.org/q/topic:%22db-migration-squash%22 [2]: https://opendev.org/openstack/cinder/src/commit/935f08e032b31d4628881d3a053… [3] : https://review.opendev.org/c/openstack/glance-specs/+/914639 [4]: https://review.opendev.org/c/openstack/glance-specs/+/915475/1/specs/2024.2… [5]: https://etherpad.opendev.org/p/apr2024-ptg-glance#L1

1 0

[ironic][PTG] Ironic 2024.2 PTG summary
by Riccardo Pittau 16 Apr '24

16 Apr '24

Hello Ironicers! Here's a summary of the discussions around the major topics at the Ironic PTG 2024.2 # Caracal retrospective # We took our time to look back at the Caracal development cycle, how it was for the ironic project, celebrate what we've accomplished, and of course also understand if some of the work started during Caracal needed to be continued in Dalmatian. We're quite happy to say that most of the goals we've set for Caracal have been reached, and we've actually been able to do much more than planned! For example we delivered the HttpBoot support for Redfish, we've been able to tackle a lot of old bugs and keep pace with the migration from storyboard to launchpad, almost finished the merge of Inspector into Ironic, and marked multiple old drivers for removal. # Present and Future of Ironic # We've analyzed the current status of Ironic and asked ourselves: "Where do we want Ironic to go?" In the past years we started looking beyond Openstack and we've took some steps to get into that direction, and we can say that Ironic went over the Openstack borders, becoming more independent. Currently Ironic can be used in three ways: - Openstack Integrated, the more clear and established way - as the core engine of Metal3, with good documentation for various use cases, but still growing and with margin for improvement - standalone, which still requires some work, a much clearer path forward and requirements We really want to keep investing in standalone Ironic more in the future, but to do that we need to take some decisions based on the vision we have on server provisioning from here to 5 years. Lots of questions we asked ourselves on how to resolve some problems connected to having a standalone bare metal provisioning service, mostly related to networking (e.g. secureboot, virtualmedia), will expand and find possible answers in other topics. # Redfish topics # Talking about present and future, Redfish is the current constantly evolving and improving standard protocol to directly communicate with controllers of bare metal devices. Ironic supports it since a while and of course there's more and more demand for new features to support or to expand the current capabilities. The discussion was mainly around an alternative, and possibly easier way to provide images for firmware updates and virtual media boot using a "push" model instead of the current "pull", with two proposals: - using the UpdateService already present in the Redfish standard - using the Virtual Media Push as a new feature currently under discussion with DMTF Another major topic is the support for TLS connection to the BMC, which already has a design proposal from one major contributor, Fujitsu. In addition we'll also have to explore the "other side" of the connection, so from the BMC to the ironic services, which will be one of the goals for Dalmatian. # CI related topics# CI has always had great importance for the Ironic project being one of the major consumers in terms of resources. It's not a case that that first related topic was how to slim down the CI usage! During the Caracal cycle we've been able to reduce the number of jobs run by the Ironic project, but we've also added some more. We came to the conclusion that we need to take an approach where we stack more tests in fewer jobs, trying to consolidate down jobs as much as possible and minimize boot interface variation testing. During the Dalmatian cycle we'll work first on trying to have Redfish and ipmi in a single ironic job, and update the list of jobs to understand where we can avoid duplication. At some point we'll also have to make space for some more deep tests on ARM machines. We've also decided to give Codespell tests a try, and explore the possibility of having some pre-commit hooks, and, after some technical considerations, move away from Tinycore as our base for the IPA ramdisk in CI, gentoo being the most likely candidate as a replacement. # xPUs # With xPUs in this case we mean IPU and DPU (not to be confused with TPU or GPU) which are some Processing Units that are designed for specific applications and data types. Usually DPUs are embedded in edge network cards, while IPUs are specifically designed for Artificial Intelligence/Machine Learning. There are multiple challenges in order to support those devices, depending on the "vision" of the vendors, some view them having full Operating Systems, some as having an OS just as "firmware", most or all of them would like remote management, which is where Ironic can really make an impact. Even though there are multiple use cases and scenarios where the use of xPUs is highly valued, and the wishlist of potential features is quite long, the first step we need to take is to get some real hardware and start testing, which is the real challenge we'll have to deal with next. # Secure Boot # We've made huge steps forward in supporting Secure Boot, the next big one would be supporting custom keys and providing means to upload and update them. Before doing that we need to clarify some aspects of Secure Boot, especially the "security vs convenience". In the Dalmatian cycle will focus mainly on updating our documentation, provide a better picture to users, and start exploring possible solutions for the secure custom keys management. # Future of Networking # Reconnecting to the future of Ironic, networking represents the next step for a truly standalone Ironic. This means finding alternatives to Openstack-integrated scenarios and therefore to Neutron. In Metal3 we're solving the problem using a separate IPAM, but this is still not enough for a complete Ironic standalone, in the end what we really need is flat network control, which at the moment we don't really have an alternative for. We realized that the topic is vast, we've just scratched the surface and we need to invest much more time on it, having more discussions to go deeper into it and start researching alternatives and tools for our goal. That is (almost) all folks! You can find all the topics and more details in the discussions notes here: https://etherpad.opendev.org/p/ironic-ptg-april-202 Feel free to answer directly in this thread in case you have questions or comments, or contact the ironic team on IRC. Thanks to everyone who participated in the PTG! Have a great Dalmatian development cycle! Ciao Riccardo

1 0

[publiccloud-sig] Reminder - next meeting April 17th - 0700 UTC
by Felix Kronlage-Dammers 16 Apr '24

16 Apr '24

Hi everyone, This Wednesday (April 17th) the next meeting of the public cloud sig is going to happen. We’ll be meeting at 0700 UTC here: https://meetpad.opendev.org/publiccloud-sig See also here for all other details: https://wiki.openstack.org/wiki/PublicCloudSIG See you tomorrow! felix -- Felix Kronlage-Dammers Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[all] ORM apocalypse mostly averted (a.k.a. SQLAlchemy 2.0 is here)
by Stephen Finucane 16 Apr '24

16 Apr '24

o/ A whole 2 years and 8 months to the day after I first communicated about it [1], it would appear the requirements bump to SQLAlchemy 2.0 [2] is finally ready to go. I suspect most if not all service projects are familiar with SQLAlchemy 2.0 at this point, having needed to fix whatever compatibility issues were seen. For those that aren't, I suggest reading my previous emails on the matter. So what happens next? Hopefully for the vast majority of projects the answer should be "nothing": many projects now have sqlalchemy-tips jobs that are designed to run unit and/or integration tests with the latest versions of SQLAlchemy, Alembic and oslo.db so the switchover should be seamless. I foresee three situations: * For projects that continue working as-is post-bump, you may wish to delete the sqlalchemy-tips jobs now. The jobs were designed as a stop gap measure to prevent projects regressing on SQLAlchemy 2.0 support while we got other projects over the line. While there is some limited value in keeping these jobs around, they do consume resources and - to the best of my knowledge - the SQLAlchemy community is not planning to release a 3.0 version of SQLAlchemy (or 2.0 version of Alembic) any time soon :) The requirements jobs gates on many projects and should identify compatibility issues with these libraries going forward, so it's probably best to drop them from the projects. I'll push a couple of patches shortly to do just this. Feel free to accept or reject them as you see fit. * For projects that are supposed to be SQLAlchemy 2.0 Ready (TM) but start seeing issues that the sqlalchemy-tips jobs didn't catch (because, say, the job was running unit tests rather than functional tests and the unit tests are missing coverage), I'd encourage you to look through what is now a fairly extensive repository of patches [3] that we've built up while addressing SQLAlchemy 2.0 compatibility issues. These are mostly all tagged with the sqlalchemy-20 topic and there's a very good chance that a patch addressing your specific issue has already been submitted to another project. * For projects that were not in the gating queue for requirements bumps, have not prepped for SQLAlchemy, and start failing catastrophically post-bump: have you not been listening to me?! :) You've got some work to do but fortunately that work should be relatively simple (if a bit tedious). As noted above, there are a *lot* of patches that you can look through to identify solutions to your compatibility issues. My preferred strategy - which is the one also advocated for by the SQLAlchemy 2.x docs [4] - has been to configure the warnings filter to raise an error for all SQLAlchemy deprecation warnings then filter out those that we actually hit and iteratively work to remove these filters. That won't work now that SQLAlchemy 2.0 is in upper-constraints so you're going to have to fix it all in one go. You may wish to adopt the standard approach locally though, before squashing all the commits upon review. This has already consumed far (far!) too much of my time so I won't offer to undertake the migration of any outstanding projects. I am, however, happy to field questions and point to existing patches that resolve a given issue. Hopefully none of it will be necessary though. Cheers, Stephen [1] https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.… [2] https://review.opendev.org/c/openstack/requirements/+/879743 [3] https://review.opendev.org/q/topic:sqlalchemy-20 [4] https://docs.sqlalchemy.org/en/20/changelog/migration_20.html

11 13