- openstack-discuss - lists.openstack.org

[openstack-ansible][masakari][yoga]Masakari ignoring host failure
by Danish Khan 11 Feb '25

11 Feb '25

Dear All, *Although, Masakari service seems to be working fine with few warning in one of my cluster but it is still ignoring all host failure:* Feb 10 10:34:04 compute7 masakari-introspectiveinstancemonitor[2379]: 2025-02-10 10:34:04.965 2379 WARNING masakarimonitors.introspectiveinstancemonitor.qemu_utils [-] Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied: libvirt.libvirtError: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied *Crm status shows no resources:* root@compute7:~# crm status Cluster Summary: * Stack: corosync * Current DC: compute6 (version 2.0.3-4b1f869f0f) - partition with quorum * Last updated: Mon Feb 10 10:24:49 2025 * Last change: Fri Feb 7 11:05:54 2025 by hacluster via crmd on compute7 * 6 nodes configured * 0 resource instances configured Node List: * Online: [ compute2 compute3 compute4 compute5 compute6 compute7 ] Full List of Resources: * No resources *From horizon instance-ha notification:* Notification UUID 0e745a900479 Source Host UUID 21c0d6d8e9f1 Type COMPUTE_HOST Status ignored Generated Time Feb. 10, 2025, 3:35 p.m. Created At Feb. 10, 2025, 3:35 p.m. Updated At Feb. 10, 2025, 3:35 p.m. Payload {'event': 'STOPPED', 'cluster_status': 'OFFLINE', 'host_status': 'UNKNOWN'} Regards, Danish Khan

2 3

[elections][PTL][Telemetry] Non-candidacy
by Erno Kuvaja 10 Feb '25

10 Feb '25

Hi all, After a recent shift on the work focus, I'd like to thank you all for the efforts and support navigating the Telemetry boat around. I've been blessed working with this small but actively pushing for better community and truly wish someone finds the energy to keep the momentum going. Especially after Jadnesh moved on to next journey recently as well. Can't wait to see what's next ahead for the project. - Erno "jokke" Kuvaja

1 0

[blazar] virtual instance scheduling
by Winicius Allan 10 Feb '25

10 Feb '25

When applying Blazar filter to check if have available hosts, it is considering allocation ratio? File "/var/lib/kolla/venv/lib/python3.10/site-packages/blazar/plugins/instances/instance_plugin.py", line 310, in pickup_hosts ERROR oslo_messaging.rpc.server raise mgr_exceptions.NotEnoughHostsAvailable() ERROR oslo_messaging.rpc.server blazar.manager.exceptions.NotEnoughHostsAvailable: Not enough hosts available The host in blazar aggregate has 48 vcpus and 8.0 allocation ratio, 40 of this vcpus are consumed. Considering the allocation ratio, I must be able to create a lease with more than 8vCPU. Anyone can help? Regards.

2 2

[neutron] Bug deputy report (week starting on February 3)
by Bernard Cafarelli 10 Feb '25

10 Feb '25

Hey neutrinos, here are the reported during the first week of February All bugs have a fix (merged in several cases) or assignee. Only pending one is the last one reported on neutron-vpnaas (but on EOL branch) Critical * [OVN] Neutron OVN metadata agent broken - https://bugs.launchpad.net/neutron/+bug/2097273 Fixed by https://review.opendev.org/c/openstack/neutron/+/940679 * networking-bagpipe CI is broken since linuxbridge driver removal - https://bugs.launchpad.net/neutron/+bug/2097367 Mentioned in last week team meeting, picked up by Lajos High * test_async_process_respawns_with_race_condition functional test fails randomly - https://bugs.launchpad.net/neutron/+bug/2097284 Recent new functional test failure - fixed with https://review.opendev.org/c/openstack/neutron/+/940687 * test_async_process_respawns functional test failing in fips functional job - https://bugs.launchpad.net/neutron/+bug/2095598 Another recent failure - fixed in the end with previous fix + https://review.opendev.org/c/openstack/neutron/+/939627 Medium * [eventlet-removal] Use ``ThreadPoolExecutor`` for ``periodics.PeriodicWorker`` - https://bugs.launchpad.net/neutron/+bug/2097257 Title says it all - reported and picked up by Rodolfo * devstack-tobiko-neutron failing in zed and 2023.1 with missing designate - https://bugs.launchpad.net/neutron/+bug/2097286 Tobiko job failures in unmaintained branches since job config cleanup - should be good with https://review.opendev.org/c/x/devstack-plugin-tobiko/+/940694 * [OVN] OVN metadata agent should build the response according to the content - https://bugs.launchpad.net/neutron/+bug/2097431 Fix when body response is empty with merged https://review.opendev.org/c/openstack/neutron/+/940679 Untriaged * No connection from outside host to VPNaaS host after upgrading from Ussuri to Victoria/Wallaby - https://bugs.launchpad.net/neutron/+bug/2097610 VPNaaS issue on EOL branch - I did not close immediately in case vpnaas experts have an idea -- Bernard Cafarelli

1 0

[cinder] Bug report about cinder-migrate
by Gioh Kim (김기오) 10 Feb '25

10 Feb '25

Hi, I'd like to report a bug I found when doing cinder-migrate test. I really would appreciate it if somebody inform me if it is solved at the latest tag or not solved yet. I checked the latest tag 24.2.0 (https://github.com/openstack/cinder/releases/tag/24.2.0) but failed to find a bug patch looking relevant to my issue. I tested the volume migration of nvme volume (nvme-tcp volume with LVM) with "openstack volume migrate --host <servername@diskname#typename> <UUID>" command. The first migration worked well but the second migration failed always with "Skipping remove_export. No NVMe subsystem for volume: <uuid>" error. Below is what I found If the original volume UUID is '23c5155e' that exists in server-0002: = The first migration: server-0002 => server-0001 = - Create a new LVM volume with a different name UUID 'ed7a9c7b' => new nvme subsystem name is 'ed7a9c7b' (server-0001) - Set the UUID of new volume 'A' and set new attribute "os-vol-mig-status-attr:name_id | ed7a9c7b" - Remove/unmap the original volume on server-0002: unmap nvme subsystem name is '23c5155e' = The second migration server-0002 => server-0001 = - Create a new LVM volume with a different name UUID 'af60b1c7' on server-0001 - Try to unmap the nvme subsystem of 'af60b1c7' of server-0001 ==> it failed because there is no subsystem 'af60b1c7'. It should've unmap the subsystem "ed7a9c7b". I guess there is a problem at unmapping a subsystem of already migrated volume. It happens 100% whenever I migrated a volume twice. FYI, I am still using a ZED version. I hope it's solved at the latest tag. The HEAD commit of my repository is below: commit 51dbd1e8061012bee4baca8aa60bf16a459279f9 Merge: d854529c2 cb4682fb8 Author: Zuul <zuul(a)review.opendev.org> Date: Thu May 11 18:00:27 2023 +0000 Merge "Reject unsafe delete attachment calls" into stable/zed Gioh Kim (김기오) Cloud Storage Email gurugio(a)linecorp.com

1 0

[tacker] Cancelling the next meetings
by Yasufumi Ogawa 10 Feb '25

10 Feb '25

Hi all, Since I'm going to be out of office tomorrow, I'd skip the next IRC meeting. Thanks, Yasufumi

1 0

[kayobe][a-universe-from-nothing] How does one restart controller0/compute0 after a reboot?
by Jordan Olin (OLINSolutions) 10 Feb '25

10 Feb '25

Hello, I’m following a-universe-from-nothing (2024.2) here <https://github.com/stackhpc/a-universe-from-nothing/blob/stable/2024.2/READ…>.in order to upskill on OpenStack deployment technologies. I’ve played a bit with the RDO-based installs, but a-u-f-n seemed closer to reality. My host VM is running Jammy (hostname: aufn-seed-jammy), with 64GB RAM and 100GB disk. I successfully made it through "Configure and deploy OpenStack to the control plane”, had running “seed”, “controller0”, “compute0” VMs, and was able to login to Horizon. I thought, “great! Let’s snapshot it all and keep going." So, I shutdown the host VM, aufn-seed-jammy (the one that is hosting “seed”, “controller0”, and “compute0”. I created a snapshot. I then restarted the host VM. The host VM (aufn-seed-jammy), and the seed VM started and I’m able to “ssh stack@seed” from the Host VM without problems. But… neither “controller0” nor “compute0” started up on their own. After sourcing Kayobe, from the host VM, I can “sudo virsh list —all” and see the following: (kayobe in ${KAYOBE_ENVIRONMENT:-base}) ubuntu@aufn-seed-jammy:~/deployment/src/kayobe-config/etc/kayobe$ sudo virsh list --all Id Name State ------------------------------ 1 seed running - compute0 shut off - controller0 shut off I then tried to manually start them: (kayobe in ${KAYOBE_ENVIRONMENT:-base}) ubuntu@aufn-seed-jammy:~/deployment/src/kayobe-config/etc/kayobe$ sudo virsh start controller0 error: Failed to start domain 'controller0' error: Cannot get interface MAC on 'p-contr0-0-tap': No such device I then ran an “ip a” to see what interfaces I have, but did not find one with that name. (kayobe in ${KAYOBE_ENVIRONMENT:-base}) ubuntu@aufn-seed-jammy:~/deployment/src/kayobe-config/etc/kayobe$ ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:37:79:fd brd ff:ff:ff:ff:ff:ff inet 192.168.4.131/24 brd 192.168.4.255 scope global dynamic noprefixroute enp1s0 valid_lft 277sec preferred_lft 277sec inet6 fe80::a8ed:6b67:f6ac:d095/64 scope link noprefixroute valid_lft forever preferred_lft forever 3: braio: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc noqueue state UP group default qlen 1000 link/ether de:55:16:2e:d0:3c brd ff:ff:ff:ff:ff:ff inet 192.168.33.4/24 brd 192.168.33.255 scope global braio valid_lft forever preferred_lft forever inet 10.0.2.1/24 scope global braio valid_lft forever preferred_lft forever inet6 fe80::dc55:16ff:fe2e:d03c/64 scope link valid_lft forever preferred_lft forever 4: p-braio-ovs@p-braio-phy: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc noqueue state UP group default qlen 1000 link/ether 66:18:cb:57:05:41 brd ff:ff:ff:ff:ff:ff inet6 fe80::6418:cbff:fe57:541/64 scope link valid_lft forever preferred_lft forever 5: p-braio-phy@p-braio-ovs: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc noqueue master braio state UP group default qlen 1000 link/ether 52:23:1b:ab:1d:0c brd ff:ff:ff:ff:ff:ff 6: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000 link/ether 52:54:00:d3:fa:cc brd ff:ff:ff:ff:ff:ff inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0 valid_lft forever preferred_lft forever 7: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc noqueue master braio state UNKNOWN group default qlen 1000 link/ether fe:54:00:aa:0f:f6 brd ff:ff:ff:ff:ff:ff inet6 fe80::fc54:ff:feaa:ff6/64 scope link valid_lft forever preferred_lft forever 8: dummy1: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1442 qdisc noqueue master braio state UNKNOWN group default qlen 1000 link/ether 8a:a1:62:09:27:46 brd ff:ff:ff:ff:ff:ff The only interfaces starting with “p-“ are “p-braio-ovs@p-braio-phy” and “p-braio-phy@p-braoi-ovs”. I’m not clear on and would appreciate any suggestions and advice on the following: 1) What are the commands and sequence to properly/cleanly bring down an “a-universe-from-nothing” based environment? 2) What are the commands and sequence to properly/cleanly start up an “a-universe-from-nothing” based environment? Thank you in advance for your time and considerations. Kind regards, Jordan. =========================== Jordan Olin, TOGAF EA President and Chief Solutioneer OLINSolutions, Inc. e: jordan(a)olinsolutions.com p: +1 508-982-4872 w: olinsolutions.com This email and any files transmitted with it are confidential and are intended solely for the use of the individual or entity to which they are addressed. This communication may contain material protected by government legislation. If you are not the intended recipient or the person responsible for delivering this email to the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. If you have received this email in error, please notify the sender by replying to this email and then delete the email from your computer.

1 0

[ovn][neutron] sriov port not able to get ip from DHCP
by Satish Patel 10 Feb '25

10 Feb '25

Folks, I have built a new openstack using kolla-ansible with 2024.1 with OVN plugin. So far everything is working good. Today I have added a new compute node with SRIOV port and it turns out vm is not able to obtain IP from DHCP. With the same subnet if I create general vm then it works and obtains IP but with sriov port it doesn't work. If I assign IP static on the vm interface then my VM is able to ping the outside world. I have deployed many clouds with OVN and SRIOV and never had an issue so I never dig into any kind of troubleshooting. This is the first time I have seen this issue. I have checked port mapping etc and all looks good. In SRIOV case what is the flow of DHCP? does physical interface send DHCP request on wire or DHCP request route via OVN br-int bridge? How will handle that DHCP request? Following two SRIOV port I have created but they are not able to obtain IP address via dhcp (ovn-nb-db)[root@os-chn-ctrl03 /]# ovn-nbctl --no-leader-only find Logical_Switch_Port type=external _uuid : a274df3b-6967-4f73-91ff-8f6eb46f70e0 addresses : ["fa:16:3e:03:55:44 10.74.2.35"] dhcpv4_options : 7829902c-5cdf-49cf-b478-7b683669c726 dhcpv6_options : [] dynamic_addresses : [] enabled : true external_ids : {"neutron:cidrs"="10.74.2.35/23", "neutron:device_id"="f4b91945-32f9-4961-a2cf-26b408218e35", "neutron:device_owner"="compute:sriov", "neutron:mtu"="", "neutron:network_name"=neutron-87131346-90d6-48ca-a045-214ff02ee9c0, "neutron:port_capabilities"="rx;tx;sg;tso;gso;gro;rxvlan;txvlan;rxhash;txudptnl", "neutron:port_name"=my-sriov-port, "neutron:project_id"="0c4956dce56a4503878007355850a04a", "neutron:revision_number"="32", "neutron:security_group_ids"="0bd505e8-d086-4d72-8e56-b913a8672f08", "neutron:subnet_pool_addr_scope4"="", "neutron:subnet_pool_addr_scope6"="", "neutron:vnic_type"=direct} ha_chassis_group : 0c41026b-3b01-485e-908a-523202aefcf9 mirror_rules : [] name : "0cbe012e-902a-4503-91a2-191b5f4acac5" options : {} parent_name : [] port_security : ["fa:16:3e:03:55:44 10.74.2.35"] tag : [] tag_request : [] type : external up : true _uuid : 1b245b6d-b720-41cd-a09c-79dc7eab3149 addresses : ["fa:16:3e:d8:4b:ab 10.74.2.181"] dhcpv4_options : 7829902c-5cdf-49cf-b478-7b683669c726 dhcpv6_options : [] dynamic_addresses : [] enabled : true external_ids : {"neutron:cidrs"="10.74.2.181/23", "neutron:device_id"="ac767d12-cff0-484e-b014-f3e42160cd8c", "neutron:device_owner"="compute:sriov", "neutron:mtu"="", "neutron:network_name"=neutron-87131346-90d6-48ca-a045-214ff02ee9c0, "neutron:port_capabilities"="rx;tx;sg;tso;gso;gro;rxvlan;txvlan;rxhash;txudptnl", "neutron:port_name"=my-sriov-p1, "neutron:project_id"="0c4956dce56a4503878007355850a04a", "neutron:revision_number"="5", "neutron:security_group_ids"="0bd505e8-d086-4d72-8e56-b913a8672f08", "neutron:subnet_pool_addr_scope4"="", "neutron:subnet_pool_addr_scope6"="", "neutron:vnic_type"=direct} ha_chassis_group : 0c41026b-3b01-485e-908a-523202aefcf9 mirror_rules : [] name : "bdf9261c-5cfc-4df1-9c11-c8ec26535eef" options : {} parent_name : [] port_security : ["fa:16:3e:d8:4b:ab 10.74.2.181"] tag : [] tag_request : [] type : external up : true

1 2

[nova] Wondering about dynamic overcommit scheduling
by Jacquet, Pierre 10 Feb '25

10 Feb '25

Hello, I am exploring the potential for resource consolidation from the initial placement perspective, specifically within the filter scheduler. From my research, I understand that OpenStack Watcher can addresses consolidation in a dynamic manner (monitoring resource usage and using live migration), while consolidation in nova seems to rely on a static overcommitment ratio, without accounting for real resource usage. Has dynamic overcommitment been considered for the initial placement phase? If not, I would like to propose implementing such a feature and would greatly appreciate any feedback or guidance from the community on this idea. Specifically, a custom filter could be interfaced with a monitoring stack to deploy servers as long as some thresholds are not exceeded Thank you for your time and input. Best regards, Pierre

3 2

[kolla-ansible] stuck at Discover nova hosts
by Satish Patel 08 Feb '25

08 Feb '25

Folks, I am deploying kolla-ansible 2024.1 but somehow it has been stuck here for the last 2 hours not giving me any kind of clue or error even when I run in -vvvv mode. Any idea what it's trying to discover and why there is no timeout for this task? TASK [nova-cell : Fail if cell settings not found] ************************************************************************************************************************ task path: /usr/local/share/kolla-ansible/ansible/roles/nova-cell/tasks/discover_computes.yml:7 skipping: [os-chn-gen-comp-001] => { "changed": false, "false_condition": "not nova_cell_settings", "skip_reason": "Conditional result was False" } TASK [nova-cell : Discover nova hosts] ************************************************************************************************************************************ task path: /usr/local/share/kolla-ansible/ansible/roles/nova-cell/tasks/discover_computes.yml:14 Using module file /usr/local/lib/python3.10/dist-packages/ansible/modules/command.py Pipelining is enabled. <os-chn-ctrl01> ESTABLISH SSH CONNECTION FOR USER: None <os-chn-ctrl01> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o 'ControlPath="/root/.ansible/cp/eaac57fe12"' os-chn-ctrl01 '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-pnxqbtjwqxuxexfjfrnpukckzrfbkegh ; /usr/bin/python3'"'"'"'"'"'"'"'"' && sleep 0'"'"'' Escalation succeeded

2 1