- openstack-discuss - lists.openstack.org

[neutron][octavia] vip port missing virtual-parent in ovn
by Maximilian Stinsky-Damke 03 Jul '25

03 Jul '25

Greetings, I'm currently debugging a strange issue related to Octavia loadbalancer failovers and was wondering if anyone has encountered something similar or has any insights. We're running Neutron with ML2/OVN and the ovn-bgp-agent. When we create Octavia load balancers using the Amphora driver, they come up fine. After attaching a floating IP, the ovn-bgp-agent correctly announces the FIP. However, during load balancer failovers, we sometimes observe that the FIP is no longer being announced. After some debugging, we found that the ovn-bgp-agent stops announcing the FIP because the VIP port in OVN is down and no longer has a neutron:host_id set in external_ids. Digging deeper, we noticed that the VIP port's options:virtual-parents only includes a single port ID and it's not the one corresponding to the current VRRP master. As a result, OVN doesn't associate the VIP with the active amphora, causing the port to be up=false and leaving the external_ids:neutron:host_id field empty. Because of this, no bgp-agent sees itself as responsible for announcing the FIP. Looking at the Octavia amphora ports in Neutron, all of them have the correct allowed address pairs set. This should normally result in the correct virtual-parents being reflected in OVN. Interestingly, I was able to reproduce this behavior even with the SINGLE topology in Octavia, a failover sometimes removes the entire options fields related to virtual-parents and virtual-ip from the VIP port. Manually unsetting and re-setting the allowed address pair on the affected port always restores the correct virtual-parents, and the announcement resumes. Now the question is how could this happen? It seems completely random, but it's very reproducible in my environment. We're running Neutron 2025.1 (26.0.0) with OVN 24.03.1-5.el9. I haven't found any errors in the Neutron server logs indicating issues writing to OVN, or any relevant tracebacks. Thanks in advance for any input or ideas! Best regards, Max

2 3

[cinder] not rescanning during VM attach
by Anthony Galica 02 Jul '25

02 Jul '25

Hi all, We have a customer problem where LUNs are not able to be scanned in time by cinder, so the operation fails. We fixed the problem in the "create image from volume workflow" by configuring "num_volume_device_scan_tries" in cinder.conf to a larger value. However, this doesn't appear to work for creating a VM and attaching to it, which still always fails. Any configuration or advice here? Thank you, Anthony

1 0

[cinder] How to retype while maintaining volume usage
by Yuta Kambe (Fujitsu) 02 Jul '25

02 Jul '25

Hi everyone, We are using a Ceph cluster as the backend for Cinder. The Ceph cluster has two pools: volumes_data_hdd for HDDs and volumes_data_ssd for SSDs. These are assigned to the Cinder volume types ceph-hdd and ceph-ssd, respectively. To change the type of a ceph-hdd volume to ceph-ssd, I executed the following command: openstack volume set --type ceph-ssd --retype-policy on-demand <ID of ceph-hdd volume> After execution, checking the capacity with the 'ceph df' command shows an increase in usage after the type change. The following is an example of migrating a newly created, empty 10GB ceph-hdd type volume to ceph-ssd: $ sudo ceph df | grep volumes_data volumes_data_hdd 19 1 98 GiB 25.51k 195 GiB 2.24 4.2 TiB volumes_data_ssd 22 1 91 GiB 23.89k 181 GiB 3.55 2.4 TiB $ openstack volume list --long | grep 7fd16e56-71ec-430b-8499-8d216a57f1c6 | 7fd16e56-71ec-430b-8499-8d216a57f1c6 | test | available | 10 | ceph-hdd | false | | | $ openstack volume set --type ceph-ssd --retype-policy on-demand 7fd16e56-71ec-430b-8499-8d216a57f1c6 $ openstack volume list --long | grep 7fd16e56-71ec-430b-8499-8d216a57f1c6 | 7fd16e56-71ec-430b-8499-8d216a57f1c6 | test | available | 10 | ceph-ssd | false | | | $ sudo ceph df | grep volumes_data volumes_data_hdd 19 1 98 GiB 25.51k 195 GiB 2.24 4.2 TiB volumes_data_ssd 22 1 101 GiB 26.45k 201 GiB 3.95 2.4 TiB Despite the volume being empty, the usage of volumes_data_ssd increases by 10GB. I suspect this is due to zero-filling of unused space during the copy process. Is it possible to change the volume type using Cinder functionality while maintaining the same usage? If not, I think it would be better to use the 'rbd cp' command internally for type changes within Ceph, as it should be able to copy the image while preserving the image size. What are your thoughts on this approach? OpenStack and Ceph versions are: $ ceph -v ceph version 17.2.8 (f817ceb7f187defb1d021d6328fa833eb8e943b3) quincy (stable) $ dnf list --installed | grep openstack-cinder openstack-cinder.noarch 1:21.3.2-1.el9s @centos-openstack-zed Best regards,

2 1

[publiccloud-sig] Next meeting of Public Cloud SIG - July 2nd (today) at 0700 UTC
by Felix Kronlage-Dammers 01 Jul '25

01 Jul '25

Hi everyone, This upcoming Wednesday (which is today, July 2nd) the next meeting of the publiccloud-sig is going to happen. We’ll be meeting at 0700 UTC here: https://meetpad.opendev.org/publiccloud-sig See also here for all other details: https://wiki.openstack.org/wiki/PublicCloudSIG Sorry for the short notice! Have fun! cheers felix -- Felix Kronlage-Dammers Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[Nova] - Upgrading to Epoxy causes Ubuntu cloud images to fail DHCP
by Karl Kloppenborg 01 Jul '25

01 Jul '25

Hi Team, We’ve got a weird one where other images (cirros, Debian, fedora etc) work fine. However upgrading to Nova Epoxy ubuntu fails cloud-init because network timed out. Note network loads fine and DHCP is received on cirros etc and setting config-drive for the installation Ubuntu works, and continues to work. It smells of some driver issue/change but I haven’t put my finger on it yet. Boot log: https://gist.github.com/inventionlabsSydney/3d491bb08ea96d1f32ed16fd6b58179a Image was uploaded as such: openstack image create --file ~/noble-server-cloudimg-amd64.img --disk-format qcow2 \ --min-disk 8 \ --min-ram 1024 \ --public \ --property hw_firmware_type=uefi \ --property hw_disk_bus=scsi \ --property hw_scsi_model=virtio-scsi \ --property hw_machine_type=pc-q35-focal-hpb ’Noble Test' Has anyone else seen this yet? Karl Kloppenborg Chief Technology Officer m: +61 437 239 565 resetdata.com<https://resetdata.com/> [cid:reset_69557fc2-1d63-4932-b5fd-93bd4f39ca7b.png] ResetData supports Mandatory Client Related Financial Disclosures – Scope 3 Emissions Reporting For more information on the phasing of these requirements for business please visit; https://treasury.gov.au/sites/default/files/2024-01/c2024-466491-policy-sta…<https://treasury.gov.au/sites/default/files/2024-01/c2024-466491-policy-sta…> This email transmission is intended only for the addressee / person responsible for delivery of the message to such person and may contain confidential or privileged information. Confidentiality and legal privilege are not waived or lost by reason of mistaken delivery to you, nor may you use, review, disclose, disseminate or copy any information contained in or attached to it. Whilst this email has been checked for viruses, the sender does not warrant that any attachments are free from viruses or other defects. You assume all liability for any loss, damage or other consequences which may arise from opening or using the attachments. If you received this e-mail in error please delete it and any attachments and kindly notify us by immediately sending an email to contact(a)resetdata.com.au<mailto:contact@resetdata.com.au>

1 0

Error Occurred After Multipath Configuration
by jiwooo.oh＠samsung.com 01 Jul '25

01 Jul '25

After setting up multipath on netapp, errors occur frequently when detaching a volume. Unexpected error while running command. Command: multipath -f 3600a098038314e7a4b24577351787462 Exit code: 1 Stderr: map in use' This occurs when executing the multipath -f command, and a 'map in use' error occurs. [Full error log] 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] Traceback (most recent call last): 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/virt/block_device.py", line 413, in driver_detach 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] virt_driver.detach_volume(context, connection_info, instance, mp, 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/virt/libvirt/driver.py", line 2860, in detach_volume 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] self._disconnect_volume(context, connection_info, instance, 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/virt/libvirt/driver.py", line 2099, in _disconnect_volume 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] vol_driver.disconnect_volume( 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] File "/var/lib/kolla/venv/lib/python3.10/site-packages/nova/virt/libvirt/volume/iscsi.py", line 74, in disconnect_volume 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] self.connector.disconnect_volume( 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] File "/var/lib/kolla/venv/lib/python3.10/site-packages/os_brick/utils.py", line 174, in trace_logging_wrapper 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] return f(*args, **kwargs) 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] File "/var/lib/kolla/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py", line 78, in inner 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] return f(*args, **kwargs) 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] File "/var/lib/kolla/venv/lib/python3.10/site-packages/os_brick/utils.py", line 415, in change_encrypted 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] res = func(**call_args) 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] File "/var/lib/kolla/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py", line 894, in disconnect_volume 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] return self._cleanup_connection(connection_properties, force=force, 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] File "/var/lib/kolla/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py", line 948, in _cleanup_connection 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] multipath_name = self._linuxscsi.remove_connection( 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] File "/var/lib/kolla/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py", line 382, in remove_connection 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] self.flush_multipath_device(multipath_name) 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] File "/var/lib/kolla/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py", line 450, in flush_multipath_device 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] self._execute('multipath', '-f', device_map_name, run_as_root=True, 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] File "/var/lib/kolla/venv/lib/python3.10/site-packages/os_brick/executor.py", line 55, in _execute 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] result = self.__execute(*args, **kwargs) 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] File "/var/lib/kolla/venv/lib/python3.10/site-packages/os_brick/privileged/rootwrap.py", line 172, in execute 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] return execute_root(*cmd, **kwargs) 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_privsep/priv_context.py", line 271, in _wrap 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] return self.channel.remote_call(name, args, kwargs, 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] File "/var/lib/kolla/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py", line 215, in remote_call 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] raise exc_type(*result[2]) 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266]oslo_concurrency.processutils.ProcessExecutiON-ERROR: Unexpected error while running command. Jun 4, 2025 @ 05:08:04.928 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] Jun 4, 2025 @ 05:08:04.928 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] Stderr: '9203485.810915 | 3600a098038315143433f58706d524b7a: map in use\n' Jun 4, 2025 @ 05:08:04.928 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] Stdout: '' Jun 4, 2025 @ 05:08:04.928 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] Exit code: 1 Jun 4, 2025 @ 05:08:04.928 1487897 ERROR nova.virt.block_device [instance: 402cec31-5d7e-4167-b057-09675dcf4266] Command: multipath -f 3600a098038315143433f58706d524b7a [Related config] use_multipath_for_image_xfer: true netapp_lun_space_reservation: disabled image_volume_cache_enabled: true image_volume_cache_max_size_gb: 0 image_volume_cache_max_count: 50 [Error Occurrence Case] - Create volume with image - Volume detach - Stop then start operation - vm resize Is there a solution to this situation?

1 0

[cinder][glance][telemetry][barbican] Deliverables defined in governance but not released yet
by Thierry Carrez 01 Jul '25

01 Jul '25

Hi everyone, The release team latest consistency check revealed a number of deliverables that are defined in the openstack/governance repository but do not have a deliverable file in the openstack/releases repository yet. [cinder] rbd-iscsi-client has been added Jan 2021, never released yet. Should it be removed from governance at some point? [glance] os-test-images has been added July 2024, never released yet. Is it a repository that is not meant to be "released" ? If yes, we can mark it "release-management: none" in governance to avoid flagging it. [telemetry] aetos and grian-ui have been recently added (April, 2025). Do you plan to release them as part of the Flamingo release? If yes we should add deliverable files for them in openstack/releases. [barbican] barbican-ui has been added Oct 2019, never released yet. Should it be removed from governance at some point? Thanks in advance for your answers! -- Thierry Carrez (ttx)

1 0

[heat][ops] Removal of standalone api services (heat-api and heat-cfn-api)
by Takashi Kajinami 01 Jul '25

01 Jul '25

Hello, Historically heat has provided two methods to deploy api services (native heat api and heat CFN api). 1. standalone script based on eventlet 2. wsgi application run by server framework like apache or uwsgi Now the heat team is trying to get rid of eventlet following the global effort to replace it, and the current blocker is 1 because we have no good alternative to re-implement the standaloe service. The global migration strategy is to remove standalone api services using eventlet and provide only wsgi application. So I'm now sending an email to ask for any input, in case if any operator has concern about our current plan below. - Backport deprecation of standalone api scripts to 2025.1 - Remove these scripts during 2025.2 This means that any operators should use 3rd party server framework like uwsgi and apache+mod_wsgi to run heat api from 2025.2 . I understand backporting deprecation is something we won't accept usually, but I've seen this pattern was done in the past for a few exceptions like Hyper-V support removal. If we hear no objections/concerns for a week then, we aim to merge the removal during this cycle. Thank you, Takashi -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

1 1

[dev][tc] Replacing CLA enforcement with DCO enforcement
by Jeremy Stanley 30 Jun '25

30 Jun '25

As previously discussed[*], tomorrow (2025-07-01) I'll merge a change[**] switching Gerrit to start requiring Signed-off-by in commit messages when pushing new changes or revisions to changes in OpenStack repositories (existing changes don't need to be amended in order to merge; only when revising, rebasing or backporting to other branches). Agreeing to the OpenInfra ICLA will no longer be necessary. The OpenStack Contributor Guide has already been updated to reflect this[***]. After this update merges, when attempting to push or alter a change's content you may receive an error (from git review/git push at the command line or from the Gerrit WebUI if you're trying to edit a change there) notifying you that you're missing a Signed-off-by trailer in your commit message. You can either add one with git's -s option like... git commit -s (to add when making a new commit) git commit --amend -s (when altering an existing commit) ...or by directly editing the commit message to include a line at the end with your address such as... Signed-off-by: Jeremy Stanley <fungi(a)yuggoth.org> Keep in mind that if you're pushing a series of changes, you may need to update all their commit messages, e.g. through an interactive rebase. If anyone has questions, please don't hesitate to ask! [*] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [**] https://review.opendev.org/950998 [***] https://docs.openstack.org/contributors/common/dco.html -- Jeremy Stanley

1 0

[openstack]Share about OSS which used for my Openstack
by Nguyễn Hữu Khôi 30 Jun '25

30 Jun '25

Hello, I would like to share some open-source software (OSS) that I used for my OpenStack environment. https://github.com/nguyenhuukhoi/openstack-note/blob/main/Cloud%20System%20… I hope this will be helpful for you, especially those who are getting started with OpenStack. Nguyen Huu Khoi

3 4