June 2025 - openstack-discuss - lists.openstack.org

[cinder] Unexpected Ceph RBD snapshot retention behavior when deleting image-based volumes
by Rocha Lobo, Vinicius 18 Jun '25

18 Jun '25

Hi, OpenStack Community. I'm part of the StarlingX Community and our starlingx-openstack distribution is currently delivering the 2024.1 (Caracal) release. We are facing an issue when using Cinder with the Ceph RBD backend: volumes created from Glance images leave behind protected snapshots and *.deleted volumes in the cinder-volumes pool after deletion, and these remnants are not being cleaned up automatically, even though I’ve enabled deferred deletion and garbage collection. So, I'm writing this email to explain our current issue and get feedback on this new Launchpad [1] I’m creating for Cinder. Environment: * OpenStack version: 2024.1 (Caracal) * Ceph version: Reef * Cinder configuration (via Helm override): conf: cinder: backend_defaults: rbd_flatten_volume_from_snapshot: True enable_deferred_deletion: True deferred_deletion_purge_interval: 10 ceph: image_volume_cache_enabled: False Issue description: 1) Create volume(s) from an image: * openstack volume create --image <image_id> --size 100 volume-test 2) Cinder creates a protected snapshot of the source image and then clones from it (as expected due to Ceph’s requirement for protected snapshots when cloning). 3) Delete the volume(s) via OpenStack CLI: * openstack volume delete volume-test After the deletion the corresponding RBD image and its snapshot are not cleaned up: * rbd -p cinder-volumes ls -l NAME SIZE PARENT FMT PROT LOCK <uuid>.deleted 100 GiB 2 <uuid>.deleted@<clone_snap>.clone_snap 1 GiB 2 yes... Even with the garbage collection features enabled (enable_deferred_deletion, rbd_flatten_volume_from_snapshot, and deferred_deletion_purge_interval), these protected snapshots and deleted volumes persist indefinitely, unless I clean them manually: * rbd snap unprotect <volume>.deleted@<snapshot> -p cinder-volumes * rbd snap rm <volume>.deleted@<snapshot> -p cinder-volumes * rbd rm <volume>.deleted -p cinder-volumes Here’s the logs that cinder pod shows while trying to delete the volume each 10 seconds: 2025-05-28 19:44:48.079 8 INFO cinder.volume.drivers.rbd [-] Purging trash for backend 'ceph-store' 2025-05-28 19:44:58.076 8 INFO cinder.volume.drivers.rbd [-] Purging trash for backend 'ceph-store' 2025-05-28 19:45:08.081 8 INFO cinder.volume.drivers.rbd [-] Purging trash for backend 'ceph-store' Questions: * Is this behavior expected in Cinder/CEPH integrations? * Shouldn't Cinder unprotect/delete the snapshot when the last volume, depending on how it is removed? Any insights or best practices would be greatly appreciated. Thanks in advance, Vinicius Lobo StarlingX contributor [1] https://bugs.launchpad.net/cinder/+bug/2114853

2 1

[all][tc][cyborg] Call for cyborg project maintainers
by Ghanshyam Maan 18 Jun '25

18 Jun '25

Hello Everyone, Cyborg project is lacking the active maintenance. The gate is broken, no response on the fix[1]. Two weeks ago, Sean raised the conern to ML[2] but there is no response from current core. We discussed it in TC meeting (last week[3] and today[4]) and as per foundation staff reachout, the next step is to call for new maintainers. If anyone using or interested to maintain it then this is right time to step up. Also, I would like existing maintainers (first of all, thanks for maintaining it till now) to give more insight about project status and their bandwidth which can help new maintainers to onboard. As per TC process to check project status, I have proposed to mark cyborg as inactive[5], feel free to provide your opinion here or in gerrit. If we do not hear back and mark project Inactive, there will be no cyborg project deliverables release in this cycle. [1] https://review.opendev.org/c/openstack/cyborg/+/949772 [2] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [3] https://meetings.opendev.org/meetings/tc/2025/tc.2025-06-10-17.00.log.html#… [4] https://meetings.opendev.org/meetings/tc/2025/tc.2025-06-17-17.00.log.html#… [5] https://review.opendev.org/c/openstack/governance/+/952798 -gmaan

1 0

Nova bug report #2114855
by Vicentini Fracarolli, Joao 17 Jun '25

17 Jun '25

Hello OpenStack Community, I'm part of the StarlingX Community and our starlingx-openstack distribution is currently delivering the 2024.1 (Caracal) release. We were exploring the openstack-client "server rebuild" feature [1], and we might have found a bug. So, I'm writing this email to explain our current issue and get feedback on this new Launchpad we are creating for the Nova component https://bugs.launchpad.net/nova/+bug/2114855 [2]. It happened in the following scenario: * Create a VM with some small disk flavor and a small image. For example, using a flavor of 1 GB with the Cirros image (~ 21 MB of disk size). * Try to rebuild the VM using a new image. Example: the ubuntu-jammy qcow image. When the steps above are executed with an image with a disk size greater than the flavor size, the Nova API returns an error and the VM is not rebuilt: Flavor's disk is too small for requested image. Flavor disk is 1073741824 bytes, image is 1283063808 bytes. (HTTP 400) (Request-ID: req-7db974bb-15f3-425f-b97b-98b789b7e10f) But when we use an image with a disk size smaller than the flavor disk size, but with a virtual size bigger than the flavor disk size, Nova will try to rebuild the VM and fail: the VM state will be ERROR. It looks like the Nova API is accepting such rebuild requests, but when Nova Compute checks for virtual size, it fails. Is that expected behavior? Shouldn't the Nova API reject the request with an explicit warning telling the user to resize the VM before trying to rebuild? Thank you very much [1] https://docs.openstack.org/python-openstackclient/latest/cli/command-object… [2]https://bugs.launchpad.net/nova/+bug/2114855 Bug #2114855 “VM rebuild starts even if image virtual size is gr...” : Bugs : OpenStack Compute (nova)<https://bugs.launchpad.net/nova/+bug/2114855> Description =========== Openstackclient + Nova API accept to rebuild server with virtual size greater than the specified flavor’s disk size. Steps to Reproduce ================== 1 - Create a server specifying a flavor with small disk size (example: 1 GB) and an image small enough to fit. openstack flavor create --ram 512 --disk 1 --vcpus 1 tiny openstack server create –flavor tiny –image cirros vm-test 2 - Try to rebuild the server using a larger image but that still fits. It’s v... bugs.launchpad.net

2 1

[nova] multiple pci types with same address
by Arnaud Morin 17 Jun '25

17 Jun '25

Hello nova team, Quick question regarding support for multiple types (see [1]) with the same address: {vendor_id: "10de", product_id: "233b", "address": "00000000:03:00.0", "resource_class": "CUSTOM_H200_A"} {vendor_id: "10de", product_id: "233b", "address": "00000000:04:00.0", "resource_class": "CUSTOM_H200_A"} {vendor_id: "10de", product_id: "233b", "address": "00000000:44:00.0", "resource_class": "CUSTOM_H200_A"} {vendor_id: "10de", product_id: "233b", "address": "00000000:45:00.0", "resource_class": "CUSTOM_H200_A"} {vendor_id: "10de", product_id: "233b", "address": "00000000:83:00.0", "resource_class": "CUSTOM_H200_B"} {vendor_id: "10de", product_id: "233b", "address": "00000000:84:00.0", "resource_class": "CUSTOM_H200_B"} {vendor_id: "10de", product_id: "233b", "address": "00000000:C3:00.0", "resource_class": "CUSTOM_H200_B"} {vendor_id: "10de", product_id: "233b", "address": "00000000:C4:00.0", "resource_class": "CUSTOM_H200_B"} This works fine, I was able to define multiple aliases: {name: "h200a", device_type:"type-PF", resource_class: "CUSTOM_H200_A"} {name: "h200b", device_type:"type-PF", resource_class: "CUSTOM_H200_B"} I did that to create two blocks of 4 resources (A or B). But now, I need to create a flavor to boot instances with these devices. I want to have only one flavor that can use either A or B: For now I created a flavor with: pci_passthrough:alias='h200a:4' And was forced to create a second flavor with h200b:4. Is there any way to achieve a single flavor with both: Something like this? pci_passthrough:alias='h200a:4|h200b:4' I can't figure that out for now, is it possible? [1] https://docs.openstack.org/nova/latest/admin/pci-passthrough.html#support-f…

2 1

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.2/R-15)
by Goutham Pacha Ravi 17 Jun '25

17 Jun '25

Hello Stackers, We're fifteen weeks away from the coordinated 2025.2 "Flamingo" release of OpenStack [1]. Milestone-2 is in a couple of weeks. This milestone is observed by several project teams for bug targeting. From the point of view of OpenStack's governance, this milestone serves as a point where we vet project deliverables to be included in the coordinated release. Releases are not tagged for deliverables from project teams that are deemed "inactive" [2]. In OpenStack, a project team is considered "inactive" when it no longer has regular contributor engagement or ongoing development. This often reflects that the problem the project once addressed is now being solved more effectively by another solution—either within or outside OpenStack. Inactivity isn't a failure, but rather a natural sign of how technology needs evolve over time. Any community member can propose a change to mark a project team inactive prior to Milestone-2. Doing so will prevent shipping broken or unmaintained code. In the past week, the OpenStack Technical Committee did not make any new governance changes. The team's focus has been on the implementation of the DCO by our deadline of 2025-07-01. I'd like to gently remind you that beyond July 1st, changes without a "Signed-off-by" line in the commit message cannot be uploaded to Gerrit. If you have a project team meeting before this date, please let folks know about this change. === Weekly Meeting === The last weekly meeting of the TC was held over IRC on OFTC's #openstack-tc channel on Tuesday, 2025-06-10 [3]. The main focus of the meeting was tracking all the changes necessary for enforcing the Developer Certificate of Origin in lieu of the ICLA beginning on 2025-07-01 [4]. We noted changes to various infrastructure pieces, and while we're hoping for a smooth transition, we do anticipate some hiccups. We are hoping for the community's patience and cooperation as we iron out the kinks. Further, concerns were raised about the Cyborg project's apparent inactivity. The last change merged into Cyborg was over two months ago, and a critical CI fix remains open, lacking review attention. Efforts to engage the team via the mailing list have not yielded any response, and there is no recent activity on IRC or Gerrit. While it's known that the Cyborg team has historically preferred WeChat/Weixin for communication, relying on unofficial channels or Foundation staff for outreach isn’t sustainable. The TC agreed that it’s important to determine whether the project is defunct or simply in need of new leadership, and to encourage maintainers to monitor and respond through community channels. Additionally, a self-report from the Vitrage team indicated that Tempest tests are currently failing due to a lack of contributor time this cycle. With the M-2 milestone deadline approaching on July 3rd, we emphasized the importance of project teams providing accurate and timely updates to the release team. The next meeting of the OpenStack TC is today, 2025-06-17. It'll be held over IRC in OFTC's #openstack-tc channel at 1700 UTC. Please find the agenda on the meeting's wiki page [5]. I hope you'll be able to join us. === Governance Proposals === ==== Open for review ==== - Require declaration of affiliation from TC Candidates | https://review.opendev.org/c/openstack/governance/+/949432 === Upcoming Events === - 2025-07-03: OpenStack's 15th Birthday, Colombia User Group: https://www.meetup.com/colombia-openinfra-user-group/events/308383244 - 2025-06-28: OpenInfra+Cloud Native Day, Vietnam: https://www.vietopeninfra.org/void2025 - 2025-07-19: OpenInfra Days, Indonesia: https://2025.openinfra.id/ === How to Contact the TC === You can reach the TC in several ways: - Email: Send an email with the tag [tc] on this mailing list. - IRC: Ping us using the tc-members keyword on the #openstack-tc IRC channel on OFTC. - Meeting: Join us at our weekly meeting. The Technical Committee meets every week on Tuesdays at 1700 UTC [4]. Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.2 "Flamingo" Release Schedule: https://releases.openstack.org/flamingo/schedule.html [2] Inactive projects: https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [3] TC Meeting IRC Log 2025-06-10: https://meetings.opendev.org/meetings/tc/2025/tc.2025-06-10-17.00.html [4] Replace ICLA with DCO: https://governance.openstack.org/tc/resolutions/20250520-replace-the-cla-wi… [5] TC Meeting Agenda, 2025-06-17: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[all][dev][ops][tc] Bridging the gap between community and contributing orgs
by Jeremy Stanley 16 Jun '25

16 Jun '25

The tl;dr on this is that I'm opening a broad community discussion on how we can, collectively through improved communication, better bring together our established contributors with casual and prospective contributors who struggle to find successful patterns of contribution, so that we might all support each other in order to benefit OpenStack as a whole. Read on for details... OpenStack's TC has requested that foundation staff, when in conversation with representatives of member organizations, encourage participation in the project and collect feedback on any related challenges those organizations encounter in their attempts to do so. In order to better understand the feedback and brainstorm achievable recommendations for the broader community[*], Community Managers on the foundation staff started to work with a small focus group of established contributors with a solid understanding of what successful patterns of contribution look like. It also came to light that established contributors experience many of the same sorts of challenges, even more so if they don't have the luxury of focusing full-time upstream, and so success often comes down to knowing how to effectively navigate those challenges. The community's goal with this exercise is to improve everyone's experience, and your help is needed to do that! The most common themes reported by those struggling to contribute are not related to tooling or workflow confusion, but instead seem to come down to basic communication challenges. Ideas so far to address these gaps include: * Review bandwidth - Ensure contributors and companies understand the importance and benefits of code review - Incentivize and credit meaningful "+1" reviews more - Understand that the volume of changes and limited reviewer bandwidth often means proposals aren't reviewed quickly (or at all), and their proponents may need some additional tenacity and engagement with the team to get their work noticed - Utilize "review dashboards" as a way to find stale changes which have fallen through the cracks * Review strategy and etiquette - Be clear about the meaning of votes on changes, especially negative ones, and set explicit timelines for things like "procedural -2" or WIP blocks when their approval needs to be delayed - Focus on comments and requests that affect whether or not the change can be merged after the fixes are submitted, rather than requesting trivial adjustments to an unsuitable change which has more fundamental problems - Reciprocate with reviews of changes from new reviewers you see leaving insubstantial reviews, demonstrating to them what deeper and more meaningful reviewing looks like * Mentoring newcomers - Established contributors with sufficient bandwidth can help mentoring newcomers, potential core reviewers and new leaders - Mentoring can happen as part of internship programs as well as by helping newcomers determined to become active contributors; both ways should be embraced and utilized What challenges are you facing? And, how would you improve them? Two weeks from now, we'll have a forum session at the OpenInfra Summit Asia in Suwon[**] where our community can refine these possible approaches and integrate others, as well as bring attention to them and promote their application. I've also proposed a similar forum session for OpenInfra Days North America in Indianapolis a month after that, where we can hopefully continue this conversation. The goal for this multi-stage effort is to improve the contribution experience for established participants, casual contributors and newcomers alike. While it's easy to jump to blaming in these situations, it will be far more productive if we can focus on the challenges and experiences that we would each like to remove or improve for ourselves and others in our community. As we find ways together to improve our efficiency, we can reduce the load on all contributors and lower the barriers for people to join and participate. [*] https://etherpad.opendev.org/p/r.2205024e55689bccb82c20c960853cb5 [**] https://2024.openinfraasia.org/a/schedule#view=calendar&title=Bridging%20th… -- Jeremy Stanley

2 10

[neutron] bug deputy report for week of 2025-06-09
by Bence Romsics 16 Jun '25

16 Jun '25

Hi Team, We only had a handful of new bug reports last week: High: * https://bugs.launchpad.net/neutron/+bug/2113861 [FT] Test ``test_initialize_network_segment_range_support_parallel_execution`` randomly failing already unblocked on master, backport in progress: https://review.opendev.org/q/topic:bug/2113861 Medium: * https://bugs.launchpad.net/neutron/+bug/2114176 [L3] L3 agent ``ProcessMonitor`` is not correctly registering the new processes fix merged: https://review.opendev.org/c/openstack/neutron/+/952450 Low: * https://bugs.launchpad.net/neutron/+bug/2114190 [whitebox-neutron-tempest-plugin] ``test_only_dropped_traffic_logged`` test failing with the WSGI Neutron API migration fix merged: https://review.opendev.org/c/x/whitebox-neutron-tempest-plugin/+/951264 [BTW: Maybe this project would be in a better place under the openstack namespace?] RFE: * https://bugs.launchpad.net/neutron/+bug/2114451 [RFE] ports should contain a reference to the segment_id they are bound to Related to https://bugs.launchpad.net/neutron/+bug/2105855. Probably a good idea to discuss on the drivers meeting. Cheers, Bence

1 0

[nova] Is It Possible to Attach a Cinder Volume as device='lun' ?
by Janaka Wickramasinghe 16 Jun '25

16 Jun '25

Hi, I'm running an OpenStack setup with Fibre Channel (FC) storage using Cinder volumes. I'm currently trying to configure a Windows Failover Cluster in guest VMs and have encountered an issue. Windows is querying the SCSI device for the Device Identification VPD (Page 0x83). The LUN is attached as a SCSI device using a virtio-scsi controller. In the domain XML (via libvirt), I see that the disk is defined as <disk type='block' device='disk'> also has a <serial>cinder-vol-uuid</serial> When Page 0x83 is queried, it returns the serial value (i.e., the cinder-vol-uuid), which Windows fails to recognize or accept. However, if I modify the XML to use: <disk type='block' device='lun'> and remove the <serial> tag, the guest is able to retrieve the expected information (I had to stop the compute service and restart the libvirt domain). Unfortunately, any changes made directly to the domain XML do not persist. Is there a supported way to specify device='lun' instead of device='disk' for volumes attached ? Any guidance would be greatly appreciated. Thank you With Best regards, Janaka

3 3

Re: [Octavia] - Loadbalancer appear offline but are running
by Wells, Aaron 14 Jun '25

14 Jun '25

Hi Karl, We ran into and fixed this issue on our installation by making sure that the UDP heartbeat packets on port 5555 were reaching the controllers. Thanks. Aaron -- Aaron Wells wellsaar(a)iu.edu DevOps Engineer Jetstream 2 https://kb.iu.edu/d/bfde

2 1

[neutron] Neutron BGP integration upstream sync
by Jakub Libosvar 13 Jun '25

13 Jun '25

Hi all, as per the previous suggestions I created the doodle [1] to pick the right time when we could meet. Finding a good timeslot is challenging as I noticed interest from all around the globe (US, EU, Australia). Please vote for which slot fits you best and if anyone has better suggestions, please let me know and let’s discuss. Thanks! Jakub [1] https://doodle.com/group-poll/participate/aK9Nkknd/vote

1 1