- openstack-discuss - lists.openstack.org

[barbican] Cancel Weekly Meeting
by Mauricio Harley 16 Apr '25

16 Apr '25

Hello, everyone. Barbican current's PTL here. We need to cancel the weekly meeting that would happen on Monday the 21st. This is necessary since this day will be a public holiday in many countries in Europe. Looking forward to meeting you all on the Monday following: April the 28th. Best regards, Mauricio

1 0

[trove-dashboard][designate-dashboard] Stuck translation proposal jobs
by dcu995＠gmail.com 16 Apr '25

16 Apr '25

Hello, team. Wanted to bring your attention for stuck translation proposal jobs for both trove-dashboard and designate-dashboard (https://zuul.opendev.org/t/openstack/build/e6849eafd29c4ee78c57fff6a8768ac6 ,https://zuul.opendev.org/t/openstack/build/3489b918e89642a4a607665edd6a5d41) As I have found out this behaviour is caused by jobs running on Ubuntu Focal with Python 3.8. Is there anything can be done to fix this issue? Thanks.

1 0

[kolla-ansible][dalmatian]Apply a patch to fix a bug
by Franck VEDEL (UGA) 16 Apr '25

16 Apr '25

Hello, and first sorry for coming back with my problem. After 3 days of trying to resolve it (applying a patch to fix a known bug in Horizon), it's still not working. No changes have been made to the problematic file (workflows.py). First question: This link discusses the topic (« Patching Customization") https://docs.openstack.org/kolla/latest/admin/image-building.html But not in the one for version 2024.2: https://docs.openstack.org/kolla/2024.2/admin/image-building.html So maybe the method doesn't apply to Dalmatian. Second question: what is wrong with what I’m doing: https://review.opendev.org/changes/openstack%2Fhorizon~938238/revisions/1/p… patch Archive ZIP · 409 octets The patch is a zip file We can download it: c2cce40.diff.zip I create the directory : /home/user/PATCH/horizon My /etc/kolla/kolla-build.conf file is these 2 lines [DEFAULT] patches_path = /home/user/PATCH In this directory, we can list files: (I tried with the zip and the unzip file, the diff file) (venv) user1@pc1:~$ ll /home/user/PATCH/horizon/ total 24 drwxrwxr-x 2 user1 user1 4096 avril 14 09:14 ./ drwxrwxr-x 3 user1 user1 4096 avril 11 08:24 ../ -rw-rw-r-- 1 user1 user1 1093 déc. 25 03:01 c2cce40.diff -rw-rw-r-- 1 user1 user1 716 avril 14 09:12 c2cce40.diff.zip -rw-rw-r-- 1 user1 user1 716 avril 14 09:12 c2cce40.zip -rw-rw-r-- 1 user1 user1 13 avril 14 09:14 series The series file (the try with the zip file): (venv) user1@pc1:~$ more /home/user1/PATCH/horizon/series c2cce40.zip I also tried Without the zip extension I modify the file: /home/user/venv/share/kolla-ansible/ansible/roles/horizon/defaults/main.yml #################### # Docker #################### #horizon_image: "{{ docker_registry ~ '/' if docker_registry else '' }}{{ docker_namespace }}/{{ docker_image_name_prefix }}horizon" #horizon_tag: "{{ openstack_tag }}" # #horizon_image_full: "{{ horizon_image }}:{{ horizon_tag }}" horizon_image_full: "kolla/horizon:19.4.0 » The next command create a new docker image (horizon:19.4.0) Kolla-build -b ubuntu horizon No problem… I have a new image named horizon:19.4.0. I copy the image on my second computer (2 nodes in my test cluster) Next: kolla-ansible reconfigure -I multinode Everything works , the new image is used to build horizon containers. But no modification in the file (venv) user1@pc1:~$ docker exec -it horizon bash (horizon)[root@pc1 /]# more ./var/lib/kolla/venv/lib/python3.12/site-packages/openstack_dashboard/dashboards/project/networks/ports/workflows.py class UpdatePortSecurityGroupAction(sg_base.BaseSecurityGroupsAction): def _get_initial_security_groups(self, context): port_id = context.get('port_id', '') port = api.neutron.port_get(self.request, port_id) return port.security_groups No modification on the container, in the workflows.py file. What is wrong ? There's something I'm not doing right, but what? Do you have any ideas? I'm definitely not managing it. Thank you, really, if you have any ideas. Franck

3 3

[neutron] ML2/OVN vs ML2/OVS
by Nguyễn Hữu Khôi 16 Apr '25

16 Apr '25

Hello. I am going to deploy an openstack network for a DC. I would like to have some advice on choosing between them. Could you show me some pros and cons between them? About OVN, I don't see many articles about it. And How can I check traffic routes via which GW, I can check this via ha_state with OVS Thank you very much. Nguyen Huu Khoi

4 8

[heat][ptg][ops] Flamingo PTG summary
by Takashi Kajinami 16 Apr '25

16 Apr '25

Hello, Last week during the PTG the heat team gathered to discuss a few topics. Thank you all who attended the session and the valuable discussions. The details can be found in https://etherpad.opendev.org/p/apr2025-ptg-heat but I'll leave some summary. There are a few plans about which we want to ask for any feedback from operators so I'm adding [ops] tag here. 1. Eventlet removal During the session we reviewed the current global plan to remove/replace eventlet and discussed the plan to follow it in heat. Our current idea is to remove/deprecate unmaintained code first so that we can shrink the scope. At this moment we aim to deprecate and remove the following items. - Standalone heat-api and heat-cfn-api - These are deprecated due to deprecation of WSGI server implementation from oslo.service, which depends on eventlet. Users can run api server by external server mechanism such as mod_wsgi or uwsgi. - heat-all service - This is deprecated due to deprecation of WSGI server implementation from oslo.service, which depends on eventlet. We expect this is not really used in production - legacy engine - This was deprecated in favor of convergence. Note that removing this means we no longer support adopt/abandon/check feature. Question to operators: Does anyone have concern about deprecating/removing these items ? Please be aware that the current heat team is quite small and we need actual help to replace eventlet for these features. 2. CFN API We discussed the current status of heat-cfntool which heavily depends on unmaintained boto library. However we also agreed that replacing boto by boto3 requires some effort due to the fact that boto3 does not allow overriding endpoint quite easily. While we aim to deprecate CFN api, we agreed that we still have to asses the impact because CFN API is used for signaling by default. Question to operators: Is anyone using CFN API in any other use case than signaling ? For the first topic, if we don't hear any objections, we'll follow the plan to move the removal work forward. Thank you, Takashi -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

1 0

[ironic][ptg] 2025.2 (Flamingo) PTG Summary
by Julia Kreger 16 Apr '25

16 Apr '25

Greetings everyone! Last week Ironic had a wildly successful PTG! We averaged around fifteen attendees most days, we drew upwards of 22 attendees for most of Wednesday while we discussed a number of related topics around networking. Overall, networking aspects seem to show the most current interest. This is the result in a change in market ecosystem where vendors are less focused on SDN integrations, and also deliberate limitations inside of ironic in order to limit scope creep and still attempt to meet the majority of infrastructure operator requirements when the original networking multi-tenancy effort was executed back in 2015-2016. Other major topics were Eventlet, some operator feedback on quarks of features, which highlighted some possible bugs and areas for improvement. We also got into some areas which have been a bit contentious in the past, but we reached some reasonable compromises which allowed us to find better paths forward. Monday: We largely discussed where we were at and where we were going. To highlight, ironic-lib has now been retired. Redfish based Graphical Console support merged and we identified some more work was likely needed. Bootable Container support *and* support for artifacts from an OCI container registry support was also viewed as completed during the cycle which ultimately improves options to infrastructure operators who are operating in mixed environments or with mixed requirements outside of the classical "everything is a VM in OpenStack" context. We've improved the linting across the majority of project repositories. Out of band inspection rules likely need more work around ensuring data structures are what we expect, but otherwise we are re-affirming our deprecation of ironic-inspector as a standalone project. Work to support Kea as a DHCP backend has paused for now. The plan exists, but realistically the contributor working in that area has a different focus at the moment which is okay! Container based IPA steps also didn't make it into the Epoxy cycle release, but are almost done already in Flamingo. Schema validation and OpenAPI work is also still underway and we broadly expect this to merge into Flamingo. In-band disk encryption as part of the ``direct`` deploy interface also didn't make any forward progress due to shifting contributor priorities. As a note, the bootable container work did extend this as a possible option, however it remains unclear if that will solve the overall need for the contributor who did propose adding support for encrypted volumes to the ``direct`` interface. Efforts to find an alternative to TinyCore usage in CI also stalled out. Turns out is less of a trivial problem to make a super-low memory ramdisk, and other fixes to the build jobs improved general reliability in the meantime so it is less pressing. We also didn't make any progress on Project Mercury as it was likely framed to try and keep too many options open for operators while we also lost contributor velocity due to some CVE work this past cycle. Having discussed what we achieved, this allowed us to focus on key aspects contributors know they needed to focus on this next cycle. Networking was the biggest topic in this area. Promptly followed by eventlet removal. There was also a consensus that we needed to ensure we're working together a bit better, but also don't explicitly block any one aspect, as there is power in user/operator choice as well. Some possible interest was also expressed around extending network device firmware upgrades, further delineating/improving metrics, possibly supporting more of a "push" firmware update model which more vendors are adopting. We promptly shifted to CI, and discussed challenges and path forward. Generally, there is some interest in trying to make some of our job executions a bit more selective, and also to dial back our reliance upon integration scenario jobs. Overall, it is a broad area of work which will require some further discussion to try and frame an ideal future state. In this we also discussed Centos Stream 10, python versions, and ultimately possible paths forward to at least highlighting other options which may enable maintenance activities to ensure these cases work. Afterwards, we shifted to CI knowledge sharing later in the day outside of the PTG schedule, to help spread overall context among newer members of the team. Tuesday! We started Tuesday with the topic of supporting a use case desired by some operators wishing to "allocate" baremetal nodes into a state which, to Ironic signifies the node has been deployed. The use case behind that is a bit odder, more for research and academic cases where other tools may be desired to meet very specific requirements. The discussion yielded an operator in the scientific space which could benefit, so some ideas and a possible path was identified. Then we got into the world of eventlet and a mini-retrospective of the challenges and identification of future steps. What we thought would be simple, turned out to be the hardest problem. Ironic Python Agent, originally identified as a good candidate for an early eventlet migration, has turned out to be extremely difficult due to it's need to spawn it's own WSGI server late in the process. Upon further reflection; Ironic itself has this issue as well, because we actually have two places a WSGI server might be spun up: ironic-api (obviously) and the conductor when using json-rpc. It was clear that solving that technical problem was the first step in our migration. We found some examples of gunicorn used in this way; that may be what we look to for an initial prototype. We then spoke about node / hardware monitoring patterns that operators are using and interested in. Redfish based hardware supports sending a notification to a web service on a threshold violation so we discussed adding a listener into Ironic that could receive these and integrate this into the current hardware event notifications. We agreed that a spec would be necessary and we would craft one and proceed from there. Further improvements around making it easier for operators to separate monitoring/events of the services from the hardware itself was discussed but oslo_messaging does not allow for this type of split routing today. We spoke about looking further into ironic-prometheus-exporter to see how it could be configured to potentially support this separation. To wrap up Tuesday, we drifted into discussion of the serial console support which Ironic has today. In essence, there is quite a bit of room for possible improvement there and some operators who are not using Nova wouldn't mind this area to see attention moving forward. The broad idea is we might explore creation of a new interface which enables SSH connections to be proxied through to the IPMI Serial-over-LAN capabilities in a BMC. This distinctly would require IPMI, since there really is not a great answer to serial console/interface access with Redfish. Ultimately this may also mean we extend redfish to support an IPMI interface as well, which seems weird, but discussion was in broad agreement that this space has unique challenges where this makes sense. Some initial ideas were entered into an spec document for discussion and refinement as some of the interested parties are also on the perimeter of the Ironic project. Wednesday! Wednesday was our deep dive into networking! Our very first topic was focused on improving the capabilities around the intentional limitations which were encoded in Ironic's early multi-tenant networking work. The early work was mainly focused on ensuring tenant level isolation on to separate L2 domains, such as VLANs. In large part, this boils down to the mapping, scheduling, and ultimately the dynamic assembly of a port grouping (bonded interfaces). We came into this session with a specification document from a discussion which occurred a few weeks prior and there was general consensus in this option and discussion highlighted a few other intertwined aspects which need to be accounted for. We then dove into ACL support for Networking Generic Switch. Overall, consensus resulted. This discussion also highlighted a number of aspects we also likely need to keep in mind, and potentially may start best be described as overall needs for additional documentation. There are also some further discussions which may need to be revisited networking wise. Ultimately, this is very much an "integrated" use case which was only appealing to about half of the attendees, which is typical given the variety of use cases Ironic solves and is able to support. Then we reached the topic of Standalone Switch management. This quickly became a retrospective into why we didn't make progress with Mercury, and then drove into what are the real minimal requirements which would need to be taken into account. Some of the discussion revolved around questions which were semi-answered in the mercury plan itself, to try and provide enough overlap to generally be usable in more than one use model/case while also enabling or preventing fragmentation. This discussion also crossed over into support of DPUs and how to support them, in large part because the overall model is very different. Some discussion also sort of shifted into maybe merging and tools together. Ultimately this topic requires much more discussion, and to make solid progress on networking aspects we need to enable ourselves to move together in two directions at once, while not blocking each other. With this in mind, we're forming up into two sub-teams which will focus on each area with their own advocates/champions. The goal being to report back to the community each week in a quick update style of engagement. Thursday! Thursday marked our final day of PTG for Ironic. We started by wrapping up some of the networking discussions, in large part the focus on how to make progress. We used this time to identify our high level organizational plans, determine broad interest levels and ultimately outreach. One key aspect which was highlighted, is the initial primary focus for some will likely be making progress on eventlet before shifting gears into networking. We then shifted gears to discussing interest and requirements of mapping storage volumes to overall hosts via DPU devices, which somewhat crossed over into the networking topics. The broad idea was presented with a potential need to frame compound drivers around facilitating broadly different configuration actions. For example, directly via SSH to invoke commands, or for example update a CRD in a distinct OpenShift cluster. Overall, the discussion yielded that maybe invoking a CRD update was not worthwhile given the flux those areas can experience along with competing requirements. Ultimately this is an area in early discussion and would require investment in time and hardware to move forward, but there were really no objections to doing so if we could model and extend what already exists in a useful and applicable way. We then drifted back into networking with our next topic, bridging distinctly different types of fabric together. This was raised much more as a question to see if there was an interest/need and/or requirement. In essence, this is similar to the original l2gw project. A huge highlight of embracing or extending into a more ideal model is the reality that if this existed, it might not be necessary for VXLAN to be considered on the physical side. Discussions then shifted to improving servicing interactions. Servicing functionality was added to Ironic during one of the recent past development cycles to enable firmware upgrades to take place on deployed nodes. In discussion, it was highlighted that in the current model this could take five or more reboots to facilitate the deployment as the current mechanism is largely modeled on the use of agent heartbeats. The discussion shifted to how we could avoid this. And the obvious answer was "add a periodic". Then concerns over adding more periodic queries and jobs shifted the discussion into how we can address that challenge. Ultimately, this may end up with us in a better place for tasks which need to follow-up on state changes in hardware and revisit the state before proceeding with the next step to execute. As the final topic, we dove into deploy steps. This really boiled down to "how do I know what will occur" when I ask Ironic to do something along with "how do I know what occurred". Consensus reached that largely the key aspect to highlight back to the user is "what was done" or "what do we expect to be done" in terms of steps. An idea was raised to record this into node history, which is a historical record list of actions/errors to a node which has existed in Ironic for a number of cycles now, but is most useful if you're trying to figure out prior events without going to logs. Consensus was that doing it this way would be a "quick win". At some point, Ironic contributors also sat down and reviewed jammed outside of the PTG schedule to review/discuss and merge existing items sitting awaiting reviews. Overall, the week was extremely productive! Thanks to everyone who took part. Additional thanks to everyone who collaborated on this summary, and everyone who also helped push our project priorities published for this work cycle[3]. Please remember if you have action items to follow-up. And onward to Flamingo! - The Ironic Team [0]: https://etherpad.opendev.org/p/ironic-ptg-april-2025 [1]: https://review.opendev.org/c/openstack/ironic-specs/+/945642 [2]: https://review.opendev.org/c/openstack/ironic-specs/+/946723 [3]: https://specs.openstack.org/openstack/ironic-specs/priorities/2025-2-workit…

1 0

[glance] Flamingo PTG summary
by Cyril Roelandt 15 Apr '25

15 Apr '25

Hi, We have just had our virtual PTG. The full list of topics, along with notes, can be found on our Etherpad[1]. Here is a summary of the topics that will be targeted during this cycle. If you would like to view one of the recordings, please reach out to us so we can figure out a way to share it. # glance-tempest-plugin work There has not been a lot of activity on the Glance Tempest Plugin for a while. We decided to put more effort into it this cycle. The first step is to decide what stale patches to abandon or rebase. We will also try to add tests for newer APIs added in the past cycles (location, quota, cache). # Eventlet removal While Glance can already be deployed in production without eventlet, this library is still required for half of our functional tests, as well as for non-core features (namely the scrubber). We will be focusing on migrating our functional tests away from eventlet during this cycle. # Image encryption Not much progress has been made on this topic in the past cycle. The original authors of this feature are willing to spend time working on the required patches during the Flamingo cycle; the Glance core dev will make it a priority to quickly review the required patches so that we can finally merge the feature. # Glanceclient chunked transfer By default, the Glance client transfers data to the Glance API in chunks. This may force the Cinder backend to resize the volume multiple times, slowing down the upload. Providing the total size of the image will allow us to fix this issue. # Cinder common configuration options Operators often have parameters set to the exact same values for multiple Cinder backends. We plan on making it possible to configure "common configuration options", thus allowing operators not to repeat configuration snippets. # Freeze glanceclient development As we approach feature parity between the Glance client and the OpenStack unified client, we believe it will be best to freeze development of the Glance client. In the future, we will only fix bugs and security issues. During this cycle, we will work on actually reaching feature parity with the unified client. # new location API (Cinder & Nova) Glance has introduced two new Location APIs in the Dalmatian cycle. We can use these APIs to address OSSN-0090 and OSSN-0065. Patches for Nova and Cinder must still be merged, hopefully during the Flamingo cycle. # freeze glance client development Cinder and Nova will need to use the OpenStack SDK instead of the glanceclient. There is no need to complete this work during this cycle, but we should at least have a good idea of what APIs are currently being used, so that we can have a plan for the next cycles. # cache-cleaner/pruner We discussed possible minor improvements to both the cache-pruner and the cache-cleaner. This work is not going to be a priority for this cycle. # Deprecate the filesystem_store_datadirs configuration option This option has not been needed since the introduction of multistore support. Happy Hacking! Cyril Roelandt [1] https://etherpad.opendev.org/p/apr2025-ptg-glance

1 0

[horizon] Launch instance: some port names not displayed
by Eugen Block 15 Apr '25

15 Apr '25

Hi *, I've been searching for an existing bug report but haven't found one yet. OpenStack Zed (Antelope 2023.1 is also affected) openstack-dashboard versions 23.0.0 and also 23.2.0 Both environments have been upgraded from Victoria relatively quick, that's why we haven't noticed it until now, but apparently there have been major workflow changes between Victoria and Wallaby how network ports are displayed in Horizon (Launch Instance). When we open the "Launch Instance" dialog in Horizon, and we want to attach an existing port (created via CLI) to the instance, only some of the ports in the list are displayed with a name, many don't have one, although all ports were created with a name via CLI (it would have failed without providing a name). A colleague of mine helped debugging and he suspects a JavaScript error, I'm translating his statement: > Some sort of cache is built, coming from an array which is supposed > to store all relevant ports. Apparently, not all ports have been > stored yet in that array in time, leading to this inconsistency. If > there's a missing "async" or "await" somewhere, it would explain > this behavior. He then points to these functions: getPortsObj, isPortsObjGenerated which are called by getNameOrId. I found those calls here: /usr/lib/python3/dist-packages/openstack_dashboard/dashboards/project/static/dashboard/project/workflow/launch-instance/networkports/ports.controller.js and here /usr/lib/python3/dist-packages/openstack_dashboard/dashboards/project/static/dashboard/project/workflow/launch-instance/networkports/ports.spec.js Is anyone aware of an existing bug report for this? I can create one, if necessary. Thanks! Eugen

2 2

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.2/R-24)
by Goutham Pacha Ravi 15 Apr '25

15 Apr '25

Hello Stackers, Welcome back from a week of design confabulations. In the past couple of weeks, we moved quickly from the coordinated release of OpenStack 2025.1 "Epoxy" to the virtual Project Teams Gathering to plan the next release, OpenStack 2025.2 "Flamingo." We've got 24 weeks to go [1] to implement these plans. Now is a good time for project teams to add any project-specific deadlines to the release schedule. As PTG summaries slowly trickle in, I wanted to send out a brief note about the TC's weekly meeting that happened prior to the PTG, and our plans for the next one. === Weekly Meeting === The last weekly meeting of the OpenStack Technical Committee was held simultaneously on Zoom [2] and IRC [3] on 2025-04-01. The committee discussed security concerns with UI projects, mainly Skyline. Our attempts to independently rebuild the project failed partially because JavaScript libraries included had changed, and there were no build trackers. We didn't get an opportunity to continue this discussion at the PTG. For now, we'll try to keep the conversation on the mailing list and simultaneously explore solutions with the Skyline project team. The rest of the meeting was used to discuss CI updates, updates to the TC's tracker, and scheduling conversations for the vPTG. I'll share the minutes of the TC's PTG proceedings with you soon. In the meanwhile, please feel free to peruse the etherpad [4] or the playlist with recordings of the proceedings [5]. The next meeting of the OpenStack TC is today, 2025-04-15 at 1700 UTC. Please note the change in timing — it is an hour earlier than our past meetings used to be. This meeting will be held on IRC (OFTC's #openstack-tc channel). Please find the agenda on the meeting's wiki page [6]. I hope you'll be able to join us there. === Governance Proposals === ==== Merged ==== - Fix nova PTL IRC nick | https://review.opendev.org/c/openstack/governance/+/946175 ==== Open for Review ==== - [resolution] Extend scope of VMT to cover all project teams | https://review.opendev.org/c/openstack/governance/+/944817 - Fix Dan Smith's email | https://review.opendev.org/c/openstack/governance/+/946960 - Add aetos to telemetry project | https://review.opendev.org/c/openstack/governance/+/946744 === Upcoming Events === - 2025-04-29: StackConf https://stackconf.eu/ - 2025-04-29: SCS Hackathon / OpenStack Birthday: https://events.sovereigncloudstack.org/hackathon5-2025/ - 2025-05-13: OpenInfra Board Meeting: https://board.openinfra.org/ - 2025-05-22: OpenInfra UG Meetup, Sweden: https://www.meetup.com/openinfra-user-group-sweden/events/306139678/ Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.2 "Flamingo" Release Schedule: https://releases.openstack.org/flamingo/schedule.html [2] TC Meeting Video Recording, 2025-04-01: https://youtu.be/HcWJY2DeLYc [3] TC Meeting IRC Log 2025-04-01: https://meetings.opendev.org/meetings/tc/2025/tc.2025-04-01-17.00.html [4] OpenStack TC's 2025.2 PTG Etherpad: https://etherpad.opendev.org/p/r.51cb4e74dd9980932d32e9a9f4e8a0ce [5] OpenStack TC's 2025.2 PTG recordings: https://www.youtube.com/playlist?list=PLhwOhbQKWT7XYzqrx21j1uizxNxnbDf5Q [6] TC Meeting Agenda, 2025-04-15: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[I18n] Broken ko_KR translation for OpenStack-Ansible
by Dmitriy Rabotyagov 15 Apr '25

15 Apr '25

Hey folks, Today it was brought to my attention that OpenStack-Ansible does not get any periodic updates for translations for around 2 years now. Once I've checked the result of periodic job, it seems that a ko_KR translation makes Zuul job to fail: https://zuul.opendev.org/t/openstack/build/fc90cb4be70e45e8bec07e981eaff01e Specifically, following translations are containing the error (/n in translated version, which are not present in original message): * resource-id:3db2c4ee55f9791e84ff3137e05be661 * resource-id:9f3ab9a8642b37abfcdc47804c05a49f * resource-id:8cd037448cc79d28322aeb8ec25dac58 * resource-id:182414aa5a44b5a113335bc17346b853 It would be much appreciated if somebody with access to ko_KR could fix that as soon as possible. Or, alternatively, grant me permissions to translations of the project, as to PTL. Thanks in advance!

2 2

[Magnum] Issues with Magnum (K8s 1.21.11 Clusters)
by Oliver Weinmann 14 Apr '25

14 Apr '25

Hi all, We’re running a Kolla-Ansible 2023.2 deployment with Magnum. While we primarily use CAPI, we also have some older Kubernetes 1.21.11 clusters. In these clusters, we’re encountering two specific issues: Random Crashes of openstack-cloud-controller-manager Pod In one of the older clusters, the openstack-cloud-controller-manager pod is crashing intermittently with the following error: F0410 12:02:45.673777 1 main.go:84] Cloud provider could not be initialized: could not init cloud provider "openstack": Post "http://openstack.xxxxxxx:5000/v3/auth/tokens": dial tcp 172.xxx.xxx.xxx:5000: i/o timeout However, when I manually test connectivity to the Keystone API from the master node, it appears to work fine: while true; do status=$(curl -s -o /dev/null -w "%{http_code}" http://172.xxx.xxx.xxx:5000/v3 2>/dev/null) || status="failed" echo "$(date): Keystone API status: $status" sleep 10 done Sample output: Thu Apr 10 19:19:50 UTC 2025: Keystone API status: 200 Thu Apr 10 19:20:00 UTC 2025: Keystone API status: 200 Thu Apr 10 19:20:10 UTC 2025: Keystone API status: 200 ... Despite the API being reachable, the pod continues to crash sporadically with the timeout error. "Too Many Open Files" Error in Pods Users running pods in these clusters are reporting frequent "Too many open files" errors. Since Kubernetes 1.21.11 relies on CoreOS, I’m looking for a way to permanently increase the maximum number of open file descriptors on CoreOS. Any guidance or suggestions on how to address these issues would be greatly appreciated! Best regards, Oliver

2 2

[openstack-ansible][ptg] Flamingo PTG results
by Dmitriy Rabotyagov 14 Apr '25

14 Apr '25

Hey folks, Last week along with other teams we had a PTG session where a development focus for the project was discussed. I want to thank everyone who attended the session and participated in discussions. Despite the small amount of time scheduled for the session, it was very productive and you might find the summary below. Finalizing Epoxy release As OpenStack-Ansible does have a trailing release model, we still have some time left for releasing the Epoxy, with the deadline being set at June 6, 2025. With that a few topics were highlighted to be worked on for the release to be finished: * Finalize migration for Ansible to use FQCN. While this is not obligatory as of today nor a blocker for further development - we decided to adopt our roles to the new standard introduced with ansible-core 2.10. And this topic is pretty much merged at the moment: https://review.opendev.org/q/topic:%22osa/fqcn%22 * Deprecate openstack-ansible-tests repo to avoid releasing it for Epoxy. At the moment, the only useful data in the repo is the sync tool (proposal bot). We agreed to move the content/functionality we need to the integrated repository. * Finalize migration of services to HTTPD role. At the moment, only Keystone and Horizon are left for the migration (the toughest ones). One of the challenges for the migration was highlighted the need to install/gather list of required modules for HTTPD once, to avoid service restarts needed for loading new modules vs reloads for vhosts updates. Though it still should not be a hard blocker for the migration, as this issue persists today anyway. * Linux Bridge Removal. While main patches for removal were merged: https://review.opendev.org/q/topic:%22osa/lxb_cleanup%22 we still have a Zun relying on LXB tests, which is obviously broken now. Some love needs to be given for tests to pass. * Fix compressed upload of CoreOS for Magnum Heat driver tests. As CoreOS is compressed out of the box, and Glance has no support for uploads of compressed files anymore, we need to implement decompression logic to the openstack_resources roles. Topic https://review.opendev.org/q/topic:%22osa/decompress_image%22 is created to track progress. * Document ceph source & version constraints for Noble, as there are no community repositories, only "native" one with ceph reef. Scoped for the Flamingo (2025.2): * PKI * driver for hashi-vault to be scoped for Flamingo as it requires quite a refactoring and we're too late into the release now. * Refactor standalone to create CSR and private key on the service side only * Issue certificate on the deploy host * Upgrade script to remove secret data from the deploy host * Reload libvirt config on certificates change to address VNC isuses * Shorter lifetimes * Drop Ubuntu 22.04 early in the cycle, as it is a blocker now for ansible-core 2.18 * ceph jobs are still running jammy * Add support for CentOS Stream 10 / Rocky 10 * We need to handle / create nodeset with infra to target only supported providers for tests * modular libvirt support * Use smaller nodes for molecule tests * Import Freezer role from https://github.com/noonedeadpunk/openstack-ansible-os_freezer once central scheduler support is merged to Freezer itself. * Continue coverage with Molecule tests * HAProxy role is smth we need to cover next * Complexity for TLS backends logic * IPv6/Multiple frontends * Coverage for individual roles inside of the plugins repository

1 0

[openstack][kolla-ansible][neutron] router ha was split brain when restarting l3 agent docker
by Nguyễn Hữu Khôi 14 Apr '25

14 Apr '25

Hello., I deploy a multi node cluster with 2 network nodes separately with l3_ha=true and max_l3_agents_per_router=2. It is ok but when I restart neutron_l3_agent on a host which has an active router then 2 routers become standby-standby or active-active. I feel 2 network nodes very easily to split brain. Could I have some ideas on this problem. Thank you. openstack 2025.1-master kolla-ansible2025.1 master ubuntu 22.04 Nguyen Huu Khoi

2 5

[Swift] S3/Swift compatibility: getBucketPolicy
by coolnodje＠gmail.com 14 Apr '25

14 Apr '25

I'm meeting some issues providing S3 compatible storage on OpenStack with editor solutions requiring getBucketPolicy in their S3 usage. I'm checking what can be done from the editor side, but I'd like to know if there exists a roadmap where getBucketPolicy would become supported. Thanks

1 0

OpenInfra Days Korea 2025 발표자 모집 안내
by Seongsoo Cho 14 Apr '25

14 Apr '25

안녕하세요 오픈인프라 한국 사용자모임 대표 조성수입니다. OpenInfra Days Korea 2025 발표자 모집이 시작되어 공지합니다. 미래를 여는 열쇠, 오픈인프라 ( Open Infrastructure, the Key to the Future ) 최근 클라우드 시장의 다양한 변화와 함께 오픈스택과 오픈인프라의 열기가 다시 뜨거워지고 있습니다. 특히 AI, HPC, 5G와 같은 떠오르는 첨단 기술들의 뒤에는 그 모든 가능성을 현실로 만드는 기반, 오픈인프라가 있습니다. 유연하고 확장 가능한 오픈인프라는 변화하는 기술 흐름에 민첩하게 대응하며 AI와 같은 고도화된 기술이 안정적으로 작동할 수 있도록 뒷받침합니다. 화려한 기술의 이면에서, 미래를 향한 문을 열고 있는 진짜 열쇠는 오픈인프라입니다. 지금 이 순간에도, 오픈인프라는 기술의 가능성을 넓히고, 세상을 연결하며, 더 나은 내일을 만들어가고 있습니다. OpenInfra Days Korea 2025에서 미래를 여는 열쇠를 함께 제안하고 함께 만들어보시기 바랍니다. 우리 손으로 기술의 내일을 열어가는 여정에 발표자 여러분들을 초대합니다 행사 일시 : 2025년 8월 26일 (화) 장소 : 서울 aT센터 발표자 모집 일정 : 4월 14일 (월) ~ 6월 15일 (일) 발표자 선정 안내 : 6월 30일 (월) 발표 시간 : 30분 문의 : program(a)openinfradays.kr 지원하러 가기 : https://2025.openinfradays.kr 이후 모든 공지는 https://forum.openinfra-kr.org 의 공지 게시판을 참고해주시기 바랍니다. -- Seongsoo Cho OpenStack Korea User Group / Community Leader IRC #seongsoocho

1 1

[ironic][networking][ops] PTG - Networking cases in Ironic
by Iury Gregory 11 Apr '25

11 Apr '25

Greetings everyone, During this week's PTG, we discussed networking aspects and the challenges we've encountered getting everyone on the same page. Ultimately, our biggest challenge is competing needs *and* requirements. The best way to describe the challenge is as a giant venn diagram where some interests overlap such that there are logical relationships and paths forward which can be drawn. But then there are also broadly useful themes to discuss, and requirements visit which then can also cloud the discussion. At the end of the day, the challenge is not to boil the ocean, but to focus on the distinct cases while also maybe draw the appropriate boundaries to defer specific items or have the necessary plan or idea of a plan in our back pocket before proceeding forward. So we tried bucketing in the form of hands raised during the PTG[0] to begin to give us some initial data points in an unscientific model of +1s. - Standalone operators who need non-neutron networking (mvp: tag/untag pots): +6 - Neutron Integrated Operators who are just using switches and need better existing ML2 plugin support: +3 - 3rd party SDN integration (w/neutron): +2 - Ironic additional capabilities at the front-end (dynamic portgroup assembly/weightable scheduling of specific ports): +7 If you have specific interest in any of these buckets, please reach out to Ironic contributors by replying to this mailing list post, messaging us on IRC, or even leaving some additional comments on our etherpad[0] from the PTG. That being said, the challenge is sort of how to get started and begin to make velocity. Consensus at the PTG was to sort of take a "two specific focus areas" approach and allow sub-teams to self organize. The key aspect is to get both subteams to talk to each other on a regular basis to ensure there is any coss-collaboration, reviews, and other overall awareness is highlighted for the purposes of collaboration. Discussion has also yielded that there is some consensus in trying to keep that sync as a higher bandwidth video call. Thoughts? - The Ironic Team [0]: https://etherpad.opendev.org/p/ironic-ptg-april-2025 -- *Att[]'s* *Iury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Ironic Core* *Senior Software Engineer at Red Hat Brazil* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory(a)gmail.com <iurygregory(a)gmail.com>*

1 0

[release] Release countdown for week R-24
by Thierry Carrez 11 Apr '25

11 Apr '25

Welcome back to the release countdown emails! These will be sent at major points in the 2025.2 "Flamingo" development cycle, which should conclude with a final release on October 1st, 2025. Development Focus ----------------- At this stage in the release cycle, focus should be on planning the 2025.2 "Flamingo" development cycle, assessing Flamingo community goals and approving Flamingo specs. General Information ------------------- The 2025.2 "Flamingo" cycle is a standard 26-week development cycle. In case you haven't seen it yet, please take a look over the schedule for this release: https://releases.openstack.org/flamingo/schedule.html By default, the team PTL is responsible for handling the release cycle and approving release requests. This task can (and probably should) be delegated to release liaisons. Now is a good time to review release liaison information for your team and make sure it is up to date: https://opendev.org/openstack/releases/src/branch/master/data/release_liais… By default, all your team deliverables from the 2025.1 Epoxy release are continued in 2025.2 Flamingo with a similar release model. Upcoming Deadlines & Dates -------------------------- Flamingo-1 milestone: May 15, 2025 -- Thierry Carrez (ttx)

1 0

[octavia] Purging ERROR'ed amphora and load balancers stuck in transitional states
by Paul Browne 11 Apr '25

11 Apr '25

Hello Octavia users, For a while I've been trying to find a systematic way to remove Octavia LBs+Amphora that have become stuck in ERROR or transitional PENDING_* states ; $ openstack loadbalancer amphora list -f value | grep ERROR a2323f14-3aaa-418a-8249-111aaa9c21fe 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR MASTER 10.8.0.242 192.168.3.247 e5b236ba-e7ee-4ed7-9f58-57ce7a408489 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR BACKUP 10.8.0.190 192.168.3.247 6b556f28-93c9-49dd-b6ee-4379288e7957 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR MASTER 10.8.0.39 192.168.3.126 c669db5d-8686-4d5c-9e95-e02030b34301 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR BACKUP 10.8.0.174 192.168.3.126 $ openstack loadbalancer list -f value | grep -vi active 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe k8s-clusterapi-cluster-default-ci-6386871107-kube-upgrade-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.247 PENDING_UPDATE amphora d5e402fe-2c4b-49af-a700-532cb408cee5 k8s-clusterapi-cluster-default-ci-6386871107-latest-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.126 PENDING_DELETE amphora These resources are marked immutable and so cannot be failed over or deleted; $ openstack loadbalancer amphora failover a2323f14-3aaa-418a-8249-111aaa9c21fe Load Balancer 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe is immutable and cannot be updated. (HTTP 409) (Request-ID: req-6e66c4e8-c3d3-4549-a03c-367017c8c8b3) $ openstack loadbalancer failover 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe Invalid state PENDING_UPDATE of loadbalancer resource 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe (HTTP 409) (Request-ID: req-84b44212-e7a8-4101-a16f-18c774c0577e) The backing Nova instances for these Amphora do seem to exist and be in good working order. Is there any API way to purge these out of Octavia's service state, or would (very careful) DB hackery be required here? Many thanks, Paul Browne ******************* Paul Browne Research Computing Platforms University Information Services Roger Needham Building JJ Thompson Avenue University of Cambridge Cambridge United Kingdom E-Mail: pfb29(a)cam.ac.uk<mailto:pfb29@cam.ac.uk> Tel: 0044-1223-746548 *******************

4 6

Bug deputy week 15 (April 7 - April 11)
by Sahid Orentino Ferdjaoui 11 Apr '25

11 Apr '25

Hello Guys, Here is a quick summary of Neutron related bugs this week == neutron-server error with ml2_distributed_port_bindings https://bugs.launchpad.net/neutron/+bug/2106374/ Status: LOW This issue appears to be related to tenant network/database handling. It throws an unexpected database exception which ideally should be handled more gracefully. Possibly a low-hanging fruit, that is said more investigation may be needed to understand the root cause. == [ovn-octavia-provider] Loadbalancer member list show operating status as ONLINE when member is created for a SHUTOFF state vm https://bugs.launchpad.net/neutron/+bug/2106357/ Status: LOW This seems to be a corner case where a stopped VM can be added to a load balancer, and it incorrectly appears as ONLINE. Unfortunately, I didn’t get the chance to reproduce it this week, apologies for that. == ovn-sb leader constantly changing https://bugs.launchpad.net/neutron/+bug/2106690/ Status: INCOMPLETE but reported is active and will share more detail soon Seem like point of instability with OVN SB database leader election. Waiting for more details. == [OVN] No network type availability zone https://bugs.launchpad.net/neutron/+bug/2106719/ Status: NEW This one may require someone with better picture of ML2/OVN and AZ to properly evaluate and move forward. == after running ``os-nova-install.yml`` lost instances connectivity https://bugs.launchpad.net/neutron/+bug/2106793 Status: INCOMPLETE at that point The issue seems to occur after a Nova restart, where instances lose network connectivity. Needs more info and confirmation. == Unable to enable/disable src_nat per external gateway https://bugs.launchpad.net/neutron/+bug/2106776 Status: TRIAGED This is related to OVN and occurs in scenarios involving multiple external gateways. == Have a good day! I unfortunately wont be available for driver meeting. s.

1 0

Question about patch to fix problem
by Franck VEDEL (UGA) 11 Apr '25

11 Apr '25

Hello, A few days ago, I asked a question about applying a patch to resolve a problem in my OpenStack environment. I didn't get a response. My question was probably stupid and not appropriate for this list. I apologize. Having to resolve this problem, I searched high and low using search engines. It seems that the solution, with kolla-ansible, is to upgrade to Epoxy, which takes the patch into account. Since this update could cause other problems, I don't want to do it for the moment. Is there another solution? Is it possible to modify the code (it's simply a line to modify in a file (https://review.opendev.org/c/openstack/horizon/+/942946)) in the two Horizon containers in my cluster? I'm really sorry if this is a stupid question, but I've been searching and can't find a method for applying these patches. Franck

5 8

[kolla] IRC meeting cancelled - 9-th of April 2025
by Bartosz Bezak 11 Apr '25

11 Apr '25

Hi, Kolla IRC meeting today is cancelled due to ongoing Project Teams Gathering (PTG) this week. Best regards, Bartosz Bezak

2 3

[octavia] Point releases
by Mikhail Okhrimenko 10 Apr '25

10 Apr '25

Hello community! We found that the latest octavia bobcat release packages for different OS like Rocky Linux 9, Debian 12, Ubuntu 22.04 do not contain the latest bug fixes from the release notes https://docs.openstack.org/releasenotes/octavia/2023.2.html of version 13.0.0-37. And there is no tag with these fixes in the octavia repository https://github.com/openstack/octavia/tags. So could you please create a point release in the octavia repository so that we can build packages with these changes. We also noticed the same situation with the release versions of caracal and dalmatian.

4 3

[nova][NFS] Changing disk permission on reboot, permission denied
by Nell Jerram 10 Apr '25

10 Apr '25

Hi; I'm seeing a problem with a hard reboot operation, when /var/lib/nova/instances for the relevant compute node is on an NFS share exported from another compute node. I'm hoping someone here might be able to point to where to debug this next. 3 compute nodes: A, B and C. A exports its /var/lib/nova/instances over NFS v4. B and C mount that at /var/lib/nova/instances. (This is the kind of setup that I understand to be required for live migration to be possible.) If I create a VM on A, wait for it to be ACTIVE, then `nova reboot --hard testvm1`, it's fine, i.e. it reboots and returns to ACTIVE state. If I create a VM on B, wait for it to be ACTIVE, then `nova reboot --hard testvm2`, it goes permanently into ERROR state. The Nova compute log has Stderr: "qemu-img: Could not open '/var/lib/nova/instances/02bf258b-41bc-451a-9d3c-af6741c6060f/disk': Could not open '/var/lib/nova/instances/02bf258b-41bc-451a-9d3c-af6741c6060f/disk': Permission denied\n": nova.exception.InvalidDiskInfo: Disk info file is invalid: qemu-img failed to execute on /var/lib/nova/instances/02bf258b-41bc-451a-9d3c-af6741c6060f/disk : Unexpected error while running command. and the ownership on that file is in fact now: -rw-r----- 1 root root 29491200 Apr 7 18:27 disk Before the reboot operation it was: -rw-r----- 1 libvirt-qemu kvm 29491200 Apr 7 18:25 disk There isn't a similar ownership change when rebooting a VM on node A, so presumably this is something to do with NFS being involved. But I believe I've followed standard NFS setup advice, and my user and group IDs are the same on all compute nodes: $ getent group nova nova:x:1003:libvirt-qemu $ getent passwd nova nova:x:1002:1003::/home/nova:/bin/bash $ getent passwd libvirt-qemu libvirt-qemu:x:64055:109:Libvirt Qemu,,,:/var/lib/libvirt:/usr/sbin/nologin $ getent group kvm kvm:x:109:nova Any clues on how to debug or fix this next? Many thanks - Nell

3 9

Re: [openstack-magnum ui] Cannot choose an external network ID when creating a cluster template
by Eugen Block 10 Apr '25

10 Apr '25

Please don't drop the list from your responses. Zitat von Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com>: > Hello, They r different, > I check and see this happen because horizon switched to SDK > > https://github.com/openstack/horizon/commit/56f0e2685e2692467442d5be846b863… > > and Magnum-UI still keep old > > https://github.com/openstack/magnum-ui/blob/master/magnum_ui/static/dashboa… > > Thank you for replying to me. > > Nguyen Huu Khoi > > > On Thu, Apr 10, 2025 at 5:11 PM Eugen Block <eblock(a)nde.ag> wrote: > >> Could it be this bug? >> >> https://bugs.launchpad.net/horizon/+bug/2077329 >> >> Zitat von Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com>: >> >> > Hello, When I create a cluster template, I cannot choose an external >> > network ID, it disappears from horizon. >> > My env: >> > >> > Ubuntu 22.04 >> > Openstack 2024.2 >> > >> > Thank you. Regards >> > Nguyen Huu Khoi >> >> >> >>

2 1

[PTL][release][stable] 2023.2 Bobcat transition to End of Life
by Elõd Illés 10 Apr '25

10 Apr '25

Hi teams, As 2025.1 Epoxy was released last week and we are in a less busy period (except the PTG :)), now is the good time to call your attention to the following: In 3 weeks 2023.2 Bobcat is planned to transition to End of Life [1] (planned date: 2025-04-30). Note that 2023.2 Bobcat is not a SLURP release, hence it is not eligible to transition to Unmaintained [2] and will move to End of Life directly and the tip of stable/2023.2 branches will be tagged with 2023.2-eol tag and stable/2023.2 branches will be deleted afterwards. To projects with cycle-with-trailing release model (Kolla, OpenStack Ansible, Puppet OpenStack): transition of your projects have an extended deadline, ~2 months after the planned EOL date (2025-04-30). I have generated the list of the current *open* and *unreleased* changes in stable/2023.2 for every repositories [3] (where there are such patches). These lists could help the teams who are planning to do a *final* release on Bobcat before deleting stable/2023.2 branches. Feel free to edit and extend these lists to track your team's progress! *NOTE*: teams, please focus on wrapping up your libraries first if there is any concern about the changes, in order to avoid broken (final!!) releases! Also please try to avoid any regression in the final stable release and do not propose any release that has only non-functional code changes. [1] https://releases.openstack.org/ [2] https://docs.openstack.org/project-team-guide/stable-branches.html#unmainta… [3] https://etherpad.opendev.org/p/2023.2-bobcat-final-release-before-eol Thanks, Előd Illés irc: elodilles @ #openstack-release

1 0