April 2025 - openstack-discuss - lists.openstack.org

[I18n] Broken ko_KR translation for OpenStack-Ansible
by Dmitriy Rabotyagov 14 Apr '25

14 Apr '25

Hey folks, Today it was brought to my attention that OpenStack-Ansible does not get any periodic updates for translations for around 2 years now. Once I've checked the result of periodic job, it seems that a ko_KR translation makes Zuul job to fail: https://zuul.opendev.org/t/openstack/build/fc90cb4be70e45e8bec07e981eaff01e Specifically, following translations are containing the error (/n in translated version, which are not present in original message): * resource-id:3db2c4ee55f9791e84ff3137e05be661 * resource-id:9f3ab9a8642b37abfcdc47804c05a49f * resource-id:8cd037448cc79d28322aeb8ec25dac58 * resource-id:182414aa5a44b5a113335bc17346b853 It would be much appreciated if somebody with access to ko_KR could fix that as soon as possible. Or, alternatively, grant me permissions to translations of the project, as to PTL. Thanks in advance!

2 2

[Magnum] Issues with Magnum (K8s 1.21.11 Clusters)
by Oliver Weinmann 14 Apr '25

14 Apr '25

Hi all, We’re running a Kolla-Ansible 2023.2 deployment with Magnum. While we primarily use CAPI, we also have some older Kubernetes 1.21.11 clusters. In these clusters, we’re encountering two specific issues: Random Crashes of openstack-cloud-controller-manager Pod In one of the older clusters, the openstack-cloud-controller-manager pod is crashing intermittently with the following error: F0410 12:02:45.673777 1 main.go:84] Cloud provider could not be initialized: could not init cloud provider "openstack": Post "http://openstack.xxxxxxx:5000/v3/auth/tokens": dial tcp 172.xxx.xxx.xxx:5000: i/o timeout However, when I manually test connectivity to the Keystone API from the master node, it appears to work fine: while true; do status=$(curl -s -o /dev/null -w "%{http_code}" http://172.xxx.xxx.xxx:5000/v3 2>/dev/null) || status="failed" echo "$(date): Keystone API status: $status" sleep 10 done Sample output: Thu Apr 10 19:19:50 UTC 2025: Keystone API status: 200 Thu Apr 10 19:20:00 UTC 2025: Keystone API status: 200 Thu Apr 10 19:20:10 UTC 2025: Keystone API status: 200 ... Despite the API being reachable, the pod continues to crash sporadically with the timeout error. "Too Many Open Files" Error in Pods Users running pods in these clusters are reporting frequent "Too many open files" errors. Since Kubernetes 1.21.11 relies on CoreOS, I’m looking for a way to permanently increase the maximum number of open file descriptors on CoreOS. Any guidance or suggestions on how to address these issues would be greatly appreciated! Best regards, Oliver

2 2

[openstack-ansible][ptg] Flamingo PTG results
by Dmitriy Rabotyagov 14 Apr '25

14 Apr '25

Hey folks, Last week along with other teams we had a PTG session where a development focus for the project was discussed. I want to thank everyone who attended the session and participated in discussions. Despite the small amount of time scheduled for the session, it was very productive and you might find the summary below. Finalizing Epoxy release As OpenStack-Ansible does have a trailing release model, we still have some time left for releasing the Epoxy, with the deadline being set at June 6, 2025. With that a few topics were highlighted to be worked on for the release to be finished: * Finalize migration for Ansible to use FQCN. While this is not obligatory as of today nor a blocker for further development - we decided to adopt our roles to the new standard introduced with ansible-core 2.10. And this topic is pretty much merged at the moment: https://review.opendev.org/q/topic:%22osa/fqcn%22 * Deprecate openstack-ansible-tests repo to avoid releasing it for Epoxy. At the moment, the only useful data in the repo is the sync tool (proposal bot). We agreed to move the content/functionality we need to the integrated repository. * Finalize migration of services to HTTPD role. At the moment, only Keystone and Horizon are left for the migration (the toughest ones). One of the challenges for the migration was highlighted the need to install/gather list of required modules for HTTPD once, to avoid service restarts needed for loading new modules vs reloads for vhosts updates. Though it still should not be a hard blocker for the migration, as this issue persists today anyway. * Linux Bridge Removal. While main patches for removal were merged: https://review.opendev.org/q/topic:%22osa/lxb_cleanup%22 we still have a Zun relying on LXB tests, which is obviously broken now. Some love needs to be given for tests to pass. * Fix compressed upload of CoreOS for Magnum Heat driver tests. As CoreOS is compressed out of the box, and Glance has no support for uploads of compressed files anymore, we need to implement decompression logic to the openstack_resources roles. Topic https://review.opendev.org/q/topic:%22osa/decompress_image%22 is created to track progress. * Document ceph source & version constraints for Noble, as there are no community repositories, only "native" one with ceph reef. Scoped for the Flamingo (2025.2): * PKI * driver for hashi-vault to be scoped for Flamingo as it requires quite a refactoring and we're too late into the release now. * Refactor standalone to create CSR and private key on the service side only * Issue certificate on the deploy host * Upgrade script to remove secret data from the deploy host * Reload libvirt config on certificates change to address VNC isuses * Shorter lifetimes * Drop Ubuntu 22.04 early in the cycle, as it is a blocker now for ansible-core 2.18 * ceph jobs are still running jammy * Add support for CentOS Stream 10 / Rocky 10 * We need to handle / create nodeset with infra to target only supported providers for tests * modular libvirt support * Use smaller nodes for molecule tests * Import Freezer role from https://github.com/noonedeadpunk/openstack-ansible-os_freezer once central scheduler support is merged to Freezer itself. * Continue coverage with Molecule tests * HAProxy role is smth we need to cover next * Complexity for TLS backends logic * IPv6/Multiple frontends * Coverage for individual roles inside of the plugins repository

1 0

[openstack][kolla-ansible][neutron] router ha was split brain when restarting l3 agent docker
by Nguyễn Hữu Khôi 14 Apr '25

14 Apr '25

Hello., I deploy a multi node cluster with 2 network nodes separately with l3_ha=true and max_l3_agents_per_router=2. It is ok but when I restart neutron_l3_agent on a host which has an active router then 2 routers become standby-standby or active-active. I feel 2 network nodes very easily to split brain. Could I have some ideas on this problem. Thank you. openstack 2025.1-master kolla-ansible2025.1 master ubuntu 22.04 Nguyen Huu Khoi

2 5

[Swift] S3/Swift compatibility: getBucketPolicy
by coolnodje＠gmail.com 14 Apr '25

14 Apr '25

I'm meeting some issues providing S3 compatible storage on OpenStack with editor solutions requiring getBucketPolicy in their S3 usage. I'm checking what can be done from the editor side, but I'd like to know if there exists a roadmap where getBucketPolicy would become supported. Thanks

1 0

OpenInfra Days Korea 2025 발표자 모집 안내
by Seongsoo Cho 14 Apr '25

14 Apr '25

안녕하세요 오픈인프라 한국 사용자모임 대표 조성수입니다. OpenInfra Days Korea 2025 발표자 모집이 시작되어 공지합니다. 미래를 여는 열쇠, 오픈인프라 ( Open Infrastructure, the Key to the Future ) 최근 클라우드 시장의 다양한 변화와 함께 오픈스택과 오픈인프라의 열기가 다시 뜨거워지고 있습니다. 특히 AI, HPC, 5G와 같은 떠오르는 첨단 기술들의 뒤에는 그 모든 가능성을 현실로 만드는 기반, 오픈인프라가 있습니다. 유연하고 확장 가능한 오픈인프라는 변화하는 기술 흐름에 민첩하게 대응하며 AI와 같은 고도화된 기술이 안정적으로 작동할 수 있도록 뒷받침합니다. 화려한 기술의 이면에서, 미래를 향한 문을 열고 있는 진짜 열쇠는 오픈인프라입니다. 지금 이 순간에도, 오픈인프라는 기술의 가능성을 넓히고, 세상을 연결하며, 더 나은 내일을 만들어가고 있습니다. OpenInfra Days Korea 2025에서 미래를 여는 열쇠를 함께 제안하고 함께 만들어보시기 바랍니다. 우리 손으로 기술의 내일을 열어가는 여정에 발표자 여러분들을 초대합니다 행사 일시 : 2025년 8월 26일 (화) 장소 : 서울 aT센터 발표자 모집 일정 : 4월 14일 (월) ~ 6월 15일 (일) 발표자 선정 안내 : 6월 30일 (월) 발표 시간 : 30분 문의 : program(a)openinfradays.kr 지원하러 가기 : https://2025.openinfradays.kr 이후 모든 공지는 https://forum.openinfra-kr.org 의 공지 게시판을 참고해주시기 바랍니다. -- Seongsoo Cho OpenStack Korea User Group / Community Leader IRC #seongsoocho

1 1

[ironic][networking][ops] PTG - Networking cases in Ironic
by Iury Gregory 11 Apr '25

11 Apr '25

Greetings everyone, During this week's PTG, we discussed networking aspects and the challenges we've encountered getting everyone on the same page. Ultimately, our biggest challenge is competing needs *and* requirements. The best way to describe the challenge is as a giant venn diagram where some interests overlap such that there are logical relationships and paths forward which can be drawn. But then there are also broadly useful themes to discuss, and requirements visit which then can also cloud the discussion. At the end of the day, the challenge is not to boil the ocean, but to focus on the distinct cases while also maybe draw the appropriate boundaries to defer specific items or have the necessary plan or idea of a plan in our back pocket before proceeding forward. So we tried bucketing in the form of hands raised during the PTG[0] to begin to give us some initial data points in an unscientific model of +1s. - Standalone operators who need non-neutron networking (mvp: tag/untag pots): +6 - Neutron Integrated Operators who are just using switches and need better existing ML2 plugin support: +3 - 3rd party SDN integration (w/neutron): +2 - Ironic additional capabilities at the front-end (dynamic portgroup assembly/weightable scheduling of specific ports): +7 If you have specific interest in any of these buckets, please reach out to Ironic contributors by replying to this mailing list post, messaging us on IRC, or even leaving some additional comments on our etherpad[0] from the PTG. That being said, the challenge is sort of how to get started and begin to make velocity. Consensus at the PTG was to sort of take a "two specific focus areas" approach and allow sub-teams to self organize. The key aspect is to get both subteams to talk to each other on a regular basis to ensure there is any coss-collaboration, reviews, and other overall awareness is highlighted for the purposes of collaboration. Discussion has also yielded that there is some consensus in trying to keep that sync as a higher bandwidth video call. Thoughts? - The Ironic Team [0]: https://etherpad.opendev.org/p/ironic-ptg-april-2025 -- *Att[]'s* *Iury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Ironic Core* *Senior Software Engineer at Red Hat Brazil* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory(a)gmail.com <iurygregory(a)gmail.com>*

1 0

[release] Release countdown for week R-24
by Thierry Carrez 11 Apr '25

11 Apr '25

Welcome back to the release countdown emails! These will be sent at major points in the 2025.2 "Flamingo" development cycle, which should conclude with a final release on October 1st, 2025. Development Focus ----------------- At this stage in the release cycle, focus should be on planning the 2025.2 "Flamingo" development cycle, assessing Flamingo community goals and approving Flamingo specs. General Information ------------------- The 2025.2 "Flamingo" cycle is a standard 26-week development cycle. In case you haven't seen it yet, please take a look over the schedule for this release: https://releases.openstack.org/flamingo/schedule.html By default, the team PTL is responsible for handling the release cycle and approving release requests. This task can (and probably should) be delegated to release liaisons. Now is a good time to review release liaison information for your team and make sure it is up to date: https://opendev.org/openstack/releases/src/branch/master/data/release_liais… By default, all your team deliverables from the 2025.1 Epoxy release are continued in 2025.2 Flamingo with a similar release model. Upcoming Deadlines & Dates -------------------------- Flamingo-1 milestone: May 15, 2025 -- Thierry Carrez (ttx)

1 0

[octavia] Purging ERROR'ed amphora and load balancers stuck in transitional states
by Paul Browne 11 Apr '25

11 Apr '25

Hello Octavia users, For a while I've been trying to find a systematic way to remove Octavia LBs+Amphora that have become stuck in ERROR or transitional PENDING_* states ; $ openstack loadbalancer amphora list -f value | grep ERROR a2323f14-3aaa-418a-8249-111aaa9c21fe 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR MASTER 10.8.0.242 192.168.3.247 e5b236ba-e7ee-4ed7-9f58-57ce7a408489 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe ERROR BACKUP 10.8.0.190 192.168.3.247 6b556f28-93c9-49dd-b6ee-4379288e7957 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR MASTER 10.8.0.39 192.168.3.126 c669db5d-8686-4d5c-9e95-e02030b34301 d5e402fe-2c4b-49af-a700-532cb408cee5 ERROR BACKUP 10.8.0.174 192.168.3.126 $ openstack loadbalancer list -f value | grep -vi active 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe k8s-clusterapi-cluster-default-ci-6386871107-kube-upgrade-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.247 PENDING_UPDATE amphora d5e402fe-2c4b-49af-a700-532cb408cee5 k8s-clusterapi-cluster-default-ci-6386871107-latest-kubeapi 3a06571936a0424bb40bc5c672c4ccb1 192.168.3.126 PENDING_DELETE amphora These resources are marked immutable and so cannot be failed over or deleted; $ openstack loadbalancer amphora failover a2323f14-3aaa-418a-8249-111aaa9c21fe Load Balancer 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe is immutable and cannot be updated. (HTTP 409) (Request-ID: req-6e66c4e8-c3d3-4549-a03c-367017c8c8b3) $ openstack loadbalancer failover 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe Invalid state PENDING_UPDATE of loadbalancer resource 1fa7bd54-f60c-420c-94f3-d4c02f03d4fe (HTTP 409) (Request-ID: req-84b44212-e7a8-4101-a16f-18c774c0577e) The backing Nova instances for these Amphora do seem to exist and be in good working order. Is there any API way to purge these out of Octavia's service state, or would (very careful) DB hackery be required here? Many thanks, Paul Browne ******************* Paul Browne Research Computing Platforms University Information Services Roger Needham Building JJ Thompson Avenue University of Cambridge Cambridge United Kingdom E-Mail: pfb29(a)cam.ac.uk<mailto:pfb29@cam.ac.uk> Tel: 0044-1223-746548 *******************

4 6

Bug deputy week 15 (April 7 - April 11)
by Sahid Orentino Ferdjaoui 11 Apr '25

11 Apr '25

Hello Guys, Here is a quick summary of Neutron related bugs this week == neutron-server error with ml2_distributed_port_bindings https://bugs.launchpad.net/neutron/+bug/2106374/ Status: LOW This issue appears to be related to tenant network/database handling. It throws an unexpected database exception which ideally should be handled more gracefully. Possibly a low-hanging fruit, that is said more investigation may be needed to understand the root cause. == [ovn-octavia-provider] Loadbalancer member list show operating status as ONLINE when member is created for a SHUTOFF state vm https://bugs.launchpad.net/neutron/+bug/2106357/ Status: LOW This seems to be a corner case where a stopped VM can be added to a load balancer, and it incorrectly appears as ONLINE. Unfortunately, I didn’t get the chance to reproduce it this week, apologies for that. == ovn-sb leader constantly changing https://bugs.launchpad.net/neutron/+bug/2106690/ Status: INCOMPLETE but reported is active and will share more detail soon Seem like point of instability with OVN SB database leader election. Waiting for more details. == [OVN] No network type availability zone https://bugs.launchpad.net/neutron/+bug/2106719/ Status: NEW This one may require someone with better picture of ML2/OVN and AZ to properly evaluate and move forward. == after running ``os-nova-install.yml`` lost instances connectivity https://bugs.launchpad.net/neutron/+bug/2106793 Status: INCOMPLETE at that point The issue seems to occur after a Nova restart, where instances lose network connectivity. Needs more info and confirmation. == Unable to enable/disable src_nat per external gateway https://bugs.launchpad.net/neutron/+bug/2106776 Status: TRIAGED This is related to OVN and occurs in scenarios involving multiple external gateways. == Have a good day! I unfortunately wont be available for driver meeting. s.

1 0