April 2024 - openstack-discuss - lists.openstack.org

[Neutron] [VPNaaS] VMs unable to hit tunnel remote cidr
by Karl Kloppenborg 23 Apr '24

23 Apr '24

Hi Neutron/VPNaaS teams. We run an openstack which neutron runs on the following VPNaaS setup: service_provider: VPN:strongswan:neutron_vpnaas.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default L3-agent vpn_device_driver: neutron_vpnaas.services.vpn.device_drivers.strongswan_ipsec.StrongSwanDriver With Neutron running the following service plugins: service_plugins: neutron_dynamic_routing.services.bgp.bgp_plugin.BgpPlugin,neutron.services.l3_router.l3_router_plugin.L3RouterPlugin,router,vpnaas,trunk,segments,bgp And Ml2 config Openvswitch_agent -> Security Group -> firewall driver: openvswitch We have an IPSec tunnel up between a remote site (Fortinet device) and us. Computers within the remote site can reach the VM’s internal IP’s on Openstack across the tunnel, however VM’s from openstack cannot reach the internal network on the remote side. A traceroute from the VM to a known IP on the remote side shows the IP instead transiting out the router gateway. Has anyone seen this before? It looks like something isn’t being picked up/a route not being set. Any assistance would be greatly appreciated. Thanks, Karl.

3 4

[nova][kolla-ansible] having issue running nova API bootstrap container
by James Leong 23 Apr '24

23 Apr '24

Hi all, I am currently trying to deploy a yoga version of OpenStack, However, I am having the following issues when doing a deployment using kolla-ansible. The error occurs on task "[nova : Running Nova API bootstrap container]." "os_traits.COMPUTE_ADDRESS_SPACE_PASSTHROUGH,", "AttributeError: module 'os_traits' has no attribute 'COMPUTE_ADDRESS_SPACE_PASSTHROUGH' Best, James

2 2

[tc] Weekly TC meeting agenda for Apr 23rd 2024
by Goutham Pacha Ravi 22 Apr '24

22 Apr '24

Hello Stackers, We're four weeks [1] into the 2024.2 release of OpenStack and there's a lot of exciting governance proposals at the table already. Please join us to discuss these at the weekly meeting of the OpenStack Technical Committee tomorrow, Tuesday April 23rd, at 1800 UTC in the #openstack-tc channel on OFTC. If you haven't already done so, JayF wrote up a detailed summary of our recently concluded PTG for this list [2]. We'll be using this to drive some of the discussions tomorrow. Thank you, Jay! The agenda for tomorrow's meeting is: * Roll call * AIs from the TC vPTG 2024.2 #link https://etherpad.opendev.org/p/apr2024-ptg-os-tc (Technical Committee vPTG notes) * 2024.2 TC Tracker #link https://etherpad.opendev.org/p/tc-2024.2-tracker (Technical Committee activity tracker) * Ongoing business ** Clarify the timeline for project to move to Inactive state (https://review.opendev.org/c/openstack/governance/+/908880) ** Appoint Wenxiang Wu as PTL of Skyline (https://review.opendev.org/c/openstack/governance/+/915108) ** Transition Freezer project to DPL (https://review.opendev.org/c/openstack/governance/+/914911) ** Assign Dmitriy Rabotyagov as Freezer PTL (https://review.opendev.org/c/openstack/governance/+/915727) ** Move docs tools into TC governance (https://review.opendev.org/c/openstack/governance/+/914950) * Other governance proposals that will transition to "formal-vote" * 2025.1 Elections * TC Summary Email * Open Discussion and Reviews See you there! Thank you, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] https://releases.openstack.org/dalmatian/schedule.html [2] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack…

1 0

[neutron] Bug deputy report (week starting on April 15)
by Elvira Garcia Ruiz 22 Apr '24

22 Apr '24

Hi folks, This is the Neutron deputy report for the week of April 15 - 21. Thanks a lot to froyo for taking a look at a wave of ovn-octavia-provider bugs that appeared on our Launchpad during these last couple of hours! High ------ - https://bugs.launchpad.net/neutron/+bug/2062385 - [ovn-octavia-provider] Member with FIP not reachable Assigned to Fernando Royo - Might actually just be a known limitation of OVN. - https://bugs.launchpad.net/neutron/+bug/2061883 - [fwaas] Duplicate entry <uuid> for key default_firewall_groups.PRIMARY' Unassigned Medium ----------- - https://bugs.launchpad.net/neutron/+bug/2062536 - [ovn] agent_health_check does not work for ovn agents Assigned to Liu Xie. In progress. - https://bugs.launchpad.net/neutron/+bug/2062511 - [ovn] ovn metadata agent occasionally down if SB is snapshoting Fix proposed by Liu Xie https://review.opendev.org/c/openstack/neutron/+/916387 - https://bugs.launchpad.net/neutron/+bug/2062009 - Neutron-server + uwsgi deadlocks whenr unning rpc workers Fix proposed by sebageek https://review.opendev.org/c/openstack/neutron/+/916123 Incomplete --------------- - https://bugs.launchpad.net/neutron/+bug/2061350 - [ovn] The 'ovn_client' of the L3 plugin is occasionally NoneType Undecided -------------- These have been recently moved to neutron launchpad from the octavia one because they belong to the ovn-octavia-provider: - https://bugs.launchpad.net/neutron/+bug/2062965 - octavia/ovn: filedescriptor out of range in select() Unassigned - https://bugs.launchpad.net/neutron/+bug/2063043 - octavia/ovn: missed healthmon port cleanup Unassigned And this was just filed about an hour ago, also ovn-octavia-provider related: - https://bugs.launchpad.net/neutron/+bug/2063081 - [ovn] connectivity to load balancer with FIP from a server connected to a router with external gateway Unassigned Currently Fernando Royo is taking a look at the three of them for further triage. Best regards, Elvira

1 0

[tc] Congratulations, Goutham Pacha Ravi is new TC Chair
by Jay Faulkner 20 Apr '24

20 Apr '24

Hi all, I'd like to congratulate Goutham Pacha Ravi on becoming the TC Chair for 2024.2 cycle [1]. Thanks for the opportunity to serve as chair for the last six months; please treat Goutham kindly during his time in the role as you did me. Thanks, Jay Faulkner OpenStack TC 1: https://review.opendev.org/c/openstack/governance/+/916024

9 8

[openstack-helm] Cleanup OSH ingress chart
by Vladimir Kozhukalov 20 Apr '24

20 Apr '24

Hi dear colleagues, Some time ago we switched all our test deployment jobs to using the upstream Ingress-nginx chart. So it looks like we can recommend using one of the third party ingress controller charts to all our users. The deployment requirements for third party ingress controllers like ingress class name and ingress pod labels are described in the documentation [1] and this script [2] can be used as an example. I suggest removing the OSH ingress chart to reduce the code base that we need to support. Opinions are very welcome. [1] https://docs.openstack.org/openstack-helm/latest/install/deploy_ingress_con… [2] https://opendev.org/openstack/openstack-helm/src/branch/master/tools/deploy… -- Best regards, Kozhukalov Vladimir

4 4

[kolla][kolla-ansible] Promethus libvirt_export with project_name label
by Satish Patel 19 Apr '24

19 Apr '24

Folks, I am collecting metrics of libvirt using prometheus_libvirt_exporter and it works but I am trying to add project_name tag in each libvirt label so I can shorten them by project_name. It Looks like kolla default image of prometheus libvirt version doesn't have that option to send project name. Based on this repo - https://github.com/kumina/libvirt_exporter?tab=readme-ov-file We can add --libvirt.export-nova-metadata flag to send project_name label in each job. Do I need different versions of libvirt_exporter to make it work or am I missing something? # /opt/libvirt-exporter --help usage: libvirt_exporter [<flags>] Prometheus metrics exporter for libvirt Flags: --help Show context-sensitive help (also try --help-long and --help-man). --web.listen-address=":9177" Address to listen on for web interface and telemetry. --web.telemetry-path="/metrics" Path under which to expose metrics. --libvirt.uri="qemu:///system" Libvirt URI from which to extract metrics. --version Show application version.

1 0

[tc] TC vPTG 2024.2 Summary / Notes
by Jay Faulkner 19 Apr '24

19 Apr '24

Hi all, We had TC PTG sessions during the vPTG last week. I'm going to give an extremely high level summary of topics/action items here, as always for full context please see the etherpad: https://etherpad.opendev.org/p/apr2024-ptg-os-tc -- and, as always, remember TC decisions are made via gerrit patches; PTG conversations just help us rapidly spin up context for input to those decisions. # TC <> Community Interaction session Job failures were a hot topic during this session, with TC and community members discussing strategies for helping newer contributors troubleshoot and understand the importance of troubleshooting intermittent gate failures. Several suggestions were made, but no concrete actions were identified to take. Getting things done in Openstack / 'modernization' was the other topic discussed in this session. Some contributors identified that changes such as more strict typing may not be considered a straightforward benefit over duck-typed python. Generally there was no conclusion here either for the whole of OpenStack, however, individual projects should not feel like they need to wait for "permission" to perform these kinds of modernization activities. # OpenStack - where are we, where are we going ## User Survey Analysis Slaweq summarized the user survey in a governance patch, https://review.opendev.org/c/openstack/governance/+/900516, as usual (thank you!) and the TC members and community discussed it. It was noted that as he was summarizing the survey results, Slaweq indicated he found some comments which went against community spirit, attributing a lack of contribution to "not helping competition". Some of the other comments were around data and the validity thereof; such as the 24% of users who responded they don't upgrade OpenStack potentially answering that because they replace the cluster in-place. ACTION: Goutham volunteered to help reword the questions to avoid pitfalls mentioned during the session. ## Re-Visioning OpenStack aka "What is OpenStack Now" JayF, in proposing this topic, suggested that the mission of OpenStack had gotten so large, and the projects so varied, that it's difficult to know what OpenStack itself means -- much less manage it as a whole. This reality was potentially proven via the unfixed Murano CVE during the last cycle. We had some discussion around what being in OpenStack means, and it was reiterated that OpenStack projects are not required to have security issues managed by the VMT or their process. It was pointed out that one of the side effects is that it's difficult to determine what common OpenStack components to teach given the number on the periphery. There was a suggestion that the TC should be doing project reviews to address this concern, but no concrete action was taken. # TC - Specific new governance issues ## Affiliation Diversity Handling in OpenStack TC As most reading this likely already know, during the last TC election there was an issue where affiliation diversity requirements combined with the elected TC members came into conflict. While Bauzas resolved this conflict by stepping aside, we want to prevent this kind of situation from deadlocking in the future. There was a large amount of async discussion and comments on this topic, so I hesitate to summarize them here. Please review the etherpad directly for the myriad of ideas and issues presented by the large amount of engagement in this issue. ACTION: gmann offered to propose guidelines for resolving this in the TC charter. ## Raising the bar for Activity This topic was proposed in response to the Murano CVE and the PTL not participating in a security fix. This raises the question as to how we assessed Murano was an active project, and how to prevent this kind of issue in the future. No consensus was reached on any of the larger-scoped items, however, it was indicated by Julia that the new Cyber Resiliancy Act guidelines, when they are released, may require us to take some kind of action on this front. There were some small items which garnered consensus, such as marking all leaderless projects as inactive by default -- as originally discussed in the 2024.1 PTG. This was another very active discussion with lots of both sync and async engagement -- I strongly suggest you read the direct comments for this topic to get a full feeling of the breadth of this issue. ACTION: https://review.opendev.org/c/openstack/governance/+/914726 was proposed by gmann, which implements the consensus idea to mark leaderless projects as inactive by default. ## Eventlet/AsyncIO migration goal This issue was raised to give a venue for https://review.opendev.org/c/openstack/governance/+/902585 to get synchronous comments/discussion. Dan indicated he had not voted in favor of the change, and didn't plan to until he felt it had a reasonable likelihood to succeed in OpenStack -- early tests of this functionality failed. Jay indicated that his primary concern was elimination of eventlet; the new maintainer group does not plan on keeping eventlet alive forever. Bauzas pointed out that this is more than a goal -- we need dedicated effort in order to execute on this idea. Sean Mooney indicated that we need a non-wsgi service to migrate as an example. Along these lines, we had agreement to reduce scope of the goal to remove eventlet dependencies/requirements from shared OpenStack libraries to enable individual projects to have an easier time with proof of concepts, and revisit the larger goal after such a proof of concept exists. ACTION: JayF to propose a new, smaller scoped goal around removing eventlet dependencies. # Follow-ups from Ongoing TC Activities ## Unmaintained Migration pain points There have been some lingering pain points while migrating from the Extended Maintenance model to Unmaintained. Firstly, despite these branches being "unmaintained", we've set a minimum requirement of green CI with no Zuul config errors in order for a project to continue having their branch open for submission. This does not dictate a minimum level of testing; only that such tests are running and passing. These issues must be repaired in order for an unmaintained branch to remain open, but there are significant numbers that are failing. Please see Elod's recent email to the list for more information and deadlines on these cases. Secondly, tooling for unmaintained branch EOL still needs to have automation completed -- the etherpad contains a full list of the items needed. This goes along with the overall sentiment of this session: the unmaintained branch migration has created an enormous amount of extra work. ACTION: Please go look at Elod's email! This email being sent was a primary action related to this PTG session. ## PyPI Maintainer Cleanup The PyPi maintainer cleanup effort, detailed here: https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup has been ongoing for over a year. As of March 15, 2024, we still have 232 project/maintainer pairs to cleanup and 110 separate humans who have access to at least one openstack pypi deliverable. The TC discussed next steps; consensus was that in cases where openstackci holds owner permissions, we are in favor of using those permissions to cleanup where possible. However, in cases where openstackci does not hold owner permissions, it gets a bit more difficult and we have fewer options: either engage with the PSF's process to recover the pypi account, which could take months; or migrate to a different pypi deliverable name. Both are difficult and time-consuming. ACTION: TC to encourage PTLs to once again reach out to any pypi owners/maintainers who have not yet acted to attempt to cleanup their access. ## Revise the project inactive state process This was some specific, targeted discussion around if we needed to revise inactive policy to include the case which happened to Murano -- because our process dictates that inactive projects are only removed from the release prior to milestone-2, finding out Murano was inactive and removing it from the release after M-2 was a special case. This discussion centered primarily around whether any revision to the policy was needed. In the end, no specific action items were created. ## Leaderless projects Details on this item are in https://etherpad.opendev.org/p/2024.2-leaderless and have been sent to the list separately. # Allowing other languages for client facing tools (rust) Notes in the etherpad for this topic was minimal. Essentially, there is no restriction on what languages can be used -- the runtimes document is meant to be specific information about those languages, not restricting other languages that could be used. There were two primary concerns around this item, JayF asked why we were looking into new rust-based tooling instead of integrating with the existing openstack-rust community. Other contributors expressed concerns around being able to understand or debug code in different languages. No action items came out of this discussion. # OpenAPI in OpenStack There was minimal discussion around this item. There appeared to be clear consensus with moving forward with OpenAPI on a project-by-project basis, with any efforts towards creating a goal being pushed until after there is more adoption. # Deprecate all native clients in favor of OpenStackSDK/openstackclient Again, minimal discussion on this item. There seemed to be little objection to just getting it done, but there were no immediate volunteers to do the work. See the etherpad for some specific examples of gaps that need to be resolved. Thank you all for participating in the TC PTG. Please do not let these items fall by the wayside; if you had an idea that didn't reach consensus in the PTG, one possible way to move it forward is a concrete proposal where folks can discuss -- either to the governance repo, or to the mailing list. -- Jay Faulkner OpenStack TC

1 0

[kolla-ansible][Antelope][Heat] Heat UpdateWaitConditionHandle Problems
by Devin Patterson 19 Apr '24

19 Apr '24

Hello, We are attempting to use Heat stacks with wait conditions with nested stacks. We have a test stack similar to this one https://github.com/openstack/heat-templates/tree/master/hot/autohealing where we are scaling a template. We are testing with the proposed fix in the link below. https://review.opendev.org/c/openstack/heat/+/915209/2/heat/engine/resource… Problem being faced is when updating the heat stack, all of the resources are replaced. This is not ideal as we would like the server resources to update with rolling updates. The rolling updates do take place as described in the auto-scaling group, but only after all resources are replaced. Seems the _needs_update call for the wait handle is triggering a replaced for all resources in the nested stack which is not the desired outcome. 2024-04-19 18:12:03Z [stack_1-j190L]: UPDATE_IN_PROGRESS Stack UPDATE started 2024-04-19 18:12:03Z [stack_1-j190L.asg]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:04Z [stack_1-j190L.asg]: UPDATE_IN_PROGRESS Stack UPDATE started 2024-04-19 18:12:04Z [stack_1-j190L.asg.4s52xugnxsrl]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:05Z [stack_1-j190L.asg.fpe6jz2dccpg]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:05Z [stack_1-j190L.asg.4s52xugnxsrl]: UPDATE_IN_PROGRESS Stack UPDATE started 2024-04-19 18:12:06Z [stack_1-j190L.asg.4s52xugnxsrl.wait_handle]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:06Z [stack_1-j190L.asg.o7dhu7neswv4]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:06Z [stack_1-j190L.asg.fpe6jz2dccpg]: UPDATE_IN_PROGRESS Stack UPDATE started 2024-04-19 18:12:07Z [stack_1-j190L.asg.4s52xugnxsrl.wait_handle]: CREATE_COMPLETE state changed 2024-04-19 18:12:07Z [stack_1-j190L.asg.fpe6jz2dccpg.wait_handle]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:07Z [stack_1-j190L.asg.4s52xugnxsrl.instance1]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:07Z [stack_1-j190L.asg.4s52xugnxsrl.wait_condition]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:07Z [stack_1-j190L.asg.4s52xugnxsrl.wait_condition]: UPDATE_COMPLETE The Resource wait_condition requires replacement. 2024-04-19 18:12:07Z [stack_1-j190L.asg.4s52xugnxsrl.wait_condition]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:07Z [stack_1-j190L.asg.o7dhu7neswv4]: UPDATE_IN_PROGRESS Stack UPDATE started 2024-04-19 18:12:07Z [stack_1-j190L.asg.fpe6jz2dccpg.wait_handle]: CREATE_COMPLETE state changed 2024-04-19 18:12:08Z [stack_1-j190L.asg.o7dhu7neswv4.wait_handle]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:08Z [stack_1-j190L.asg.fpe6jz2dccpg.instance1]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:08Z [stack_1-j190L.asg.fpe6jz2dccpg.wait_condition]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:08Z [stack_1-j190L.asg.fpe6jz2dccpg.wait_condition]: UPDATE_COMPLETE The Resource wait_condition requires replacement. 2024-04-19 18:12:08Z [stack_1-j190L.asg.fpe6jz2dccpg.wait_condition]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:08Z [stack_1-j190L.asg.o7dhu7neswv4.wait_handle]: CREATE_COMPLETE state changed 2024-04-19 18:12:09Z [stack_1-j190L.asg.o7dhu7neswv4.instance1]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:09Z [stack_1-j190L.asg.o7dhu7neswv4.wait_condition]: UPDATE_IN_PROGRESS state changed 2024-04-19 18:12:09Z [stack_1-j190L.asg.o7dhu7neswv4.wait_condition]: UPDATE_COMPLETE The Resource wait_condition requires replacement. 2024-04-19 18:12:09Z [stack_1-j190L.asg.o7dhu7neswv4.wait_condition]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:11Z [stack_1-j190L.asg.4s52xugnxsrl.instance1]: UPDATE_COMPLETE The Resource Server "instance1" [27bbecab-bca3-48f0-9b1b-75d240d741a4] Stack "stack_1-j190L-asg-fmddtpch4ekg-4s52xugnxsrl-n56r3c5cxvwm" [8c4b9dcb-d4e3-4ff5-a442-c2b4f6302c38] requires replacement. 2024-04-19 18:12:11Z [stack_1-j190L.asg.4s52xugnxsrl.instance1]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:11Z [stack_1-j190L.asg.fpe6jz2dccpg.instance1]: UPDATE_COMPLETE The Resource Server "instance1" [58cffc7c-9278-43eb-bc36-fbbf9537a580] Stack "stack_1-j190L-asg-fmddtpch4ekg-fpe6jz2dccpg-7ipvlcpw237k" [1fd36182-6ac6-4f7b-aa3c-8ddaac1d933c] requires replacement. 2024-04-19 18:12:11Z [stack_1-j190L.asg.fpe6jz2dccpg.instance1]: CREATE_IN_PROGRESS state changed 2024-04-19 18:12:12Z [stack_1-j190L.asg.o7dhu7neswv4.instance1]: UPDATE_COMPLETE The Resource Server "instance1" [61dbda35-f20e-4018-85ad-66ba2e03c506] Stack "stack_1-j190L-asg-fmddtpch4ekg-o7dhu7neswv4-bqwgkjmslxkq" [c4b73bac-fdc8-4a78-8c18-1963054bc2fe] requires replacement. Regards, Devin

1 0

[openstack-ansible][magnum-cluster-api]
by pahrialtkj＠gmail.com 19 Apr '24

19 Apr '24

I tried magnum-cluster-api on openstack ansible with Antelope version, I have finished creating the k8s control plane containers and then Run the magnum-cluster-api deployment, but at the time of running it an error was found, where the role openstack.osa.lxc_container_setup is not available, anyone knows how to solve it? root@aio1:/opt/openstack-ansible# openstack-ansible osa_ops.mcapi_vexxhost.k8s_install Variable files: "-e @/etc/openstack_deploy/user_secrets.yml -e @/etc/openstack_deploy/user_variables.yml -e @/etc/openstack_deploy/user_variables_octavia.yml " [WARNING]: Unable to parse /etc/openstack_deploy/inventory.ini as an inventory source [WARNING]: running playbook inside collection osa_ops.mcapi_vexxhost ERROR! the role 'openstack.osa.lxc_container_setup' was not found in osa_ops.mcapi_vexxhost:ansible.legacy:/etc/ansible/ansible_collections/osa_ops/mcapi_vexxhost/playbooks/roles:/etc/ansible/roles:/etc/ansible/roles/ceph-ansible/roles:/etc/ansible/ansible_collections/osa_ops/mcapi_vexxhost/playbooks The error appears to be in '/etc/ansible/ansible_collections/osa_ops/mcapi_vexxhost/playbooks/k8s_install.yml': line 36, column 15, but may be elsewhere in the file depending on the exact syntax problem. The offending line appears to be: - import_role: name: openstack.osa.lxc_container_setup ^ here EXIT NOTICE [Playbook execution failure] ************************************** ===============================================================================

2 5