- openstack-discuss - lists.openstack.org

Antelope Barbican/Keystone: failure to delete secrets (Keystone TrustNotFound)
by Michel Jouvin 05 Jun '24

05 Jun '24

Hi, I've been validating Yoga -> Antelope on our preprod cloud. It was working successfully but "suddenly" (i.e. without a kwown/identified change!) when deleting Magnum clusters, the deletion of Barbican secrets (after stack deletion) fail. In the Magnum log, I find the following error: ---- Failed to delete trust: keystoneauth1.exceptions.http.Unauthorized: The request you have made requires authentication. (HTTP 401) (Request-ID: req-a4400250-e8d5-46ec-a28a-dcb97a682512) (traceback skipped for readability) 2024-05-29 10:43:09.750 2078 INFO magnum.common.cert_manager.barbican_cert_manager [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] Recursively deleting certificate container https://os-77023.lal.in2p3.fr:9311/v1/containers/edd8487a-6cf1-4f4c-9f53-17… from Barbican. 2024-05-29 10:43:09.750 2078 INFO barbicanclient.base [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] Calculated Containers uuid ref: containers/edd8487a-6cf1-4f4c-9f53-173dfae7a8b2 2024-05-29 10:43:09.760 2078 ERROR barbicanclient.client [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] 4xx Client error: Not Found: Secrets container not found. 2024-05-29 10:43:09.761 2078 ERROR magnum.common.cert_manager.barbican_cert_manager [None req-94df560b-d6da-4fba-83a9-34e0b08c64c8 - - - - - -] Error recursively deleting certificate container https://os-77023.lal.in2p3.fr:9311/v1/containers/edd8487a-6cf1-4f4c-9f53-17…: barbicanclient.exceptions.HTTPClientError: Not Found: Secrets container not found. ---- In Keystone log, I find with the same timestamp the following error that seems related to the one above: ---- 2024-05-29 10:43:09.690 1713 WARNING keystone.api._shared.authentication [None req-a4400250-e8d5-46ec-a28a-dcb97a682512 - - - - - -] Could not find trust: cf4b1b6082594025a1ff9e48df383566.: keystone.exception.TrustNotFound: Could not find trust: cf4b1b6082594025a1ff9e48df383566. 2024-05-29 10:43:09.693 1713 WARNING keystone.server.flask.application [None req-a4400250-e8d5-46ec-a28a-dcb97a682512 - - - - - -] Authorization failed. The request you have made requires authentication. from 134.158.77.23: keystone.exception.Unauthorized: The request you have made requires authentication. 2024-05-29 10:43:09.746 1723 ERROR keystone.server.flask.application [None req-8c76f304-c2a3-418c-8372-02c706ed356e 44d8cb3192b3485790ababe29991491a - 5b67a8641dd041089e018ceea9884eac - 5b67a8641dd041089e018ceea9884eac -] Could not find user: 527ec32509d849e8a49b41c3dc4c4c32.: keystone.exception.UserNotFound: Could not find user: 527ec32509d849e8a49b41c3dc4c4c32. ---- It is not clear for me if the ERROR is releated to the 2 warnings as the request is not the same but it seems to happen every time this way. In Barbican logs, there is not much things related except a 401 status for the request corresponding ot the TrustNotFound. I'm looking for advices on how to troubleshoot this. In particular, when the trust should be created and whether I should find somewhere a matching error at the time of the trust creation. It seems the error is only affecting deletion but I may be wrong. for the record, when starting Magnum there is an error complaining about /etc/magnum/keystone_auth_default_policy.json being not found but ignored it so far. And in Keystone an error seems to occur at Magnum cluster creation time: --- 2024-05-29 10:17:17.530 1711 WARNING py.warnings [None req-f484cc35-3e54-4d7f-8688-e6442c90344f a22b1199c74f4a819a49f3ac037a5e44 1dd67113c8bb4ef0a31dfca53335ffbd - - defaul t default] /usr/lib/python3.9/site-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is re quired ---- Thanks in advance for any hint. Best regards, Michel

2 3

Antelope Magnum: creating a new cluster makes the/some others unhealthy
by Michel Jouvin 05 Jun '24

05 Jun '24

Hi, I have been playing with Magnum (Antelope), creating a lot of clusters with the same template and varying the number of masters and/or nodes, having ~10 clusters started in the same project. I observed recently that cluster configuration that used to work well were not working any more (CREATE_FAILED during the master deployment in general). I have not found any evidence of the cause digging in various logs but I observed that with the current list of active clusters, if add a new one (even with a configuration as minimal as 1 master and 1 node), not only its creation fails but also the other (or at least) running clusters become unhealthy. If I delete the failed cluster, the other clusters return to the healthy state. It seems very reproducible (I did it more than 10 times) but I still don't see any message in the logs that could help to identify the cause. I've the feeling (but may be wrong) that it is related to either an insufficient project quota or an insufficient limit on one of the OpenStack service. Any idea on a possible cause or any advice on where to look for some information? Thanks in advance. Michel

2 6

Antelope: weird problem with member role after upgrade (Nova and Glance)
by Michel Jouvin 05 Jun '24

05 Jun '24

Hi, I just upgraded Nova/Cinder/Glance of our production cloud from Yoga to Antelope (after upgrading Keystone yesterday) and since the upgrade, users who are not admin cannot do anything basically, despite we changed nothing to service configuration or user's roles since Yoga. We enabled scoped tokens a while ago (several months). For (bad) historical reasons, the role "member" was called "users" but it had no impact (I was surprised), despite we are using standard policies. We thought it may be a consequence of this and we renamed the role back to "member". It was not enough to fix the problem, even after restart memcached on all servers just in case. We thought that there was may be some caching done somewhere with the old role name and modified slightly the policy rules defining what is a member or read with: "project_member_api": "(role:member or role:users) and project_id:%(project_id)s" "project_reader_api": "(role:reader or role:users) and project_id:%(project_id)s" It first works but the change was reverted by mistake and now it doesn't work anymore. I am really completely stuck, without any clue about what happen and on how to troubleshoot it. I googled a bit but was not able to find something looking similar... Any help would be greatly appreciated. Best regards, Michel -- Michel

3 2

[ops] help with manila
by Shubham Kumar Yadav 05 Jun '24

05 Jun '24

hello, I'm trying to install manila in my openstack cloud but having problem with "*Authorizing the driver to communicate with Ceph*" *1 -> *i have created a configmap with these confs and mounted it on manila-share pod [client.manila]client mount uid = 0client mount gid = 0log file = /opt/stack/logs/ceph-client.manila.logadmin socket = /opt/stack/status/stack/ceph-$name.$pid.asokkeyring = /etc/ceph/manila.keyring *2 -> *then i have created a secret with client.manila keyring and attached it to the manila-share pod at keyring = /etc/ceph/manila.keyring *3 -> *inside manila-share pod i see the logs 2024-06-03 09:00:53.682 6 INFO manila.share.drivers.cephfs.driver [None req-22cd7b29-a383-44cd-8cb8-1c639a3034dc - - - - - -] [CEPHFSNATIVE1] Ceph client found, connecting... 2024-06-03 09:00:53.713 6 ERROR manila.share.manager [None req-22cd7b29-a383-44cd-8cb8-1c639a3034dc - - - - - -] Error encountered during initialization of driver CephFSDriver@lab-11053-26006-ceph-2@cephfsnative1: manila.exception.ShareBackendException: [%(be)s] Ceph client failed to connect. 2024-06-03 09:00:53.713 6 ERROR manila.share.manager Traceback (most recent call last): 2024-06-03 09:00:53.713 6 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/drivers/cephfs/driver.py", line 421, in rados_client 2024-06-03 09:00:53.713 6 ERROR manila.share.manager self._rados_client.connect() 2024-06-03 09:00:53.713 6 ERROR manila.share.manager File "rados.pyx", line 690, in rados.Rados.connect 2024-06-03 09:00:53.713 6 ERROR manila.share.manager rados.IOError: [errno 5] RADOS I/O error (error connecting to the cluster) 2024-06-03 09:00:53.713 6 ERROR manila.share.manager 2024-06-03 09:00:53.713 6 ERROR manila.share.manager During handling of the above exception, another exception occurred: 2024-06-03 09:00:53.713 6 ERROR manila.share.manager 2024-06-03 09:00:53.713 6 ERROR manila.share.manager Traceback (most recent call last): 2024-06-03 09:00:53.713 6 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/manager.py", line 371, in _driver_setup 2024-06-03 09:00:53.713 6 ERROR manila.share.manager self.driver.do_setup(ctxt) 2024-06-03 09:00:53.713 6 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/drivers/cephfs/driver.py", line 274, in do_setup 2024-06-03 09:00:53.713 6 ERROR manila.share.manager rados_client=self.rados_client, 2024-06-03 09:00:53.713 6 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/drivers/cephfs/driver.py", line 424, in rados_client 2024-06-03 09:00:53.713 6 ERROR manila.share.manager raise exception.ShareBackendException( 2024-06-03 09:00:53.713 6 ERROR manila.share.manager manila.exception.ShareBackendException: [%(be)s] Ceph client failed to connect. 2024-06-03 09:00:53.713 6 ERROR manila.share.manager 2024-06-03 09:00:53.714 6 DEBUG manila.utils [None req-22cd7b29-a383-44cd-8cb8-1c639a3034dc - - - - - -] Finished call to 'manila.share.manager.ShareManager.init_host.<locals>._driver_setup' after 2222.651(s), this was the 12th time calling it. log_it /var/lib/openstack/lib/python3.10/site-packages/tenacity/after.py:44 2024-06-03 09:00:53.714 6 DEBUG manila.utils [None req-22cd7b29-a383-44cd-8cb8-1c639a3034dc - - - - - -] Retrying manila.share.manager.ShareManager.init_host.<locals>._driver_setup in 600.0 seconds as it raised ShareBackendException: [%(be)s] Ceph client failed to connect.. log_it /var/lib/openstack/lib/python3.10/site-packages/tenacity/before_sleep.py:65 *4 -> *and in manila-api i see logs Mon Jun 3 09:41:18 2024 - *** uWSGI listen queue of socket "0.0.0.0:8786" (fd: 3) full !!! (101/100) *** Mon Jun 3 09:41:19 2024 - *** uWSGI listen queue of socket "0.0.0.0:8786" (fd: 3) full !!! (101/100) *** even when i have set manila: DEFAULT: wsgi_default_pool_size: 1000 Hope to hear from you soon with a solution Thank you!

2 1

Is it possible to rename a role?
by Michel Jouvin 05 Jun '24

05 Jun '24

Hi, I just upgraded Nova/Cinder/Glance of our production cloud from Yoga to Antelope (after upgrading Keystone yesterday) and since the upgrade, users who are not admin cannot do anything basically with Glance and Nova, despite we changed nothing to service configuration or user's roles since Yoga. We enabled scoped tokens a while ago (several months). After some research, we found that it was linked to the fact that for (bad) historical reasons, the person who created the cloud 10 years ago renamed the role "member" to "users". It has been working without bad impact up to know. I was surprised of this absence of impact when we switched to scope tokens and enforced new rules. I realize that it is because several (all?) services up to Yoga was still supporting the old rules in addition to the new ones. On Cinder and Keystone for example, if using oslopolicy-policy-generator, it reports deprecated rules that are not defined in our configuration and I guess it is because old rules are not yet deleted on these services. Our first attempt to fix the problem has been to rename the users role (that every non-admin user has instead of the no-longer existing member role) back to its original name, member. Unfortunately it didn't solve the problem, even after restarting memcached everywhere (not sure it may have an impact) and the API services on Glance and Nova. We found a workaround modifying slightly the policy rules defining what is a member or reader with the following for Nova (and something similar for Glance, even if more rules have to be modified as there is no rule defining what is a member or reader): "project_member_api": "(role:member or role:users) and project_id:%(project_id)s" "project_reader_api": "(role:reader or role:users) and project_id:%(project_id)s" *My question is:* why renaming the role was not enough? The fact that "openstack role set --name" exists suggests that it should work, isn't it? We can probably leave with the workound we found but it would make the configuration simpler if we could fix the role name and keep the standard rules... Thanks in advance for any suggestion on how to do it or to troubleshoot the fact it doesn't work... Best regards, Michel

1 0

Is it possible to rename a role?
by Michel Jouvin 05 Jun '24

05 Jun '24

Hi, I just upgraded Nova/Cinder/Glance of our production cloud from Yoga to Antelope (after upgrading Keystone yesterday) and since the upgrade, users who are not admin cannot do anything basically with Glance and Nova, despite we changed nothing to service configuration or user's roles since Yoga. We enabled scoped tokens a while ago (several months). After some research, we found that it was linked to the fact that for (bad) historical reasons, the person who created the cloud 10 years ago renamed the role "member" to "users". It has been working without bad impact up to know. I was surprised of this absence of impact when we switched to scope tokens and enforced new rules. I realize that it is because several (all?) services up to Yoga was still supporting the old rules in addition to the new ones. On Cinder and Keystone for example, if using oslopolicy-policy-generator, it reports deprecated rules that are not defined in our configuration and I guess it is because old rules are not yet deleted on these services. Our first attempt to fix the problem has been to rename the users role (that every non-admin user has instead of the no-longer existing member role) back to its original name, member. Unfortunately it didn't solve the problem, even after restarting memcached everywhere (not sure it may have an impact) and the API services on Glance and Nova. We found a workaround modifying slightly the policy rules defining what is a member or reader with the following for Nova (and something similar for Glance, even if more rules have to be modified as there is no rule defining what is a member or reader): "project_member_api": "(role:member or role:users) and project_id:%(project_id)s" "project_reader_api": "(role:reader or role:users) and project_id:%(project_id)s" My question is: why renaming the role was not enough? The fact that "openstack role set --name" exists suggests that it should work, isn't it? We can probably leave with the workound we found but it would make the configuration simpler if we could fix the role name and keep the standard rules... Thanks in advance for any suggestion on how to do it or to troubleshoot the fact it doesn't work... Best regards, Michel -- Michel

1 0

Nova live migration
by Gk Gk 04 Jun '24

04 Jun '24

Hi, I see this in the latest nova configuration: This option is deprecated for removal since 15.0.0. Its value may be silently ignored in the future. Reason: live_migration_uri is deprecated for removal in favor of two other options that allow to change live migration scheme and target URI: live_migration_scheme and live_migration_inbound_addr respectively. So, if I want to use live_migration_scheme, what are its values ? If I use the live_migration_scheme option, can I remove the live_migration_uri option ? Thanks

3 9

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2024.2/R-17)
by Goutham Pacha Ravi 04 Jun '24

04 Jun '24

Hello Stackers, The 2024.2 release cycle is chugging along, and its last stop is 17 weeks away [1]. This week is the 2024.1 "cycle-trailing" release deadline for projects that follow this model. Typically, these are configuration management, packaging, and deployment tooling projects. In the past week, we undertook several governance activities that I'd like to summarize for you. We made a significant change to the "Distributed Leadership" model of project governance [2]. A project's governance will default to the PTL model during elections; if the project community would like to pursue the DPL model, they can do so by nominating liaisons for that release cycle. There's also going to be a TC liaison to each project following the DPL model. The TC liaison will follow the project's activity at regular intervals and interface with the TC. These changes will hopefully help signal project health concerns earlier. The TC also clarified that we'd adopt a "formal-vote" process for project team retirements. A formal vote would mean a third of the TC endorses a change through a "roll call" vote. We also cleaned up the emerging and inactive projects list [3]. Members of the TaCT SIG have started the arduous process of cleaning up PyPi, where we had a number of human maintainer accounts for OpenStack packages [4]. === Weekly Meeting === The OpenStack Technical Committee (TC) meeting on May 28, 2024, covered several topics [5]. Concerns were raised about the inactivity of projects like Freezer, Monasca, and Watcher. Discussions focused on adding TC members as temporary core reviewers to inactive projects to maintain them until new maintainers are found or to retire them if no interest is shown. A vote was held on whether the TC should be added to the Gerrit approving group for inactive projects, resulting in a split decision (4 yes, 3 no). For now, we decided to maintain the status quo. The revival of Watcher is being spearheaded by Dan Smith (dansmith) and aided by Ghanshyam Mann (gmann) and Slawek Kaplonski (slaweq). Our next weekly meeting will be held tomorrow (June 4, 2024) simultaneously on video (Zoom) and IRC. The meeting agenda and the details on how to join are in [6]. I hope that you're able to join us! === Governance Proposals === ==== Merged ==== - Remove retired project from Inactive project list | https://review.opendev.org/c/openstack/governance/+/920146 - Mark migrate-ci-jobs-to-ubuntu-jammy goal completed | https://review.opendev.org/c/openstack/governance/+/920143 - Add DPL model & liaison reset policy | https://review.opendev.org/c/openstack/governance/+/916833 - Clarify the project retirement vs project's repository retirement | https://review.opendev.org/c/openstack/governance/+/919984 - Simplifying house rule for project-updates | https://review.opendev.org/c/openstack/governance/+/920318 ==== Open for review/discussion ==== - Appoint ChunYang Wu as PTL of Trove | https://review.opendev.org/c/openstack/governance/+/920462 - Migrate eventlet usages to asyncio | https://review.opendev.org/c/openstack/governance/+/902585 - Update to include docs and miscellaneous repos for AC status | https://review.opendev.org/c/openstack/governance/+/915021 The OpenStack Technical Committee serves you! We're always willing to chat with you and would love feedback: === How to contact the TC === You can reach out to the TC in multiple ways: 1. Email: you can send the email with the tag [tc] on this email list. 2. Ping us using the 'tc-members' keyword on the #openstack-tc IRC channel. 3. Join us at our weekly meeting: The Technical Committee meets every week on Tuesdays at 1800 UTC [6]. Finally, if you got this far, I hope you found this update helpful. I'd appreciate it if you had any feedback regarding the format, content, frequency, or timing of this weekly email. Please feel free to email me off-list if you're more comfortable doing so. Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] https://releases.openstack.org/dalmatian/schedule.html [2] https://governance.openstack.org/tc/reference/distributed-project-leadershi… [3] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [4] https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup#L52 [5] https://meetings.opendev.org/meetings/tc/2024/tc.2024-05-28-18.00.log.html [6] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0

[all] SQLAlchemy minimum requirement
by Pierre Riteau 03 Jun '24

03 Jun '24

Hello everyone, Historically, many projects were using the following requirement for SQLAlchemy: SQLAlchemy!=1.1.5,!=1.1.6,!=1.1.7,!=1.1.8,>=1.0.10 These banned versions were present in global-requirements.txt until earlier this month [1]. The remaining projects with the above requirement must change it to fix their requirements-check job (ERROR: Requirement for package SQLAlchemy excludes a version not excluded in the global list). While fixing the requirement for blazar, I realised that OpenStack projects are using a wide range of minimum requirements. For example, networking-sfc had already bumped the minimum to SQLAlchemy 2.0 [2], although this is being reverted [3]. While all projects must be compatible with the upper constraint (currently 2.0.30), should we agree on a maximum version for the lower bound? The 2024.2 release targets Ubuntu Jammy, which includes python3-sqlalchemy 1.4.31, and Debian bookworm, which includes python3-sqlalchemy 1.4.46. Thanks, Pierre Riteau (priteau) [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://review.opendev.org/c/openstack/networking-sfc/+/920046 [3] https://review.opendev.org/c/openstack/networking-sfc/+/920868

2 1

The outbound unicast traffic of VMS with security group enabled is broadcast
by renwc＠chinatelecom.cn 31 May '24

31 May '24

hello everyone， Here is a description of a bug I encountered using openstack Security Group,If security groups are enabled for virtual nics on cloud hosts, The outgoing traffic of the VMS enabled in the security group should be sent in unicast mode, but it is eventually forwarded to all ports in the vlan on the ovs br-int bridge. Does the openstack open source community have a patch to fix this bug? Can you provide patch? The following is my debugging process in detail. The unicast packet sent by the remote virtual machine 172.83.160.63 to the local virtual machine 172.83.160.74 is passed to the br-ex->br-int-> VM, Because the security group is enabled on the VM NIC, The br-int cannot learn the entry whose port is patch-ex and mac address is vm2. Therefore those packets sent from vm1 to vm2 are broadcast in vlans because they fail to match mac address entries. 172.83.160.74 patch-int 172.83.160.63 vm1 --- ovs(br-int) ------ ovs(br-ex)--- tor(physical switch) ......vm2 patch-ex The following is the ovs kernel flow table. The mac address of vm1 is fa:16:3e:6c:f3:09,The mac address of vm2 is fa:16:3e:d2:14:b5, the action of the flow table is multiple ports. recirc_id(0x314123),in_port(8),ct_state(-new+est-rel-rpl),eth(src=fa:16:3e:d2:14:b5,dst=fa:16:3e:6c:f3:09),eth_type(0x0800),ipv4(proto=6,frag=no),tcp(dst=2048/0xf800), packets:60100, bytes:14257193, used:2.035s, flags:P., actions:push_vlan(vid=5,pcp=0),3,pop_vlan,push_vlan(vid=1732,pcp=0),1,pop_vlan,7,9,10,15,5,18 recirc_id(0x314123),in_port(8),ct_state(-new+est-rel+rpl-inv+trk),ct_zone(0x5),ct_mark(0),eth(src=fa:16:3e:d2:14:b5,dst=fa:16:3e:6c:f3:09),eth_type(0x0800),ipv4(frag=no), packets:1, bytes:54, used:2.037s, flags:., actions:push_vlan(vid=5,pcp=0),3,pop_vlan,push_vlan(vid=1732,pcp=0),1,pop_vlan,7,9,10,15,5,18 After careful debugging analysis, the openflow action in the flow table 82 added by neutron is the VM port instead of normal. As a result, the ovs br-int bridge cannot learn the mac entry of vm2. table=82, priority=77,ct_state=+new-est,tcp,reg5=0x9,tp_dst=0x4/0xfffc actions=ct(commit,zone=NXM_NX_REG6[0..15]),output:9 table=82, priority=77,ct_state=+est-rel-rpl,tcp,reg5=0x9,tp_dst=0x4/0xfffc actions=output:9 Looking forward to reply thanks. weichun ren

2 1

The outgoing unicast traffic from the VMS enabled in the security group is eventually forwarded to all ports.
by renwc＠chinatelecom.cn 31 May '24

31 May '24

Hello， Here is a description of a bug I encountered using openstack Security Group,If security groups are enabled for virtual nics on VMS, The outgoing traffic from the VMS enabled in the security group should be sent in unicast mode, but it is eventually forwarded to all ports in the vlan on the ovs br-int bridge. Does the openstack open source community have a patch to fix this bug? Can you provide patch fixed the issue? The following is my debugging process in detail. The unicast packet sent by the remote virtual machine 172.83.160.63 to the local virtual machine 172.83.160.74 is passed to the br-ex->br-int-> VM, Because the security group is enabled on the VM NIC, The br-int cannot learn the entry whose port is patch-ex and mac address is vm2. Therefore those packets sent from vm1 to vm2 are broadcast in vlans because they fail to match mac address entries. 172.83.160.74 patch-int 172.83.160.63 vm1 --- ovs(br-int) ------ ovs(br-ex)--- tor(physical switch) ......vm2 patch-ex The following is the ovs kernel flow table. The mac address of vm1 is fa:16:3e:6c:f3:09,The mac address of vm2 is fa:16:3e:d2:14:b5, the action of the flow table is multiple ports. recirc_id(0x314123),in_port(8),ct_state(-new+est-rel-rpl),eth(src=fa:16:3e:d2:14:b5,dst=fa:16:3e:6c:f3:09),eth_type(0x0800),ipv4(proto=6,frag=no),tcp(dst=2048/0xf800), packets:60100, bytes:14257193, used:2.035s, flags:P., actions:push_vlan(vid=5,pcp=0),3,pop_vlan,push_vlan(vid=1732,pcp=0),1,pop_vlan,7,9,10,15,5,18 recirc_id(0x314123),in_port(8),ct_state(-new+est-rel+rpl-inv+trk),ct_zone(0x5),ct_mark(0),eth(src=fa:16:3e:d2:14:b5,dst=fa:16:3e:6c:f3:09),eth_type(0x0800),ipv4(frag=no), packets:1, bytes:54, used:2.037s, flags:., actions:push_vlan(vid=5,pcp=0),3,pop_vlan,push_vlan(vid=1732,pcp=0),1,pop_vlan,7,9,10,15,5,18 After careful debugging analysis, the openflow action in the flow table 82 added by neutron is the VM port instead of normal. As a result, the ovs br-int bridge cannot learn the mac entry of vm2. table=82, priority=77,ct_state=+new-est,tcp,reg5=0x9,tp_dst=0x4/0xfffc actions=ct(commit,zone=NXM_NX_REG6[0..15]),output:9 table=82, priority=77,ct_state=+est-rel-rpl,tcp,reg5=0x9,tp_dst=0x4/0xfffc actions=output:9 Was the issue above fixed in the openstack ? Looking forward to reply thanks. Best Regards weichun ren

1 0

Re: [tc][all][senlin] Retiring the Senlin project
by Ghanshyam Mann 31 May '24

31 May '24

---- On Thu, 30 May 2024 11:43:00 -0700 Sang Tran Quoc wrote --- > > Dear Ghanshyam, > > Greeting! I’m Sang from Viet Nam and now I’m working at the FPT Smart Cloud (Silver member of OpenInfra) as the openstack team leader. We have teammate with about 10 members who maintains the senlin project on our production AutoScaling Feature (my presentation about senlin at last VietOpenInfra summithttps://www.youtube.com/watch?v=aYXDI4mwohQ&t=1s) It’s sad to hear about community retired this project from upstream so I would like to nominate our team to continue maintain this project and I will handle the PTL role for further roadmap, bugs fix and feature. > > Hopefully it’s not late, I’m look forward to hearing from you for further discuss about next chance for senlin project. ++ openstack-discuss ML Hi Sang, Thanks for reaching out and its good to hear about Senlin project usage and your interest in maintaining it. > Hopefully it’s not late, I’m look forward to hearing from you for further discuss about next chance for senlin project. Unfortunately, TC has already retired the Senlin project[1]. Since last cycle, we have been waiting for interested maintainers to step up. * No Leader in 2024.1 cycle ** https://etherpad.opendev.org/p/2024.1-leaderless * After TC discussion, Senlin has been marked as an 'inactive' project since '2024.1' cycle ** https://review.opendev.org/c/openstack/governance/+/902626 ** This was the time we were expecting someone to come forward and step up to maintain it. * No leader for Senlin in 2024.2 cycles also ** https://etherpad.opendev.org/p/2024.2-leaderless * Considering no leader for two cycles, no maintainer, and the process, TC initiated the retirement: ** https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… But this is not the end and Senlin can still re-become the official OpenStack project[2]. For that, you need to re-apply Senlin as a new project in OpenStack, please check the process: - https://governance.openstack.org/tc/reference/new-projects-requirements.html [1] - https://review.opendev.org/c/openstack/governance/+/919347 - https://review.opendev.org/q/topic:%22retire-senlin%22+ [2]https://governance.openstack.org/tc/reference/dropping-projects.html#re-b… -gmann > > > Thanks & Best Regards > > > Sang Tran Quoc (Mr.) > FPT Smart Cloud > FPT Tower, 10 Pham Van Bach Str., Cau Giay Dist., Ha Noi, Viet Nam > E: sangtq8(a)fpt.com > T: 0962949843 > W: https://fptcloud.com | https://fpt.ai > > > > > From:Ghanshyam Mann gmann(a)ghanshyammann.com> > Date: Saturday, 11 May 2024 at 09:06 > To: openstack-discuss openstack-discuss(a)lists.openstack.org> > Subject: Re: [tc][all][senlin] Retiring the Senlin project > ---- On Wed, 24 Apr 2024 13:39:33 -0700 Ghanshyam Mann wrote --- > > Hello everyone, > > > > As you might know, the Senlin project is not active and was marked 'inactive' > > since 2024.1 cycle[1]. In this cycle also, this project is leaderless[2] and no maintainer > > stepped up to maintain it. As discussed in the Leaderless project etherpad[2], the next > > step is to retire the project. > > > > I'm sending this email to propose the Senlin retirement and based on the response, > > I will proceed next. > > Seeing no response to this email or no volunteer to maintain this project, I have proposed > the retirement in governance: > > - https://review.opendev.org/c/openstack/governance/+/919347 > > -gmann > > > > > [1] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… > > [2] https://etherpad.opendev.org/p/2024.2-leaderless#L72 > > > > -gmann > > >

1 0

[neutron][all] OOM killer on coverage job
by Brian Haley 31 May '24

31 May '24

Hi, Neutron has been having issues with our coverage gate job triggering the OOM killer since last week [0], which I just confirmed by holding a node and looking in the logs. It started happening after the sqlalchemy 2.0 bump [1], but that just might be exposing the underlying issue. Running locally I can see via /proc/meminfo that memory is getting consumed: MemTotal: 8123628 kB MemFree: 1108404 kB And via ps it's the coverage processes doing it: PID %MEM RSS PPID TIME NLWP WCHAN COMMAND 4315 30.9 2516348 4314 01:29:07 1 - /opt/stack/neutron/.tox/cover/bin/python /opt/stack/neutron/.tox/cover/bin/coverage run --source neutron --parallel-mode -m stestr.subunit_runner.run discover -t ./ ./neutron/tests/unit --load-list /tmp/tmp0rhqfwhz 4313 30.0 2437500 4312 01:28:50 1 - /opt/stack/neutron/.tox/cover/bin/python /opt/stack/neutron/.tox/cover/bin/coverage run --source neutron --parallel-mode -m stestr.subunit_runner.run discover -t ./ ./neutron/tests/unit --load-list /tmp/tmpfzmqyuub (and the test hasn't even finished yet) Only workaround seems to be reducing concurrency [2]. Have any other projects seen anything similar? (and sorry for the html email) -Brian [0] https://bugs.launchpad.net/neutron/+bug/2065821 [1] https://review.opendev.org/c/openstack/requirements/+/879743 [2] https://review.opendev.org/c/openstack/neutron/+/920766

4 6

A bug about openstack security groups
by renwc＠chinatelecom.cn 30 May '24

30 May '24

hello everyone， Here is a description of a bug I encountered using openstack Security Group,If security groups are enabled for virtual nics on cloud hosts, The outgoing traffic of the VMS enabled in the security group should be sent in unicast mode, but it is eventually forwarded to all ports in the vlan on the ovs br-int bridge. Does the openstack open source community have a patch to fix this bug? Can you provide patch? The following is my debugging process in detail. The unicast packet sent by the remote virtual machine 172.83.160.63 to the local virtual machine 172.83.160.74 is passed to the br-ex->br-int-> VM, Because the security group is enabled on the VM NIC, The br-int cannot learn the entry whose port is patch-ex and mac address is vm2. Therefore those packets sent from vm1 to vm2 are broadcast in vlans because they fail to match mac address entries. 172.83.160.74 patch-int 172.83.160.63 vm1 --- ovs(br-int) ------ ovs(br-ex)--- tor(physical switch) ......vm2 patch-ex The following is the ovs kernel flow table. The mac address of vm1 is fa:16:3e:6c:f3:09,The mac address of vm2 is fa:16:3e:d2:14:b5, the action of the flow table is multiple ports. recirc_id(0x314123),in_port(8),ct_state(-new+est-rel-rpl),eth(src=fa:16:3e:d2:14:b5,dst=fa:16:3e:6c:f3:09),eth_type(0x0800),ipv4(proto=6,frag=no),tcp(dst=2048/0xf800), packets:60100, bytes:14257193, used:2.035s, flags:P., actions:push_vlan(vid=5,pcp=0),3,pop_vlan,push_vlan(vid=1732,pcp=0),1,pop_vlan,7,9,10,15,5,18 recirc_id(0x314123),in_port(8),ct_state(-new+est-rel+rpl-inv+trk),ct_zone(0x5),ct_mark(0),eth(src=fa:16:3e:d2:14:b5,dst=fa:16:3e:6c:f3:09),eth_type(0x0800),ipv4(frag=no), packets:1, bytes:54, used:2.037s, flags:., actions:push_vlan(vid=5,pcp=0),3,pop_vlan,push_vlan(vid=1732,pcp=0),1,pop_vlan,7,9,10,15,5,18 After careful debugging analysis, the openflow action in the flow table 82 added by neutron is the VM port instead of normal. As a result, the ovs br-int bridge cannot learn the mac entry of vm2. table=82, priority=77,ct_state=+new-est,tcp,reg5=0x9,tp_dst=0x4/0xfffc actions=ct(commit,zone=NXM_NX_REG6[0..15]),output:9 table=82, priority=77,ct_state=+est-rel-rpl,tcp,reg5=0x9,tp_dst=0x4/0xfffc actions=output:9 Looking forward to reply thanks. 任维春智能计算产品部/智能云网业务部邮箱：renwc(a)chinatelecom.cn 地址：北京市西城区西直门内大街118-1 中电信数智科技有限公司

1 0

[Diskimage-builder]Supported architectures for creating OS images in Diskimage-builder
by Junya Noguchi (Fujitsu) 29 May '24

29 May '24

To whom it may concern I am investigating which Linux distributions and versions for aarch64 are supported by Diskimage-builder. I found that many newer versions of Linux distributions are supported*, but there is no information about supported architectures in the commit messages. * References: Ubuntu: https://review.opendev.org/c/openstack/project-config/+/916050 RHEL: https://review.opendev.org/c/openstack/diskimage-builder/+/877347 SLES: https://review.opendev.org/c/openstack/diskimage-builder/+/619186 SUSE: https://review.opendev.org/c/openstack/diskimage-builder/+/908421 CentOS: https://review.opendev.org/c/openstack/tripleo-quickstart/+/843102 Does Diskimage-builder support building aarch64 images for those distributions? If not, a list of supported distros for aarch64, would be helpful. Best regards, Junya Noguchi.

2 4

[kolla] cancelling weekly meeting today (29th May 2024)
by Michał Nasiadka 29 May '24

29 May '24

Hello Koalas, We’re cancelling the weekly meeting today (29th May 2024) due to no show of majority key members. Let’s meet next week! Thanks for understanding, Michal

1 0

overcloud deploy fails in step 1 | tripleo | wallaby
by Swogat Pradhan 29 May '24

29 May '24

Hi, I am trying to run a stack update and am executing openstack overcloud deploy, but after reaching step 1 and prefetching the container images the deployment fails with following error: s 2024-05-16 19:58:54.138764 | 525400ff-fd40-2d4d-b69c-000000010c98 | OK | Pre-fetch all the containers | overcloud-controller-1 | item=172.25.201.68:8787/tripleomaster/openstack-cinder-backup:latest 2024-05-16 19:58:54.143338 | 525400ff-fd40-2d4d-b69c-000000010c98 | TIMING | Pre-fetch all the containers | overcloud-controller-1 | 0:23:18.464385 | 43.62s An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'config' s/_base.py\", line 384, in __get_result\n raise self._exception\n File \"/usr/lib64/python3.6/concurrent/futures/thread.py\", line 56, in run\n result = self.fn(*self.args, **self.kwargs)\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 332, in run_container\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 318, in manage_container\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1659, in execute\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1572, in make_started\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1351, in different\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 692, in __init__\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 699, in defaultize\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 661, in __init__\nKeyError: 'config'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee st2024-05-16 19:58:55.487682 | 525400ff-fd40-2d4d-b69c-000000010b27 | FATAL | Manage Puppet containers (generate config) for step 1 | overcloud-novacompute-3 | error={"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"<stdin>\", line 102, in <module>\n File \"<stdin>\", line 94, in _ansiballz_main\n File \"<stdin>\", line 40, in invoke_module\n File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n mod_name, mod_spec, pkg_name, script_name)\n File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 391, in <module>\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 387, in main\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 132, in __init__\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 370, in run\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 337, in check_failures\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 586, in result_iterator\n yield fs.pop().result()\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 425, in result\n return self.__get_result()\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 384, in __get_result\n raise self._exception\n File \"/usr/lib64/python3.6/concurrent/futures/thread.py\", line 56, in run\n result = self.fn(*self.args, **self.kwargs)\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 332, in run_container\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 318, in manage_container\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1659, in execute\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1572, in make_started\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1351, in different\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 692, in __init__\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 699, in defaultize\n File \"/tmp/ansible_tripleo_container_manage_payload_sg7woby1/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 661, in __init__\nKeyError: 'config'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1} 2024-05-16 19:58:55.488510 | 525400ff-fd40-2d4d-b69c-000000010b27 | TIMING | Manage Puppet containers (generate config) for step 1 | overcloud-novacompute-3 | 0:23:19.809572 | 82.91s 2024-05-16 19:58:55.859707 | 525400ff-fd40-2d4d-b69c-000000010c98 | OK | Pre-fetch all the containers | overcloud-controller-1 | item=172.25.201.68:8787/tripleomaster/openstack-cinder-volume:latest 2024-05-16 19:58:55.864370 | 525400ff-fd40-2d4d-b69c-000000010c98 | TIMING | Pre-fetch all the containers | overcloud-controller-1 | 0:23:20.185418 | 45.34s 2024-05-16 19:58:57.659959 | 525400ff-fd40-2d4d-b69c-000000010c98 | OK | Pre-fetch all the containers | overcloud-controller-1 | item=172.25.201.68:8787/tripleomaster/openstack-iscsid:latest 2024-05-16 19:58:57.664429 | 525400ff-fd40-2d4d-b69c-000000010c98 | TIMING | Pre-fetch all the containers | overcloud-controller-1 | 0:23:21.985477 | 47.14s 2024-05-16 19:59:00.211286 | 525400ff-fd40-2d4d-b69c-000000010c98 | OK | Pre-fetch all the containers | overcloud-controller-1 | item=172.25.201.68:8787/tripleomaster/openstack-manila-share:latest 2024-05-16 19:59:00.216236 | 525400ff-fd40-2d4d-b69c-000000010c98 | TIMING | Pre-fetch all the containers | overcloud-controller-1 | 0:23:24.537284 | 49.69s 2024-05-16 19:59:02.113037 | 525400ff-fd40-2d4d-b69c-000000010c98 | OK | Pre-fetch all the containers | overcloud-controller-1 | item=172.25.201.68:8787/tripleomaster/openstack-aodh-evaluator:latest 2024-05-16 19:59:02.117498 | 525400ff-fd40-2d4d-b69c-000000010c98 | TIMING | Pre-fetch all the containers | overcloud-controller-1 | 0:23:26.438544 | 51.59s 2024-05-16 19:59:03.853383 | 525400ff-fd40-2d4d-b69c-000000010c98 | OK | Pre-fetch all the containers | overcloud-controller-1 | item=172.25.201.68:8787/tripleomaster/openstack-aodh-listener:latest 2024-05-16 19:59:03.858232 | 525400ff-fd40-2d4d-b69c-000000010c98 | TIMING | Pre-fetch all the containers | overcloud-controller-1 | 0:23:28.179287 | 53.33s 2024-05-16 19:59:05.665169 | 525400ff-fd40-2d4d-b69c-000000010c98 | OK | Pre-fetch all the containers | overcloud-controller-1 | item=172.25.201.68:8787/tripleomaster/openstack-aodh-notifier:latest 2024-05-16 19:59:05.670233 | 525400ff-fd40-2d4d-b69c-000000010c98 | TIMING | Pre-fetch all the containers | overcloud-controller-1 | 0:23:29.991278 | 55.14s An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'config' /futures/_base.py\", line 384, in __get_result\n raise self._exception\n File \"/usr/lib64/python3.6/concurrent/futures/thread.py\", line 56, in run\n result = self.fn(*self.args, **self.kwargs)\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 332, in run_container\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 318, in manage_container\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1659, in execute\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1572, in make_started\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1351, in different\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 692, in __init__\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 699, in defaultize\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 661, in __init__\nKeyError: 'config'\n", "module_stdout": "", "msg": "MODULE FAILURE\2024-05-16 19:59:06.648301 | 525400ff-fd40-2d4d-b69c-000000010fc3 | FATAL | Manage Puppet containers (generate config) for step 1 | overcloud-controller-no-ceph-3 | error={"changed": false, "module_stderr": "Traceback (most recent call last):\n File \"<stdin>\", line 102, in <module>\n File \"<stdin>\", line 94, in _ansiballz_main\n File \"<stdin>\", line 40, in invoke_module\n File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n mod_name, mod_spec, pkg_name, script_name)\n File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 391, in <module>\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 387, in main\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 132, in __init__\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 370, in run\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 337, in check_failures\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 586, in result_iterator\n yield fs.pop().result()\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 425, in result\n return self.__get_result()\n File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 384, in __get_result\n raise self._exception\n File \"/usr/lib64/python3.6/concurrent/futures/thread.py\", line 56, in run\n result = self.fn(*self.args, **self.kwargs)\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 332, in run_container\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible/modules/tripleo_container_manage.py\", line 318, in manage_container\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1659, in execute\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1572, in make_started\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 1351, in different\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 692, in __init__\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 699, in defaultize\n File \"/tmp/ansible_tripleo_container_manage_payload_wnnokrwc/ansible_tripleo_container_manage_payload.zip/ansible_collections/containers/podman/plugins/module_utils/podman/podman_container_lib.py\", line 661, in __init__\nKeyError: 'config'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1} I do not have any idea why this is caused and how to fix it. Can someone please help? With regards, Swogat Pradhan

4 9

OpenInfra Summit Asia'24 CFP closes tomorrow (May 29)!
by Jeremy Stanley 29 May '24

29 May '24

I'll refer you back to my reminder from two weeks ago for details: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… The Summit CFP is still accepting talk proposals through tomorrow, May 29, 2024. If you're interested in presenting in Korea this September, the time to submit your abstract is now! https://openinfrafoundation.formstack.com/forms/openinfra_asia_summit_2024 As always, reply or find me (fungi) in the #openstack channel on the OFTC IRC network if you have questions. -- Jeremy Stanley

1 0

DevStack Setup
by Lohit S 28 May '24

28 May '24

Hi Team, I had tried to execute the ./stack.sh in devstack repo But I got the following error, How to fix this issue. +lib/neutron_plugins/ovn_agent:ovn_sanity_check:358 die 358 'The q-agt/neutron-agt service must be disabled with OVN.' +functions-common:die:285 local exitcode=0 +functions-common:die:286 set +o xtrace [Call Trace] ./stack.sh:979:stack_install_service /home/lohit22/opensource/openstack/devstack/Org/devstack/lib/stack:32:install_neutron /home/lohit22/opensource/openstack/devstack/Org/devstack/lib/neutron:551:install_ovn /home/lohit22/opensource/openstack/devstack/Org/devstack/lib/neutron_plugins/ovn_agent:373:ovn_sanity_check /home/lohit22/opensource/openstack/devstack/Org/devstack/lib/neutron_plugins/ovn_agent:358:die [ERROR] /home/lohit22/opensource/openstack/devstack/Org/devstack/lib/neutron_plugins/ovn_agent:358 The q-agt/neutron-agt service must be disabled with OVN. Error on exit World dumping... see /opt/stack/logs/worlddump-2024-05-26-053930.txt for details

3 7

[neutron] Bug deputy report for week of May 20th
by Brian Haley 28 May '24

28 May '24

Hi, I was Neutron bug deputy last week. Below is a short summary about the reported bugs. -Brian Critical bugs ------------- * None High bugs --------- * https://bugs.launchpad.net/neutron/+bug/2066221 - [dalmatian] skip level grenade jobs fails at keystone-manage db_sync - https://review.opendev.org/c/openstack/neutron/+/920075 * https://bugs.launchpad.net/neutron/+bug/2066263 - functional tests broken with ovsdbapp 2.7.1 - https://review.opendev.org/c/openstack/neutron/+/920101 * https://bugs.launchpad.net/neutron/+bug/2066369 - Validation of the auto allocated topology for member/reader user don't works with new S-RBAC policies - https://review.opendev.org/c/openstack/neutron/+/920174 - https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/920177 * https://bugs.launchpad.net/python-neutronclient/+bug/2067039 - openstack bgp [dragent|speaker] list filter commands dont return - Possibly related to the change to use OpenstackSDK code in https://review.opendev.org/c/openstack/python-neutronclient/+/868321 - Asked to make sure all libraries were updated - Might need further investigation by Lajos Medium bugs ----------- * https://bugs.launchpad.net/neutron/+bug/2066989 - Virtual function is being attached to port regardless of the exclude_devices configuration - Asked for more information * https://bugs.launchpad.net/neutron/+bug/2067183 - Properly Use Network's DNS Domain in Assignment and DHCP - https://review.opendev.org/c/openstack/neutron/+/920459 - I'd like to discuss at Neutron meeting since we've broken things like this in the past Low bugs -------- * None Misc bugs --------- * None Wishlist bugs ------------- * https://bugs.launchpad.net/nova/+bug/2065790 - Nova fails to live migrate instance with dash separated MAC - Request for Neutron to always use :" in MACs, even if API given "-" - Will have to discuss at weekly meeting, I'm inclined to document around it

1 0

[publiccloud-sig] Reminder - next meeting May 29th - 0700 UTC
by Felix Kronlage-Dammers 28 May '24

28 May '24

Hi everyone, This Wednesday (May 29th, tomorrow) the next meeting of the public cloud sig is going to happen. We’ll be meeting at 0700 UTC here: https://meetpad.opendev.org/publiccloud-sig See also here for all other details: https://wiki.openstack.org/wiki/PublicCloudSIG Likely we will pick up the discussions from our Forum Session at OID Berlin. See my summary that I sent to this list[1]. See you tomorrow! felix [1] <url: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack…> -- Felix Kronlage-Dammers Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

Re: [Octavia] octavia not setting up load balancers
by Jaime Ibar 28 May '24

28 May '24

Hi, I have set debug to true and I can see the following but not sure if they're related though Disabling service 'block-storage': Encountered an exception attempting to process config for project 'cinder' (service type 'block-storage'): no such option valid_interfaces in group [cinder]: oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group [cinder] Disabling service 'compute': Encountered an exception attempting to process config for project 'nova' (service type 'compute'): no such option valid_interfaces in group [nova]: oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group [nova] 2024-05-27 19:09:32.136 225246 WARNING openstack [None req-588327db-96d5-4377-ba72-b603731f9568 - c2270ba97baa40cbbb17f89d6980a2d6 - - default default] Disabling service 'image': Encountered an exception attempting to process config for project 'glance' (service type 'image'): no such option valid_interfaces in group [glance]: oslo_config.cfg.NoSuchOptError: no such option valid_interfaces in group [glance] Jaime On 27/05/2024 21:20, Winicius Allan wrote: > I understood. Is there any error log level message or anything like that? > > Em seg., 27 de mai. de 2024 às 15:19, Jaime Ibar <jim2k7(a)gmail.com> > escreveu: > > Hi Winicius, > > yes, I have the Octavia:health-mgr interface and it shows as > Active and up, in fact, I can ssh into the amphora from > the management network. > > Thanks > Jaime > > On Mon, 27 May 2024 at 18:54, Winicius Allan > <winiciusab12(a)gmail.com> wrote: > > Hi Jaime, > > In general, a load balancer stuck in the PENDING_CREATE state > could be a network problem. Do you have a network interface on > your control node that belongs to the same network as Amphora? > > Regards. > > Em seg., 27 de mai. de 2024 às 12:51, Jaime Ibar > <jim2k7(a)gmail.com> escreveu: > > Nope, I can't see anything. > > This is what I can see > > [...] > request url / request > /usr/lib/python3/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py:652 > request url https://172.16.4.56:9443// request > /usr/lib/python3/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py:655 > Connected to amphora. Response: <Response [200]> request > /usr/lib/python3/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py:677 > Amphora f3809732-80d8-4795-a5cb-cb36e63bc895 has API > version 1.0 _populate_amphora_api_version > /usr/lib/python3/dist-packages/octavia/amphorae/drivers/haproxy/rest_api_driver.py:104 > > Update DB for loadbalancer id > Marking all listeners of loadbalancer > ce31155c-3484-4ff0-8740-fcd8c0915a1c ACTIVE execute > > Mark ACTIVE in DB for load balancer id: > [...] > > So all seems to be OK but nothing in the amphora. > > Thanks > Jaime > > On Mon, 27 May 2024 at 16:25, Oliver Weinmann > <oliver.weinmann(a)me.com> wrote: > > Ok, anything suspicious in the Octavia worker logs? > > Von meinem iPhone gesendet > >> Am 27.05.2024 um 17:17 schrieb Jaime Ibar >> <jim2k7(a)gmail.com>: >> >> >> Hi Oliver, >> >> thanks for the tip but after increasing the quotas, >> the problem persists :( >> >> Jaime >> >> On Mon, 27 May 2024 at 12:20, Oliver Weinmann >> <oliver.weinmann(a)me.com> wrote: >> >> Hi, >> >> Just a quick guess. Check the quotas for security >> groups of the service tenant. >> >> In our deployment we had a similar issue and saw >> errors indicating this in the logs. >> >> Cheers, >> Oliver >> >> Von meinem iPhone gesendet >> >> > Am 27.05.2024 um 13:16 schrieb Jaime Ibar >> <jim2k7(a)gmail.com>: >> > >> > >> > Hi all, >> > >> > I've been running Octavia with no issues so far >> and now it stopped working >> > with no reasons(no recent upgrades other that OS). >> > When I create a load balancer, sometimes the >> amphora is launched successfully >> > but some others is not. >> > When it's launched, no listeners, backends, >> pools are created and the load balancer >> > gets stuck in either, pending create or pending >> update state, and deleting all the >> > load balancer details from the database is the >> only option of getting rid of it. >> > >> > I can ssh into the amphora but I can't see any >> error, gunicorn is listening >> > on port 9443 but no haproxy instances are >> created(haproxy-<uuid>) on >> > /var/lib/octavia >> > >> > I'm running 2023.2 Bobcat version. >> > >> > Any ideas? >> > >> > TIA >> > Jaime >> > >> > -- >> > salu2 >> > >> > Jaime >> >> >> >> -- >> salu2 >> >> Jaime > > > > -- > salu2 > > Jaime > > > > -- > salu2 > > Jaime > -- Jaime

1 0

[Tempest] [Blazar] Connection Refused Error 111 on Blazar tempest plugin
by Tran, Kenny 28 May '24

28 May '24

First time asking a question on the mailing list, I am trying to set up tempest against a quick start yoga (kolla-ansible) deployment with Keystone, Zun, Heat, Blazar, and Nova (1 controller, 1 compute). Default tempest tests are behaving as expected but when I try to run the blazar plugin tests they fail with Errno 111 Connection Refused. I ran a tcpdump on the tempest server side and saw no issues. Tcpdump on controller showed a reset when trying to connect to keystone. Is this an issue with blazar not being able to authenticate against the openstack deployment? The default tempest tests succeed so I don't think my admin_password admin_username etc. are incorrect. Kenny Tran Iowa State University Student Computer Engineer 515-441-5237 | trantk(a)iastate.edu

2 1

[Octavia] octavia not setting up load balancers
by Jaime Ibar 28 May '24

28 May '24

Hi all, I've been running Octavia with no issues so far and now it stopped working with no reasons(no recent upgrades other that OS). When I create a load balancer, sometimes the amphora is launched successfully but some others is not. When it's launched, no listeners, backends, pools are created and the load balancer gets stuck in either, pending create or pending update state, and deleting all the load balancer details from the database is the only option of getting rid of it. I can ssh into the amphora but I can't see any error, gunicorn is listening on port 9443 but no haproxy instances are created(haproxy-<uuid>) on /var/lib/octavia I'm running 2023.2 Bobcat version. Any ideas? TIA Jaime -- salu2 Jaime

3 7

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2024.2/R-18)
by Goutham Pacha Ravi 28 May '24

28 May '24

Hello Stackers, We're 18 weeks away from the 2024.2 (Dalmatian) release [1]. The Call for Proposals for the upcoming Open Infra Summit is still open [2], but it will close in a couple of days (May 29th, 2024). If you're planning to attend, I encourage you to submit a talk! In the past week, the OpenStack Technical Committee continued its spring cleaning exercise – this time, the party was on PyPi. We also identified dates for the TC and PTL elections for the 2025.1 ("E") release cycle [3]. The nomination period for the TC and PTL elections will begin on 2024-08-14 at 23:45 UTC and will conclude on 2024-08-28 at 23:45 UTC. The elections will be held between 23:45 UTC on 2024-09-04 and 23:45 UTC on 2024-09-18. === Weekly Meeting === We identified 207 maintainers for removal across 146 PyPi packages [4]. The cleanup task is large and manual due to a lack of API. We've also identified packages where "openstackci" cannot remove maintainers due to a lack of ownership. This is likely a long-drawn effort. I'll keep you posted as we make progress. We had a few remaining project updates pending from the last release cycle, and one of them concerned the OpenStack Charms team. Since the TC didn't have context on the health of the team and its plans, we decided to gather some information on this mailing list. We're also excited to publish the first draft for the next election dates/process, courtesy of our Election Official, Ian Y. Choi (ianychoi). This election will be socialized with early and periodic notifications to address concerns expressed regarding the lack of visibility and poor voter turnout in past elections. This meeting, like all of our IRC meetings, was logged to eavesdrop [5]. Our next weekly IRC meeting is tomorrow! The agenda is pretty light [6], likely owing to the fact that a few TC members are returning from a long weekend holiday. === Governance Proposals === ==== Merged ==== Retire devstack-gate | https://review.opendev.org/c/openstack/governance/+/919629 ==== Open for review/discussion ==== Appoint ChunYang Wu as PTL of Trove | https://review.opendev.org/c/openstack/governance/+/920462 Clarify the project retirement vs. project's repository retirement | https://review.opendev.org/c/openstack/governance/+/919984 Simplifying house rules for project updates | https://review.opendev.org/c/openstack/governance/+/920318 Update to include docs and miscellaneous repos for AC status | https://review.opendev.org/c/openstack/governance/+/915021 Add DPL model & liaison reset policy | https://review.opendev.org/c/openstack/governance/+/916833 Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] https://releases.openstack.org/dalmatian/schedule.html [2] https://2024.openinfraasia.org/ [3] https://review.opendev.org/c/openstack/election/+/920092 [4] https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup#L52 [5] https://meetings.opendev.org/meetings/tc/2024/tc.2024-05-21-18.00.log.html [6] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

1 0