October 2024 - openstack-discuss - lists.openstack.org

[kolla][neutron][ovn] High ovs-dbserver CPU Usage with OVN in Kolla-ansible Large-Scale Deployment
by 韩光宇 18 Oct '24

18 Oct '24

Hi, I have a Wallaby cluster with 5 controller and 130 compute node. Host OS: Rocky 8.7 Deploy tool: Kolla-ansbile kolla_base_distro(container images): centos Network Mode：**OVN** After completing the deployment, the functionality for creating a single instance works as expected, and instances are created successfully. However, the process is extremely slow. When attempting to create instances in bulk, the operation fails on a large scale. During this time, I observed that the ovsdb-server is consuming close to 100% CPU usage. Even without any tasks being processed, the CPU usage remains consistently around 90%, just from the cluster's regular operations. I redeployed the cluster and switched the network mode to Open vSwitch (OVS), and everything works as expected with normal performance. Could anyone offer guidance on what might be causing this issue? I’m aiming to use the OVN mode with Kolla in a large-scale OpenStack cluster (100+ nodes). What configuration or adjustments would I need to make? Thank you in advance for your help! Han Guangyu

2 2

[all][qa] Moving grenade to install things in venv: Enable GLOBAL_VENV
by Ghanshyam Mann 18 Oct '24

18 Oct '24

Hello Everyone, Since 2023.2 devstack by default, installs things in virtual env and due to the old node was not ready in the grenade job, we disabled that with the flag GLOBAL_VENV. Now it is time to enable it in greande also. This also fixes the failure we are getting in the grenade setup for the current master (upgrade testing from stable/2024.2 -> master) which is setuptool error for projects that have pyproject.toml[1]. I have proposed the changes in granade, but at least Octavia, Ironic, and heat jobs are failing. I have proposed some fixes for these projects. - https://review.opendev.org/q/topic:%22grenade-venv%22 I would like to request the project to test their grenade jobs with https://review.opendev.org/c/openstack/grenade/+/930507 and fix the failure if any. I would like to merge the grenade change as soon as its gate passes so that we can set up the grenade testing correctly for the current master. If any project requires a large change/more time to make their grenade jobs work in venv then you can disable the GLOBAL_VENV there for the time being and enable it once things work. [1] https://zuul.opendev.org/t/openstack/build/517b21fc57d6480f902273e58f9032b5 -gmann

1 1

[Glance] Cancelling weekly meetings Today and Next week
by Pranali Deore 17 Oct '24

17 Oct '24

Hello Team !! I won't be able to chair today's meeting, I will be out for some personal reason and also there is nothing much on agenda for this week, So cancelling the meeting for today. Sorry for the last minute update ! Next week's meeting is also cancelled because of the PTG session. So you all next week!! Thanks & Regards, Pranali Deore

1 0

[keystone] What does a multi-region service catalog look like?
by Stephen Finucane 17 Oct '24

17 Oct '24

o/ The question in $subject came up in a discussion earlier today and I neither knew the answer nor was able to discover it by looking at a service catalogs for a few production deployments I have access to (also, the multi-region feature is currently broken in DevStack and likely has been for some time). Crucially, in a "standard" multi-region deployment where there is one Keystone instances and N Nova/Cinder/Glance/Neutron/... deployments, would I have something like the below, where there is only one identity endpoint but multiple e.g. compute endpoints, or would I expect there always to be as many identity endpoints as there are e.g. compute endpoints? { "token": { ... "catalog": [ { "endpoints": [ { "id": "fb757b386a23430d8f4d636b8b45fa89", "interface": "public", "region_id": "RegionOne", "url": "http://10.0.110.125/identity", "region": "RegionOne" } ], "id": "50c4feff9d6149b898fce20ce23904a3", "type": "identity", "name": "keystone" }, { "endpoints": [ { "id": "2ed0cb59adb0415c948f8351b6546d3e", "interface": "public", "region_id": "RegionOne", "url": "http://10.0.110.125/compute/v2.1", "region": "RegionOne" }, { "id": "2ed0cb59adb0415c948f8351b6546d3e", "interface": "public", "region_id": "RegionTwo", "url": "http://10.0.112.232/compute/v2.1", "region": "RegionOne" } ], "id": "87322637c5ab4e7b90bea7a8c1eb9525", "type": "compute", "name": "nova" }, ... ] } } I'm guessing the latter, but I'd like to confirm this. Stephen

3 4

[privsep][nova] howto enable privsep logs in nova
by Benedikt Trefzer 17 Oct '24

17 Oct '24

Hi List I'd like to enable log messages for privsep executions. Currently a log file is created (/var/log/nova/privsep.log) but no content is ever written to it. How do I enable the log (Warning/Info/debug) messages to be written into that file ? Thanks for any hint Benedikt

1 0

[largescale-sig] Next meeting: October 16, 9utc
by Thierry Carrez 16 Oct '24

16 Oct '24

Hi everyone, The Large Scale SIG will be meeting this Wednesday in #openstack-operators on OFTC IRC, at 9UTC. You can doublecheck how that UTC time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20241016T09 Feel free to add topics to the agenda: https://etherpad.opendev.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez

1 1

[manila] CephFS Driver Failing to Connect in Manila on OpenStack
by Shubham Kumar Yadav 16 Oct '24

16 Oct '24

Hi, im trying to install Manila (helm chart) in my Openstack cloud but getting some errors. If you could please help me with those. Im following this doc - https://docs.openstack.org/manila/latest/configuration/shared-file-systems/… 1 - my manila conf looks like conf: manila: DEFAULT: debug: true enabled_share_protocols: NFS,CIFS,CEPHFS enabled_share_backends: cephfsnative1 cephfsnative1: driver_handles_share_servers: False share_backend_name: CEPHFSNATIVE1 share_driver: manila.share.drivers.cephfs.driver.CephFSDriver cephfs_conf_path: /etc/ceph/ceph.conf cephfs_protocol_helper_type: CEPHFS cephfs_auth_id: manila cephfs_cluster_name: rook-ceph cephfs_filesystem_name: myfs 2 - So i created the auth using (according to document) ceph auth get-or-create client.manila -o manila.keyring \ mgr 'allow rw' \ mon 'allow r' 3 - when i try to use the maila keyring i get the error *[rook@cmp-2 ceph]$ ceph --key AQBip/9mCMf4N/jFNKFlwl/hA4j64esWuA== osd ls[errno 13] RADOS permission denied (error connecting to the cluster)*this is what i get after running that not sure what permissions any idea? -> i also tried with admin rights to the auth *[rook@cmp-2 ceph]$ ceph auth get client.manila[client.manila] key = AQBip/9mCMf4N/jFNKFlwl/hA4j64esWuA== caps mds = "allow *" caps mgr = "allow *" caps mon = "allow *" caps osd = "allow *"exported keyring for client.manila * 4 - and also in manila-share pod i see the error 2024-10-08 07:37:25.090 7 INFO manila.share.drivers.cephfs.driver [None req-fffe12f7-c7aa-42dc-aea1-bed6f918b278 - - - - - -] [CEPHFSNATIVE1] Ceph client found, connecting... 2024-10-08 07:37:25.104 7 ERROR manila.share.manager [None req-fffe12f7-c7aa-42dc-aea1-bed6f918b278 - - - - - -] Error encountered during initialization of driver CephFSDriver@lab-11053-26006-ceph-1@cephfsnative1: manila.exception.ShareBackendException: [%(be)s] Ceph client failed to connect. 2024-10-08 07:37:25.104 7 ERROR manila.share.manager Traceback (most recent call last): 2024-10-08 07:37:25.104 7 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/drivers/cephfs/driver.py", line 421, in rados_client 2024-10-08 07:37:25.104 7 ERROR manila.share.manager self._rados_client.connect() 2024-10-08 07:37:25.104 7 ERROR manila.share.manager File "rados.pyx", line 690, in rados.Rados.connect 2024-10-08 07:37:25.104 7 ERROR manila.share.manager rados.IOError: [errno 5] RADOS I/O error (error connecting to the cluster) 2024-10-08 07:37:25.104 7 ERROR manila.share.manager 2024-10-08 07:37:25.104 7 ERROR manila.share.manager During handling of the above exception, another exception occurred: 2024-10-08 07:37:25.104 7 ERROR manila.share.manager 2024-10-08 07:37:25.104 7 ERROR manila.share.manager Traceback (most recent call last): 2024-10-08 07:37:25.104 7 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/manager.py", line 371, in _driver_setup 2024-10-08 07:37:25.104 7 ERROR manila.share.manager self.driver.do_setup(ctxt) 2024-10-08 07:37:25.104 7 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/drivers/cephfs/driver.py", line 274, in do_setup 2024-10-08 07:37:25.104 7 ERROR manila.share.manager rados_client=self.rados_client, 2024-10-08 07:37:25.104 7 ERROR manila.share.manager File "/var/lib/openstack/lib/python3.10/site-packages/manila/share/drivers/cephfs/driver.py", line 424, in rados_client 2024-10-08 07:37:25.104 7 ERROR manila.share.manager raise exception.ShareBackendException( 2024-10-08 07:37:25.104 7 ERROR manila.share.manager manila.exception.ShareBackendException: [%(be)s] Ceph client failed to connect. 2024-10-08 07:37:25.104 7 ERROR manila.share.manager 2024-10-08 07:37:25.105 7 DEBUG manila.utils [None req-fffe12f7-c7aa-42dc-aea1-bed6f918b278 - - - - - -] Finished call to 'manila.share.manager.ShareManager.init_host.<locals>._driver_setup' after 62.106(s), this was the 6th time calling it. log_it /var/lib/openstack/lib/python3.10/site-packages/tenacity/after.py:44 2024-10-08 07:37:25.105 7 DEBUG manila.utils [None req-fffe12f7-c7aa-42dc-aea1-bed6f918b278 - - - - - -] Retrying manila.share.manager.ShareManager.init_host.<locals>._driver_setup in 64.0 seconds as it raised ShareBackendException: [%(be)s] Ceph client failed to connect.. log_it /var/lib/openstack/lib/python3.10/site-packages/tenacity/before_sleep.py:65 any idea what could be wrong? or if there's a better guide available Hope to hear from you soon Thank you!

1 0

[octavia] cancelling weekly meetings today and next week
by Gregory Thiemonge 16 Oct '24

16 Oct '24

Hi Folks, Many contributors have a conflict today with the Octavia weekly meeting, so we are cancelling it. Next week's meeting is also cancelled because of the PTG session on the same day. Thanks, Greg

1 0

Invitation: [openstack-ansible][ptg] Epoxy Team Gathering @ Tue Oct 22, 2024 6pm - 8pm (CEST) (openstack-discuss@lists.openstack.org)
by Hosam Al Ali 16 Oct '24

16 Oct '24

[openstack-ansible][ptg] Epoxy Team Gathering Tuesday Oct 22, 2024 ⋅ 6pm – 8pm Central European Time - Stockholm Join with Google Meet https://meet.google.com/jjg-hwhi-vqg?hs=224 Hey folks! I have booked a relatively short session for next Tuesday, October 22 2024 from 16:00 to 18:00 UTC. The session will happen in a meetpad [1]. Topics for the discussion are held in etherpad [2]. We're having quite a scarce agenda this time, which mostly consist of things which were planned for previous cycles but were never accomplished. So I think we should be good with this 2 hour session for the discussion. If not, or smth will raise during the discussion, we can arrange another discussion during the PTG week. Thankfully, plenty of slots are available this time as well. Hope to see everyone interested in the project there! [1] https://meetpad.opendev.org/osa-epoxy-ptg [2] https://etherpad.opendev.org/p/osa-epoxy-ptg Organizer Hosam Al Ali a.hosam(a)gmail.com Guests Hosam Al Ali - organizer openstack-discuss(a)lists.openstack.org noonedeadpunk(a)gmail.com View all guest info https://calendar.google.com/calendar/event?action=VIEW&eid=MHI1YzRhcTNzbTI1… Reply for openstack-discuss(a)lists.openstack.org and view more details https://calendar.google.com/calendar/event?action=VIEW&eid=MHI1YzRhcTNzbTI1… Your attendance is optional. ~~//~~ Invitation from Google Calendar: https://calendar.google.com/calendar/ You are receiving this email because you are an attendee on the event. To stop receiving future updates for this event, decline this event. Forwarding this invitation could allow any recipient to send a response to the organizer, be added to the guest list, invite others regardless of their own invitation status, or modify your RSVP. Learn more https://support.google.com/calendar/answer/37135#forwarding

1 0

[tc] Weekly IRC meeting on October 22, 2024 is cancelled
by Dmitriy Rabotyagov 16 Oct '24

16 Oct '24

Hi everyone, Due to the PTG [1] which is taking place next week (October 21-25) and TC has already planned to have 2 sessions (you can check on agenda [2] and join us there), it was agreed to skip our regular IRC meeting on October 22. Next TC meeting in IRC channel #openstack-tc on OFTC network is planned on October 29, 2024. [1] https://openinfra.dev/ptg/ [2] https://etherpad.opendev.org/p/oct2024-ptg-os-tc

1 0