September 2023 - openstack-discuss

Cyborg nova reports mdev-capable resource is not available
by Karl Kloppenborg 07 Dec '23

07 Dec '23

Hi Cyborg Team! Karl from Helm Team. When creating a VM with the correct flavor, the mdev gets created by cyborg agent and I can see it in the nodedev-list --cap mdev. However Nova then fails with: nova.virt.libvirt.driver [<removed>- - default default] Searching for available mdevs... _get_existing_mdevs_not_assigned /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driver.py :8357 2023-09-21 14:34:47.808 1901814 INFO nova.virt.libvirt.driver [<removed> - - default default] Available mdevs at: set(). 2023-09-21 14:34:47.809 1901814 DEBUG nova.virt.libvirt.driver [<removed> - - default default] No available mdevs where found. Creating an new one... _allocate_mdevs /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driv er.py:8496 2023-09-21 14:34:47.809 1901814 DEBUG nova.virt.libvirt.driver [<removed> - - default default] Attempting to create new mdev... _create_new_mediated_device /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driver.py:8385 2023-09-21 14:34:48.455 1901814 INFO nova.virt.libvirt.driver [<removed> - - default default] Failed to create mdev. No free space found among the following devices: ['pci_0000_4b_03_1', … <truncated list>]. 2023-09-21 14:34:48.456 1901814 ERROR nova.compute.manager [<removed> - - default default] [instance: 2026e2a2-b17a-43ab-adcb-62a907f58b51] Instance failed to spawn: nova.exception.ComputeResourcesUnavailable: Insufficient compute resources: mdev-capable resource is not available. Once this happened, ARQ removes the mdev and cleans up. I’ve got Cyborg 2023.2 running and have a device profile like so: karl@Karls-Air ~ % openstack accelerator device profile show e2b07e11-fe69-4f33-83fc-0f9e38adb7ae +-------------+---------------------------------------------------------------------------+ | Field | Value | +-------------+---------------------------------------------------------------------------+ | created_at | 2023-09-21 13:30:05+00:00 | | updated_at | None | | uuid | e2b07e11-fe69-4f33-83fc-0f9e38adb7ae | | name | VGPU_A40-Q48 | | groups | [{'resources:VGPU': '1', 'trait:CUSTOM_NVIDIA_2235_A40_48Q': 'required'}] | | description | None | +-------------+---------------------------------------------------------------------------+ karl@Karls-Air ~ % I can see the allocation candidate: karl@Karls-Air ~ % openstack allocation candidate list --resource VGPU=1 | grep A40 | 41 | VGPU=1 | 229bf15f-5689-3d2c-b37b-5c8439ea6a71 | VGPU=0/1 | OWNER_CYBORG,CUSTOM_NVIDIA_2235_A40_48Q | karl@Karls-Air ~ % Am I missing something critical here? Because I cannot seem to figure this out… have I got a PCI address wrong, or something? Any help from the Cyborg or Nova teams would be fantastic. Thanks, Karl.

4 9

[neutron][vpnaas][ovn] Pushing VPNaaS support through OVN
by Mohammed Naser 06 Dec '23

06 Dec '23

Hi all! One of the biggest show stoppers for us for OVN is VPNaaS but I think it’s time that we’d like to be involved in getting it over the line, I want to thank Bodo who has shown a lot of patience in getting the code necessary, and it seems that the code works for them in production as well. I’d like to ask if possible for folks to review these two outstanding changes: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/847007 https://review.opendev.org/c/openstack/neutron-vpnaas/+/765353 I know Slawek had some concerns, so hoping that raising via the email list if Bodo can address those and over the next bit of time we can get this landed in, since it even has actual proper testing both in CI and with user feedback. Thanks! Mohammed

5 6

[swift][OpenStackSDK][keystone] Warnings about (my) swift endpoint URL by keystoneauth1 discovery
by Christian Rohmann 13 Nov '23

13 Nov '23

Hey openstack-discuss, we use Ceph RadosGW (Quincy release) to provide Swift as part of an OpenStack cloud. The endpoints are configured as such: > # openstack endpoint list > +----------------------------------+--------+--------------+----------------+---------+-----------+----------------------------------------------------------------------+ > | ID | Region | Service Name | Service > Type | Enabled | Interface | URL | > +----------------------------------+--------+--------------+----------------+---------+-----------+----------------------------------------------------------------------+ > | 1234567890 | region | swift | object-store | True | > public | > https://object-store.region.cloud.example.com/swift/v1/AUTH_%(tenant_id)s > | > | 2345678901 | region | swift | object-store | True | > internal | > https://object-store.region.cloud.example.com/swift/v1/AUTH_%(tenant_id)s > | > | 3456789012 | region | swift | object-store | True | > admin | > https://object-store.region.cloud.example.com/swift/v1/AUTH_%(tenant_id)s > | > so according to https://docs.ceph.com/en/latest/radosgw/keystone/#cross-project-tenant-acce…, if I am not mistaken. I can also use the openstack client to access containers: > # openstack container list > +---------------+ > | Name | > +---------------+ > | containera | > | containerb | > +---------------+ and there is no warnings or other apparent issues. But when using "project cleanup" via e.g. "openstack project cleanup --dry-run --project $PROJECT" I see multiple warnings like > Failed to contact the endpoint at > https://object-store.region.cloud.example.com/swift/v1/AUTH_f2bc4bd34567ddc… > for discovery. Fallback to using that endpoint as the base url. > but only mentioning the swift endpoint. Following the warning string this originates from the discovery method of keystoneauth1, see https://github.com/openstack/keystoneauth/blob/28048af9593740315df8d9027c3b…. Could anybody help me understand what is issue might be? Why does this warning not appear for every use of object storage / swift via openstackclient? Is there any way for me to "fix" this? Could this even be a bug? Thanks and with kind regards Christian

4 10

[kolla] [train] [cinder] Cinder issues during controller replacement
by Albert Braden 19 Oct '23

19 Oct '23

We’re replacing controllers, and it takes a few hours to build the new controller. We’re following this procedure to remove the old controller: https://docs.openstack.org/kolla-ansible/train/user/adding-and-removing-hos… After that the cluster seems to run fine on 2 controllers, but approximately 1/3 of our volumes can’t be attached to a VM. When we look at those volumes, we see this: | os-vol-host-attr:host | qde4-ctrl1.cloud.ourdomain.com@rbd-1#rbd-1 | Ctrl1 is the controller that is being replaced. Is it possible to change the os-vol-host-attr on a volume? How can we work around this issue while we are replacing controllers? Do we need to disable the API for the duration of the replacement process, or is there a better way?

6 9

[interop][ptg] Virtual Caracal vPTG Planning
by Martin Kopec 18 Oct '23

18 Oct '23

Hello everyone, here is [1] our etherpad for the 2024.1 Caracal virtual PTG. Please, add your topics there if there is anything you would like to discuss / propose ... You can also vote for time slots for our session(s), so that they fit your schedule, at [2]. I have one topic I'd like to discuss - new view on interoperability testing, see [3] and this [4] article for the background info. [1] https://etherpad.opendev.org/p/oct2023-ptg-interop [2] https://framadate.org/rNFvjKVkMnXPfObf [3] https://etherpad.opendev.org/p/openstack-interop-2.0 [4] https://superuser.openinfra.dev/articles/new-view-on-interoperability-in-op… If you have any questions, feel free to reach out to me. Thanks, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 2

[qa][ptg] Virtual Caracal vPTG Planning
by Martin Kopec 18 Oct '23

18 Oct '23

Hello everyone, here is [1] our etherpad for the 2024.1 Caracal virtual PTG. Please, add your topics there if there is anything you would like to discuss / propose ... You can also vote for time slots for our sessions so that they fit your schedule at [2]. We will most likely go with 1-hour slot per day, as they usually fit easier into everyone's schedule. The number of slots will depend on the number of topics proposed in [1]. - [1] https://etherpad.opendev.org/p/oct2023-ptg-qa [2] https://framadate.org/f26R3EcZ2BOo7r8Q Thanks, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 2

[OPENSTACK][rabbitmq] using quorum queues
by Nguyễn Hữu Khôi 17 Oct '23

17 Oct '23

Hello guys. IS there any guy who uses the quorum queue for openstack? Could you give some feedback to compare with classic queue? Thank you. Nguyen Huu Khoi

9 19

[all][infra][tact-sig] Mailman v3 migration for OpenStack mailing lists 2023-10-12
by Jeremy Stanley 12 Oct '23

12 Oct '23

On Thursday, October 12 at 15:30 UTC, the OpenDev Collaboratory systems administrators will be migrating the lists.openstack.org mailing list site from the aging Mailman 2.1.29 server to a newer Mailman 3.3.8 deployment. This maintenance window is expected to last approximately four hours. The key takeaways are as follows: * There will be an extended outage for the site while DNS is updated and configuration, subscriber lists and message archives are imported, but incoming messages should correctly queue at the sender's end and arrive at the conclusion of our work * Because this is on a new server, there are new IP addresses from which list mail will be originating: 162.209.78.70 and 2001:4800:7813:516:be76:4eff:fe04:5423 * Moderation queues will not be copied to the new server, so moderators are encouraged to process any held messages prior to the migration time in order to avoid losing them * Anyone wishing to adjust their list subscriptions, or handle moderation or list configuration after the migration, needs to use the Sign Up button on the Web page to create a new account; it will be linked to your imported roles as soon as you confirm by following a URL from an E-mail message the server sends you, and is global for all sites on the server so you only need to do this once * The software providing Web front-ends for list management and archive browsing is entirely new in Mailman v3 and therefore has a much different appearance, though we've added appropriate redirects and frozen copies of old archives in order to accommodate existing hyperlinks If you have any questions or concerns, feel free to follow up on the service-discuss mailing list or find us in the #opendev channel on the OFTC IRC network. -- Jeremy Stanley

1 2

[all][elections][ptl] Project Team Lead Election Conclusion and Results
by Tony Breeds 10 Oct '23

10 Oct '23

Thank you to the electorate, to all those who voted and to all candidates who put their name forward for Project Team Lead (PTL) in this election. A healthy, open process breeds trust in our decision making capability thank you to all those who make this process possible. Now for the results of the PTL election process, please join me in extending congratulations to the following PTLs: * Adjutant : Dale Smith * Barbican : Grzegorz Grasza * Blazar : Pierre Riteau * Cinder : Rajat Dhasmana * Cloudkitty : Rafael Weingartner * Designate : Michael Johnson * Freezer : ge cong * Glance : Pranali Deore * Heat : Takashi Kajinami * Horizon : Vishal Manchanda * Ironic : Jay Faulkner * Keystone : Dave Wilde * Kolla : Michal Nasiadka * Kuryr : Roman Dobosz * Magnum : Jake Yip * Manila : Carlos Silva * Masakari : sam sue * Murano : Rong Zhu * Neutron : Brian Haley * Nova : Sylvain Bauza * Octavia : Gregory Thiemonge * OpenStack Charms : Felipe Reyes * OpenStack Helm : Vladimir Kozhukalov * OpenStackAnsible : Dmitriy Rabotyagov * OpenStackSDK : Artem Goncharov * Puppet OpenStack : Takashi Kajinami * Quality Assurance : Martin Kopec * Solum : Rong Zhu * Storlets : Takashi Kajinami * Swift : Tim Burke * Tacker : Yasufumi Ogawa * Telemetry : Erno Kuvaja * Vitrage : Dmitriy Rabotyagov * Watcher : chen ke * Zaqar : Hao Wang * Zun : Hongbin Lu Elections: * OpenStack_Helm: https://civs1.civs.us/cgi-bin/results.pl?id=E_3cf498bb10adc5b8 Election process details and results are also available here: https://governance.openstack.org/election/ Thank you to all involved in the PTL election process, Yours Tony.

6 8

[kolla] change default domain
by Vivian Rook 10 Oct '23

10 Oct '23

I'm looking for a setting to change the default domain name from "default" to "kolla" Does such an option exist in globals.yml? Thank you! -- *Vivian Rook (They/Them)* Site Reliability Engineer Wikimedia Foundation <https://wikimediafoundation.org/>

3 3