October 2023 - openstack-discuss - lists.openstack.org

[TC][Monasca] Proposal to mark Monasca as an inactive project
by Sławek Kapłoński 05 Jan '24

05 Jan '24

Hi, I just proposed patch [1] to mark Monasca as inactive project. It was discussed during last TC meeting on 03.10.2023. Reasons for that are described in the commit message of the [1] but I will also put all of it here: previous cycle there wasn't almost any contributions to that project and most active contributors in last 180 days were Elod Illes and Dr. Jens Harbott who were fixing some Zuul configuration issues only. Here are detailed statistics about Monasca projects: Validating Gerrit... * There are 7 ready for review patches generated within 180 days * There are 2 not reviewed patches generated within 180 days * There are 5 merged patches generated within 180 days * Unreviewed patch rate for patches generated within 180 days is 28.0 % * Merged patch rate for patches generated within 180 days is 71.0 % * Here's top 10 owner for patches generated within 180 days (Name/Account_ID: Percentage): - Dr. Jens Harbott : 42.86% - Joel Capitao : 28.57% - Elod Illes : 28.57% Validate Zuul... Set buildsets fetch size to 500 * Repo: openstack/monasca-log-api gate job builds success rate: 82% * Repo: openstack/monasca-statsd gate job builds success rate: 95% * Repo: openstack/monasca-tempest-plugin gate job builds success rate: 81% * Repo: openstack/monasca-common gate job builds success rate: 84% * Repo: openstack/monasca-kibana-plugin gate job builds success rate: 83% * Repo: openstack/monasca-ceilometer gate job builds success rate: 100% * Repo: openstack/monasca-events-api gate job builds success rate: 83% * Repo: openstack/monasca-ui gate job builds success rate: 88% * Repo: openstack/monasca-specs gate job builds success rate: 100% * Repo: openstack/monasca-grafana-datasource gate job builds success rate: 100% * Repo: openstack/monasca-persister gate job builds success rate: 98% * Repo: openstack/monasca-notification gate job builds success rate: 93% * Repo: openstack/monasca-thresh gate job builds success rate: 100% * Repo: openstack/monasca-api gate job builds success rate: 76% * Repo: openstack/monasca-agent gate job builds success rate: 98% * Repo: openstack/python-monascaclient gate job builds success rate: 100% * Repo: openstack/monasca-transform gate job builds success rate: 100% What's next? According to the "Emerging and inactive projects" document [2] if there will be no volunteers who will step up and want to maintain this project before Milestone-2 of the 2024.1 cycle (week of Jan 08 2024) there will be no new release of Monasca in the 2024.1 cycle and TC will discuss if project should be retired. So if You are interested in keeping Monasca alive and active, please reach out to the TC to discuss that ASAP. Thx in advance. [1] https://review.opendev.org/c/openstack/governance/+/897520 [2] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… -- Slawek Kaplonski Principal Software Engineer Red Hat

8 14

[kolla-ansible][zun] Intention to drop support for Zun for the 2023.1 and path forward
by Jan Gutter 21 Dec '23

21 Dec '23

Hi folks, In yesterday's Kolla IRC meeting we discussed the path forward for Zun [1] and came to the conclusion that it might be better for all involved to drop support for Zun for the 2023.1 release until it can be updated to support the necessary dependencies. More specific details at the end of the message. With this in mind, the plan is to start this process in a week from now, unless a better solution is presented. There is still some time for discussion, please don't hesitate! There are two paths forward after this: If Zun can be updated to support the dependencies during the maintenance timeframe of 2023.1, then support would be restored. If no support is available at the time Caracal releases, however, it will be considered for removal in kolla and kolla-ansible. What this means: Kolla-ansible operators currently running Zun should hold off migration to 2023.1. Any development help would of course be appreciated. Please coordinate with the Zun project to help with development. Details on the dependency problems: * Currently, Zun has an external dependency on Docker 20.x, and uses a feature that has been removed in later versions of Docker. * This Docker feature also relies on an older version of etcd (3.3) * The mitigation [2] is to pin to the old versions of Docker and etcd, but this is not possible with Debian Bookworm hosts. Why this is a problem in Kolla-ansible (and Kolla): * This can't easily be solved by vendoring in an older version of etcd: the migration path it introduces to operators would incur a significant amount of risk. * Host level involvement is also particularly painful, the host version of Docker needs to connect to the etcd service running in the containers. * Even though the containers can be kept buildable and the configuration and inventory settings could be maintained, a working setup is not testable in CI. Thanks very much for everyone's patience, contributions and time! [1]: https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-09-13-13.00.log… [2]: https://bugs.launchpad.net/zun/+bug/2007142

2 2

Cyborg nova reports mdev-capable resource is not available
by Karl Kloppenborg 08 Dec '23

08 Dec '23

Hi Cyborg Team! Karl from Helm Team. When creating a VM with the correct flavor, the mdev gets created by cyborg agent and I can see it in the nodedev-list --cap mdev. However Nova then fails with: nova.virt.libvirt.driver [<removed>- - default default] Searching for available mdevs... _get_existing_mdevs_not_assigned /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driver.py :8357 2023-09-21 14:34:47.808 1901814 INFO nova.virt.libvirt.driver [<removed> - - default default] Available mdevs at: set(). 2023-09-21 14:34:47.809 1901814 DEBUG nova.virt.libvirt.driver [<removed> - - default default] No available mdevs where found. Creating an new one... _allocate_mdevs /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driv er.py:8496 2023-09-21 14:34:47.809 1901814 DEBUG nova.virt.libvirt.driver [<removed> - - default default] Attempting to create new mdev... _create_new_mediated_device /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driver.py:8385 2023-09-21 14:34:48.455 1901814 INFO nova.virt.libvirt.driver [<removed> - - default default] Failed to create mdev. No free space found among the following devices: ['pci_0000_4b_03_1', … <truncated list>]. 2023-09-21 14:34:48.456 1901814 ERROR nova.compute.manager [<removed> - - default default] [instance: 2026e2a2-b17a-43ab-adcb-62a907f58b51] Instance failed to spawn: nova.exception.ComputeResourcesUnavailable: Insufficient compute resources: mdev-capable resource is not available. Once this happened, ARQ removes the mdev and cleans up. I’ve got Cyborg 2023.2 running and have a device profile like so: karl@Karls-Air ~ % openstack accelerator device profile show e2b07e11-fe69-4f33-83fc-0f9e38adb7ae +-------------+---------------------------------------------------------------------------+ | Field | Value | +-------------+---------------------------------------------------------------------------+ | created_at | 2023-09-21 13:30:05+00:00 | | updated_at | None | | uuid | e2b07e11-fe69-4f33-83fc-0f9e38adb7ae | | name | VGPU_A40-Q48 | | groups | [{'resources:VGPU': '1', 'trait:CUSTOM_NVIDIA_2235_A40_48Q': 'required'}] | | description | None | +-------------+---------------------------------------------------------------------------+ karl@Karls-Air ~ % I can see the allocation candidate: karl@Karls-Air ~ % openstack allocation candidate list --resource VGPU=1 | grep A40 | 41 | VGPU=1 | 229bf15f-5689-3d2c-b37b-5c8439ea6a71 | VGPU=0/1 | OWNER_CYBORG,CUSTOM_NVIDIA_2235_A40_48Q | karl@Karls-Air ~ % Am I missing something critical here? Because I cannot seem to figure this out… have I got a PCI address wrong, or something? Any help from the Cyborg or Nova teams would be fantastic. Thanks, Karl.

4 9

openstack Vm shutoff by itself
by AJ_ sunny 30 Nov '23

30 Nov '23

Hi team, I am using openstack kolla ansible on wallaby version and currently I am facing issue with virtual machine, vm is shutoff by itself and and from log it seems libvirt lifecycle stop event triggering again and again Logs:- 2023-10-16 08:48:10.971 7 WARNING nova.compute.manager [req-c7b731db-2b61-400e-917f-8645c9984696 f226d81a45dd46488fb2e19515 848 316d215042914de190f5f9e1c8466bf0 default default] [instance: 4b04d3f1-1fbd-4b63-b693-a0ef316ecff3] Received unexpected - vent network-vif-plugged-f191f6c8-dff5-4c1b-94b3-8d91aa6ff5ac for instance with vm_state active and task_state None. 2023-10-21 22:42:44.589 7 INFO nova.compute.manager [-] [instance: 4b04d3f1-1fbd-4b63-b693-a0ef316ecff3] VM Stopped (Lifecyc Event) 2023-10-21 22:42:44.683 7 INFO nova.compute.manager [req-1d99b87b-7ff7-462d-ab18-fbdec6bda71d -] [instance: 4b04d3f1- fbd-4b63-b693-a0ef316ecff3] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from ti e hypervisor (4). Updating power_state in the DB to match the hypervisor. 2023-10-21 22:42:44.811 7 WARNING nova.compute.manager [req-1d99b87b-7ff7-462d-ab18-fbdec6bda71d ----] [instance: 4b04d3f 1-1fbd-4b63-b693-a0ef316ecff3] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state : None, original DB power_state: 1, current VM power_state: 4 2023-10-21 22:42:44.977 7 INFO nova.compute.manager [req-1d99b87b-7ff7-462d-ab18-fbdec6bda71d -] [instance: 4b04d3f1-1 fbd-4b63-b693-a0ef316ecff3] Instance is already powered off in the hypervisor when stop is called. Thanks & Regards Arihant Jain +91 8299719369

5 15

[swift][OpenStackSDK][keystone] Warnings about (my) swift endpoint URL by keystoneauth1 discovery
by Christian Rohmann 13 Nov '23

13 Nov '23

Hey openstack-discuss, we use Ceph RadosGW (Quincy release) to provide Swift as part of an OpenStack cloud. The endpoints are configured as such: > # openstack endpoint list > +----------------------------------+--------+--------------+----------------+---------+-----------+----------------------------------------------------------------------+ > | ID | Region | Service Name | Service > Type | Enabled | Interface | URL | > +----------------------------------+--------+--------------+----------------+---------+-----------+----------------------------------------------------------------------+ > | 1234567890 | region | swift | object-store | True | > public | > https://object-store.region.cloud.example.com/swift/v1/AUTH_%(tenant_id)s > | > | 2345678901 | region | swift | object-store | True | > internal | > https://object-store.region.cloud.example.com/swift/v1/AUTH_%(tenant_id)s > | > | 3456789012 | region | swift | object-store | True | > admin | > https://object-store.region.cloud.example.com/swift/v1/AUTH_%(tenant_id)s > | > so according to https://docs.ceph.com/en/latest/radosgw/keystone/#cross-project-tenant-acce…, if I am not mistaken. I can also use the openstack client to access containers: > # openstack container list > +---------------+ > | Name | > +---------------+ > | containera | > | containerb | > +---------------+ and there is no warnings or other apparent issues. But when using "project cleanup" via e.g. "openstack project cleanup --dry-run --project $PROJECT" I see multiple warnings like > Failed to contact the endpoint at > https://object-store.region.cloud.example.com/swift/v1/AUTH_f2bc4bd34567ddc… > for discovery. Fallback to using that endpoint as the base url. > but only mentioning the swift endpoint. Following the warning string this originates from the discovery method of keystoneauth1, see https://github.com/openstack/keystoneauth/blob/28048af9593740315df8d9027c3b…. Could anybody help me understand what is issue might be? Why does this warning not appear for every use of object storage / swift via openstackclient? Is there any way for me to "fix" this? Could this even be a bug? Thanks and with kind regards Christian

4 10

[ptls] [tc] 2023 OpenStack Deployment Data
by Allison Price 09 Nov '23

09 Nov '23

Hi everyone, Apologies for sending this during the PTG as I intended to send this prior, but below is a link to the anonymized data including responses to many project or TC contributed questions. I have removed a significant number of fields to preserve the anonymity of the organizations, but if there is a particular data point that would help your team, please let me know. Cheers, Allison https://docs.google.com/spreadsheets/d/1ehlabuqOnNK4M7xtN1GXvhLoeHn4JBaISih…

2 1

[nova] Nov 7 is a spec review day and Nov 14 is implementation review day
by Sylvain Bauza 07 Nov '23

07 Nov '23

Hey folks, I feel a bit sad to had no time yet to provide you a summary for our PTG, but please wait a bit more :-) That said, something you should already know, we will have two spec review days this cycle and two implementation review days as well, so please mark your calendars for the first to come : - Nov 7 will be a spec review day. Please ensure your specs are ready for reviews - Nov 14 will be an implementation review day. Mark your patches up ! As a reminder, the idea behind those review days is to synchronize our efforts between owners and reviewers so people can discuss at the same time. Thanks, -Sylvain

1 1

[charms] Nominating Utkarsh Bhatt for charms-ceph-core
by Peter Sabaini 03 Nov '23

03 Nov '23

Hello all, I would like to nominate Utkarsh Bhatt for charms-ceph-core. Utkarsh has worked for the Ceph charms team for 1.5 years now and has provided many contributions to the charms, especially to the Ceph charms[0]. I know him as a diligent and enthusiastic contributor and hope you will support his nomination. Thanks, peter. [0] https://review.opendev.org/q/owner:utkarsh.bhatt%2540canonical.com

4 3

[ops] Seeking Advice on Service Deployment and Layout for an 8-Node Cluster
by Forrest Fuqua 02 Nov '23

02 Nov '23

Hi there, I'm currently exploring some strategies and seeking advice on the deployment and layout of services on an 8-node cluster. Each node is equipped with dual AMD EYPC 7702s, two 100G Links, and 2TB of RAM. Additionally, I've set up two 13TB NVME drives in a Ceph configuration for each node. At present, the setup involves one management node, while the others are dedicated to running Nova, Neutron, Glance, and Cinder. The workload, while fairly basic, involves handling substantial numbers. It's primarily used for one-off research projects and a few classes, serving around 60 end users. The most demanding scenario occurs during cybersecurity competitions when we need to deploy 3100 VMs at once. When this is spread out over a few days, it's manageable. However, there's a risk of overloading the system, leading to DiskIO issues and Database inconsistencies. Currently, I'm utilizing Kolla for deployment, and I've shared my configurations on this GitHub repository: Link to Configs<https://github.com/RIT-GCI-CyberRange/openstack-configs>. I'd greatly appreciate any insights or advice on optimizing this setup to handle the peak workload more efficiently and prevent potential performance issues. Thanks in advance for your help!

3 7

[ironic] Announcement: Future of Metalsmith
by Jay Faulkner 01 Nov '23

01 Nov '23

Hi all, Just announcing that at the vPTG, the Ironic core team came to a consensus about the future of the metalsmith project. It was originally created to improve the user experience around provisioning Ironic nodes when using Ironic's API directly. At this, it was mostly successful and has completed this task. It's our intention to work on the Ironic API and python-ironicclient to provide a similarly simple deployment user experience, even for users without nova, integrating lessons learned from the metalsmith project. To this end; we are letting the overall community know we no longer expect to add additional functionality to metalsmith. We will ensure it remains maintained and well-tested until it's functionality has been duplicated, but all new work in this area will be directed at the primary Ironic API and Ironic client projects. To this end, and as discussed at the PTG, I've proposed a change to the Metalsmith readme, notifying users of this state. https://review.opendev.org/c/openstack/metalsmith/+/899761 This change, or this mailing list post, is a great place to comment if you have any comments on this decision. Thanks, Jay Faulkner Ironic PTL

3 3