September 2023 - openstack-discuss

[openstack][kolla-ansible]about group node config overwrite
by Nguyễn Hữu Khôi 30 Oct '25

30 Oct '25

Hello Koller. I want to ask if we can overwrite config for a group of nodes. For example, compute01 >> compute10 will have the same nova configuration compute11 >> compute20 will have the same nova configuration. It will be very nice if we support this way. Thank you. Regards Nguyen Huu Khoi

4 7

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

How to config DPDK compute in OVN
by dangerzone ar 22 Apr '25

22 Apr '25

Hi....as far I understand ovs does support dpdk compute to be deployed. I'm referring to the guideline ' *https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html <https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html>* ' ... It's too brief.. May I know how to config dpdk compute in OVN networking openstack? Hope someone could help what is the config to be done.... Currently compute is run without dpdk over openstack train ovn based. Please need some info on how I can proceed. Thank you

2 1

ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version
by Swogat Pradhan 24 Dec '24

24 Dec '24

Hi, I am currently trying to deploy openstack wallaby using tripleo arch. I created the network jinja templates, ran the following commands also: #openstack overcloud network provision --stack overcloud --output networks-deployed-environment.yaml custom_network_data.yaml # openstack overcloud network vip provision --stack overcloud --output vip-deployed-environment.yaml custom_vip_data.yaml # openstack overcloud node provision --stack overcloud --overcloud-ssh-key /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml and used the environment files in the openstack overcloud deploy command: (undercloud) [stack@hkg2director ~]$ cat deploy.sh #!/bin/bash THT=/usr/share/openstack-tripleo-heat-templates/ CNF=/home/stack/ openstack overcloud deploy --templates $THT \ -r $CNF/templates/roles_data.yaml \ -n $CNF/workplace/custom_network_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e $CNF/templates/node-info.yaml \ -e $CNF/templates/scheduler-hints.yaml \ -e $CNF/workplace/networks-deployed-environment.yaml \ -e $CNF/workplace/vip-deployed-environment.yaml \ -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ -e $CNF/workplace/custom-net-bond-with-vlans.yaml Now when i run the ./deploy.sh script i encounter an error stating: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command.: tripleoclient.exceptions.InvalidConfiguration: Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command. 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END return value: 1 Can someone tell me where the mistake is? With regards, Swogat Pradhan

4 4

[kolla] [openeuler] CI jobs status
by Michał Nasiadka 26 Nov '24

26 Nov '24

Hello Koalas, OpenEuler jobs in master (to be Antelope) are failing for a long time. If no one is interested in maintaining OpenEuler support in Kolla-Ansible - I will propose dropping it’s support on Wednesday Kolla meeting. Best regards, Michal

4 3

Kolla Ansible on Ubuntu 20.04 - cloud-init & other network issues
by Tobias McNulty 10 Oct '24

10 Oct '24

Hi, I'm attempting to use Kolla Ansible 14.6.0 to deploy OpenStack Yoga on a small 3-node Ubuntu 20.04 cluster. The nodes have 128 GB RAM each, dual Xeon processors, and dual 10G Intel NICs. The NICs are connected to access ports on a 10G switch with separate VLANs for the local and external networks. All the playbooks run cleanly, but cloud-init is failing in the Ubuntu 20.04 and 22.04 VMs I attempt to boot. The VM images are unmodified from https://cloud-images.ubuntu.com/, and cloud-init works fine if I mount a second volume with user-data. The error is a timeout attempting to reach 169.254.169.254. This occurs both when booting a VM in an internal routed network and directly in an external network. I tried various neutron plugin agents (ovn, linuxbridge, and openvswitch both with and without firewall_driver = openvswitch <https://docs.openstack.org/kolla-ansible/latest/reference/networking/neutro…>) first with a clean install of the entire OS each time, all with the same result. Running tcpdump looking for 169.254.169.254 shows nothing. As a possible clue, the virtual NICs are unable to pass any traffic (e.g., to reach an external DHCP server) unless I completely disable port security on the interface (even if the associated security group is wide open). But disabling port security does not fix cloud-init (not to mention I don't really want to disable port security). Are there any additional requirements related to deploying OpenStack with Kolla on Ubuntu 20.04? This is a fairly vanilla configuration using the multinode inventory as a starting point. I tried to follow the Quick Start <https://docs.openstack.org/kolla-ansible/yoga/user/quickstart.html> as closely as possible; the only material difference I see is that I'm using the same 3 nodes for control + compute. I am using MAAS so it's easy to get a clean OS install on all three nodes ahead of each attempt. I plan to try again with the standard (non-HWE) kernel just in case, but otherwise I am running out of ideas. In case of any additional clues, here are my globals.yml and inventory file, along with the playbook I'm using to configure the network, images, VMs, etc., after bootstrapping the cluster: https://gist.github.com/tobiasmcnulty/7dbbdbc67abc08cbb013bf5983852ed6 Thank you in advance for any advice! Cheers, Tobias

5 9

Live migration fails
by Szabo, Istvan (Agoda) 07 Aug '24

07 Aug '24

Hi, I have couple of compute nodes where the live migration fails with existing vms. When I quickly spawn a vm and try live migration it works so I assume shouldn't be a big problem with the compute node. However I have many existing vms where it fails with a servername not found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 ERROR nova.conductor.tasks.migrate [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Unable to find record for source node servername on servername: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Failed to compute_task_migrate_server: Compute host servername could not be found.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Setting instance to ACTIVE state.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.672 227612 ERROR oslo_messaging.rpc.server [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Exception during message handling: ComputeHostNotFound: Compute host am-osfecn-4025 Tried with this command: nova live-migration --block-migrate id. Any idea? Thank you. ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

5 16

[ptl][tc] OpenStack packages PyPi additional external maintainers audit & cleanup
by Ghanshyam Mann 16 May '24

16 May '24

Hi PTLs, As you might know or have seen for your project package on PyPi, OpenStack deliverables on PyPi have additional maintainers, For example, https://pypi.org/project/murano/, https://pypi.org/project/glance/ We should keep only 'openstackci' as a maintainer in PyPi so that releases of OpenStack deliverables can be managed in a single place. Otherwise, we might face the two sets of maintainers' places and packages might get released in PyPi by additional maintainers without the OpenStack project team knowing about it. One such case is in Horizon repo 'xstatic-font-awesome' where a new maintainer is added by an existing additional maintainer and this package was released without the Horizon team knowing about the changes and release. - https://github.com/openstack/xstatic-font-awesome/pull/2 To avoid the 'xstatic-font-awesome' case for other packages, TC discussed it in their weekly meetings[1] and agreed to audit all the OpenStack packages and then clean up the additional maintainers in PyPi (keep only 'openstackci' as maintainers). To help in this task, TC requests project PTL to perform the audit for their project's repo and add comments in the below etherpad. - https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup Thanks to knikolla to automate the listing of the OpenStack packages with additional maintainers in PyPi which you can find the result in output.txt at the bottom of this link. I have added the project list of who needs to check their repo in etherpad. - https://gist.github.com/knikolla/7303a65a5ddaa2be553fc6e54619a7a1 Please complete the audit for your project before March 15 so that TC can discuss the next step in vPTG. [1] https://meetings.opendev.org/meetings/tc/2023/tc.2023-01-11-16.00.log.html#… -gmann

18 27

Accessing virsh in openstack wallaby | tripleo
by Swogat Pradhan 24 Mar '24

24 Mar '24

Hi, Can someone please tell me how can i access the virsh utility on the compute node on openstack wallaby? Previously we used nova_libvirt but in wallaby there are multiple containers and i am unable to use virsh utility to edit my domain.xml file. List of containers: [root@dcn01-hci-2 ~]# podman ps | grep nova 10d981158842 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtlogd_wrapper e91667516afc 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtsecretd a4b07fe0e833 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtnodedevd d7e1db393e9c 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtstoraged 5308a171793e 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtqemud f490a6249ba1 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 6 weeks ago nova_virtproxyd ab0441e56957 172.25.201.68:8787/tripleomaster/openstack-nova-compute:current-tripleo kolla_start 2 months ago Up 2 months ago (healthy) nova_migration_target f35fbba5b690 172.25.201.68:8787/tripleomaster/openstack-nova-compute:current-tripleo kolla_start 2 months ago Up 6 weeks ago (unhealthy) nova_compute 82bc01993106 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo /usr/sbin/virtlog... 2 months ago Up 2 months ago Can someone please suggest how I can i edit an instance xml file in this setup? With regards, Swogat Pradhan

2 5

[Watcher] How use Watcher to scheduler Openstack resource without cpu_usage
by Nguyễn Hữu Khôi 06 Mar '24

06 Mar '24

Hello guys. I want to use watcher to scheduler openstack resources but some openstack versions are missing cpu_usage metric. How do we use Watcher without it? Thank you. Nguyen Huu Khoi

3 5

[openstack][skyline] Some works with skyline
by Nguyễn Hữu Khôi 26 Feb '24

26 Feb '24

Hello guys. I am working on Skyline. I can integrate my monitor tab on it. I will try to do something about it. If it works properly, I can contribute. It is a very nice project. I hope it will grow and grow in the future. Thanks Skyline Team. [image: image.png] Nguyen Huu Khoi

3 7

[neutron][vpnaas][ovn] Pushing VPNaaS support through OVN
by Mohammed Naser 12 Feb '24

12 Feb '24

Hi all! One of the biggest show stoppers for us for OVN is VPNaaS but I think it’s time that we’d like to be involved in getting it over the line, I want to thank Bodo who has shown a lot of patience in getting the code necessary, and it seems that the code works for them in production as well. I’d like to ask if possible for folks to review these two outstanding changes: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/847007 https://review.opendev.org/c/openstack/neutron-vpnaas/+/765353 I know Slawek had some concerns, so hoping that raising via the email list if Bodo can address those and over the next bit of time we can get this landed in, since it even has actual proper testing both in CI and with user feedback. Thanks! Mohammed

6 8

[all] SQLAlchemy 2.x support in Bobcat
by Thomas Goirand 02 Feb '24

02 Feb '24

Hi, As you may know, and to my great frustration, I'm not the maintainer of SQLAlchemy in Debian, even though OpenStack is the biggest consumer of it. The current maintainer insists that he wants to upload SQLA 2.x in Unstable, potentially breaking all of OpenStack. At the present moment, if I understand correctly, we're not there yet, and Bobcat doesn't have such a support. It would be ok for me, *IF* there are patches available on master, that I could backport to Bobcat and maintain in the debian/patches folder of each project. However, the biggest current annoyance, is that I have no idea where we are at. Are we close to such a support? Is there a list of patches to apply on top of Bobcat that is maintained somewhere? Please enlighten me... :) Cheers, Thomas Goirand (zigo)

5 10

[designate] Proposal to deprecate the agent framework and agent based backends
by Michael Johnson 09 Jan '24

09 Jan '24

TLDR: The Designate team would like to deprecate the backend agent framework and the agent based backends due to lack of development and design issues with the current implementation. The following backends would be deprecated: Bind9 (Agent), Denominator, Microsoft DNS (Agent), Djbdns (Agent), Gdnsd (Agent), and Knot2 (Agent). Designate includes many backend DNS server drivers[1], many of which are "native" (also known as xfr type backends) backend implementations. In addition to the "native" backends, Designate has an agent backend[2] that supports other backends via an agent process. To quote the agent backend documentation[2]: This backend uses an extension[3] of the DNS protocol itself to send management requests to the remote agent processes, where the requests will be actioned. The rpc traffic between designate and the agent is both unauthenticated and unencrypted. Do not run this traffic over unsecured networks. Here are the reasons we are proposing to deprecate the agent framework now: 1. The agent protocol used by Designate is using an "unassigned"[4][5] DNS opcode (14) that is causing problems with the dnspython library >= 2.3.0 which is now validating the opcode when building DNS messages. It is a bad practice to use "unassigned" values as they may be officially assigned at any time, likely with an incompatible message format. 2. The agent backends are not tested in the OpenStack jobs[1]. 3. Many of the agent backends have been marked as "Experimental" since 2016 with no additional contributions beyond general repository code maintenance. 4. The protocol between the Designate worker process and the agent process is unauthenticated and unencrypted. 5. We do not know of a development resource to rewrite the agent framework protocol to address issue #1 and #4 above. 6. The introduction of catalog zones[6] may eliminate the need for some of the agent based backend drivers. By marking the agent framework and agent based backends "deprecated" in the Antelope cycle, we would remove the code no earlier than the "C" release of OpenStack (per the OpenStack deprecation policy[7]). In the meantime, issue #1 has been worked around by overriding the dnspython opcode validation[7] as needed in the Designate code (similar to a monkey patch). This is not a sustainable long term solution. The following backend agent based drivers would be marked "deprecated" in addition to the agent framework itself: Bind9 (Agent) Denominator Microsoft DNS (Agent) Djbdns (Agent) Gdnsd (Agent) Knot2 (Agent) We plan to propose patches for the deprecation over the next week, but will not merge them until at least January 24th to allow time for comment from the community. If you have concerns about this deprecation plan or are interested in rewriting the agent framework protocol to address the above issues, please reply to this announcement. Michael [1] https://docs.openstack.org/designate/latest/admin/support-matrix.html [2] https://docs.openstack.org/designate/latest/admin/backends/agent.html [3] https://github.com/openstack/designate/blob/master/designate/backend/privat… [4] https://www.rfc-editor.org/rfc/rfc6895.html#section-2.2 [5] https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-pa… [6] https://www.ietf.org/archive/id/draft-ietf-dnsop-dns-catalog-zones-08.txt [7] https://docs.openstack.org/project-team-guide/deprecation.html [8] https://review.opendev.org/c/openstack/designate/+/870678

3 13

[kolla-ansible][zun] Intention to drop support for Zun for the 2023.1 and path forward
by Jan Gutter 21 Dec '23

21 Dec '23

Hi folks, In yesterday's Kolla IRC meeting we discussed the path forward for Zun [1] and came to the conclusion that it might be better for all involved to drop support for Zun for the 2023.1 release until it can be updated to support the necessary dependencies. More specific details at the end of the message. With this in mind, the plan is to start this process in a week from now, unless a better solution is presented. There is still some time for discussion, please don't hesitate! There are two paths forward after this: If Zun can be updated to support the dependencies during the maintenance timeframe of 2023.1, then support would be restored. If no support is available at the time Caracal releases, however, it will be considered for removal in kolla and kolla-ansible. What this means: Kolla-ansible operators currently running Zun should hold off migration to 2023.1. Any development help would of course be appreciated. Please coordinate with the Zun project to help with development. Details on the dependency problems: * Currently, Zun has an external dependency on Docker 20.x, and uses a feature that has been removed in later versions of Docker. * This Docker feature also relies on an older version of etcd (3.3) * The mitigation [2] is to pin to the old versions of Docker and etcd, but this is not possible with Debian Bookworm hosts. Why this is a problem in Kolla-ansible (and Kolla): * This can't easily be solved by vendoring in an older version of etcd: the migration path it introduces to operators would incur a significant amount of risk. * Host level involvement is also particularly painful, the host version of Docker needs to connect to the etcd service running in the containers. * Even though the containers can be kept buildable and the configuration and inventory settings could be maintained, a working setup is not testable in CI. Thanks very much for everyone's patience, contributions and time! [1]: https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-09-13-13.00.log… [2]: https://bugs.launchpad.net/zun/+bug/2007142

2 2

Cyborg nova reports mdev-capable resource is not available
by Karl Kloppenborg 08 Dec '23

08 Dec '23

Hi Cyborg Team! Karl from Helm Team. When creating a VM with the correct flavor, the mdev gets created by cyborg agent and I can see it in the nodedev-list --cap mdev. However Nova then fails with: nova.virt.libvirt.driver [<removed>- - default default] Searching for available mdevs... _get_existing_mdevs_not_assigned /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driver.py :8357 2023-09-21 14:34:47.808 1901814 INFO nova.virt.libvirt.driver [<removed> - - default default] Available mdevs at: set(). 2023-09-21 14:34:47.809 1901814 DEBUG nova.virt.libvirt.driver [<removed> - - default default] No available mdevs where found. Creating an new one... _allocate_mdevs /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driv er.py:8496 2023-09-21 14:34:47.809 1901814 DEBUG nova.virt.libvirt.driver [<removed> - - default default] Attempting to create new mdev... _create_new_mediated_device /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driver.py:8385 2023-09-21 14:34:48.455 1901814 INFO nova.virt.libvirt.driver [<removed> - - default default] Failed to create mdev. No free space found among the following devices: ['pci_0000_4b_03_1', … <truncated list>]. 2023-09-21 14:34:48.456 1901814 ERROR nova.compute.manager [<removed> - - default default] [instance: 2026e2a2-b17a-43ab-adcb-62a907f58b51] Instance failed to spawn: nova.exception.ComputeResourcesUnavailable: Insufficient compute resources: mdev-capable resource is not available. Once this happened, ARQ removes the mdev and cleans up. I’ve got Cyborg 2023.2 running and have a device profile like so: karl@Karls-Air ~ % openstack accelerator device profile show e2b07e11-fe69-4f33-83fc-0f9e38adb7ae +-------------+---------------------------------------------------------------------------+ | Field | Value | +-------------+---------------------------------------------------------------------------+ | created_at | 2023-09-21 13:30:05+00:00 | | updated_at | None | | uuid | e2b07e11-fe69-4f33-83fc-0f9e38adb7ae | | name | VGPU_A40-Q48 | | groups | [{'resources:VGPU': '1', 'trait:CUSTOM_NVIDIA_2235_A40_48Q': 'required'}] | | description | None | +-------------+---------------------------------------------------------------------------+ karl@Karls-Air ~ % I can see the allocation candidate: karl@Karls-Air ~ % openstack allocation candidate list --resource VGPU=1 | grep A40 | 41 | VGPU=1 | 229bf15f-5689-3d2c-b37b-5c8439ea6a71 | VGPU=0/1 | OWNER_CYBORG,CUSTOM_NVIDIA_2235_A40_48Q | karl@Karls-Air ~ % Am I missing something critical here? Because I cannot seem to figure this out… have I got a PCI address wrong, or something? Any help from the Cyborg or Nova teams would be fantastic. Thanks, Karl.

4 9

[swift][OpenStackSDK][keystone] Warnings about (my) swift endpoint URL by keystoneauth1 discovery
by Christian Rohmann 13 Nov '23

13 Nov '23

Hey openstack-discuss, we use Ceph RadosGW (Quincy release) to provide Swift as part of an OpenStack cloud. The endpoints are configured as such: > # openstack endpoint list > +----------------------------------+--------+--------------+----------------+---------+-----------+----------------------------------------------------------------------+ > | ID | Region | Service Name | Service > Type | Enabled | Interface | URL | > +----------------------------------+--------+--------------+----------------+---------+-----------+----------------------------------------------------------------------+ > | 1234567890 | region | swift | object-store | True | > public | > https://object-store.region.cloud.example.com/swift/v1/AUTH_%(tenant_id)s > | > | 2345678901 | region | swift | object-store | True | > internal | > https://object-store.region.cloud.example.com/swift/v1/AUTH_%(tenant_id)s > | > | 3456789012 | region | swift | object-store | True | > admin | > https://object-store.region.cloud.example.com/swift/v1/AUTH_%(tenant_id)s > | > so according to https://docs.ceph.com/en/latest/radosgw/keystone/#cross-project-tenant-acce…, if I am not mistaken. I can also use the openstack client to access containers: > # openstack container list > +---------------+ > | Name | > +---------------+ > | containera | > | containerb | > +---------------+ and there is no warnings or other apparent issues. But when using "project cleanup" via e.g. "openstack project cleanup --dry-run --project $PROJECT" I see multiple warnings like > Failed to contact the endpoint at > https://object-store.region.cloud.example.com/swift/v1/AUTH_f2bc4bd34567ddc… > for discovery. Fallback to using that endpoint as the base url. > but only mentioning the swift endpoint. Following the warning string this originates from the discovery method of keystoneauth1, see https://github.com/openstack/keystoneauth/blob/28048af9593740315df8d9027c3b…. Could anybody help me understand what is issue might be? Why does this warning not appear for every use of object storage / swift via openstackclient? Is there any way for me to "fix" this? Could this even be a bug? Thanks and with kind regards Christian

4 10

[kolla] [train] [cinder] Cinder issues during controller replacement
by Albert Braden 20 Oct '23

20 Oct '23

We’re replacing controllers, and it takes a few hours to build the new controller. We’re following this procedure to remove the old controller: https://docs.openstack.org/kolla-ansible/train/user/adding-and-removing-hos… After that the cluster seems to run fine on 2 controllers, but approximately 1/3 of our volumes can’t be attached to a VM. When we look at those volumes, we see this: | os-vol-host-attr:host | qde4-ctrl1.cloud.ourdomain.com@rbd-1#rbd-1 | Ctrl1 is the controller that is being replaced. Is it possible to change the os-vol-host-attr on a volume? How can we work around this issue while we are replacing controllers? Do we need to disable the API for the duration of the replacement process, or is there a better way?

6 9

[interop][ptg] Virtual Caracal vPTG Planning
by Martin Kopec 19 Oct '23

19 Oct '23

Hello everyone, here is [1] our etherpad for the 2024.1 Caracal virtual PTG. Please, add your topics there if there is anything you would like to discuss / propose ... You can also vote for time slots for our session(s), so that they fit your schedule, at [2]. I have one topic I'd like to discuss - new view on interoperability testing, see [3] and this [4] article for the background info. [1] https://etherpad.opendev.org/p/oct2023-ptg-interop [2] https://framadate.org/rNFvjKVkMnXPfObf [3] https://etherpad.opendev.org/p/openstack-interop-2.0 [4] https://superuser.openinfra.dev/articles/new-view-on-interoperability-in-op… If you have any questions, feel free to reach out to me. Thanks, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 2

[qa][ptg] Virtual Caracal vPTG Planning
by Martin Kopec 18 Oct '23

18 Oct '23

Hello everyone, here is [1] our etherpad for the 2024.1 Caracal virtual PTG. Please, add your topics there if there is anything you would like to discuss / propose ... You can also vote for time slots for our sessions so that they fit your schedule at [2]. We will most likely go with 1-hour slot per day, as they usually fit easier into everyone's schedule. The number of slots will depend on the number of topics proposed in [1]. - [1] https://etherpad.opendev.org/p/oct2023-ptg-qa [2] https://framadate.org/f26R3EcZ2BOo7r8Q Thanks, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 2

[OPENSTACK][rabbitmq] using quorum queues
by Nguyễn Hữu Khôi 17 Oct '23

17 Oct '23

Hello guys. IS there any guy who uses the quorum queue for openstack? Could you give some feedback to compare with classic queue? Thank you. Nguyen Huu Khoi

9 19

[all][infra][tact-sig] Mailman v3 migration for OpenStack mailing lists 2023-10-12
by Jeremy Stanley 12 Oct '23

12 Oct '23

On Thursday, October 12 at 15:30 UTC, the OpenDev Collaboratory systems administrators will be migrating the lists.openstack.org mailing list site from the aging Mailman 2.1.29 server to a newer Mailman 3.3.8 deployment. This maintenance window is expected to last approximately four hours. The key takeaways are as follows: * There will be an extended outage for the site while DNS is updated and configuration, subscriber lists and message archives are imported, but incoming messages should correctly queue at the sender's end and arrive at the conclusion of our work * Because this is on a new server, there are new IP addresses from which list mail will be originating: 162.209.78.70 and 2001:4800:7813:516:be76:4eff:fe04:5423 * Moderation queues will not be copied to the new server, so moderators are encouraged to process any held messages prior to the migration time in order to avoid losing them * Anyone wishing to adjust their list subscriptions, or handle moderation or list configuration after the migration, needs to use the Sign Up button on the Web page to create a new account; it will be linked to your imported roles as soon as you confirm by following a URL from an E-mail message the server sends you, and is global for all sites on the server so you only need to do this once * The software providing Web front-ends for list management and archive browsing is entirely new in Mailman v3 and therefore has a much different appearance, though we've added appropriate redirects and frozen copies of old archives in order to accommodate existing hyperlinks If you have any questions or concerns, feel free to follow up on the service-discuss mailing list or find us in the #opendev channel on the OFTC IRC network. -- Jeremy Stanley

1 2

[all][elections][ptl] Project Team Lead Election Conclusion and Results
by Tony Breeds 10 Oct '23

10 Oct '23

Thank you to the electorate, to all those who voted and to all candidates who put their name forward for Project Team Lead (PTL) in this election. A healthy, open process breeds trust in our decision making capability thank you to all those who make this process possible. Now for the results of the PTL election process, please join me in extending congratulations to the following PTLs: * Adjutant : Dale Smith * Barbican : Grzegorz Grasza * Blazar : Pierre Riteau * Cinder : Rajat Dhasmana * Cloudkitty : Rafael Weingartner * Designate : Michael Johnson * Freezer : ge cong * Glance : Pranali Deore * Heat : Takashi Kajinami * Horizon : Vishal Manchanda * Ironic : Jay Faulkner * Keystone : Dave Wilde * Kolla : Michal Nasiadka * Kuryr : Roman Dobosz * Magnum : Jake Yip * Manila : Carlos Silva * Masakari : sam sue * Murano : Rong Zhu * Neutron : Brian Haley * Nova : Sylvain Bauza * Octavia : Gregory Thiemonge * OpenStack Charms : Felipe Reyes * OpenStack Helm : Vladimir Kozhukalov * OpenStackAnsible : Dmitriy Rabotyagov * OpenStackSDK : Artem Goncharov * Puppet OpenStack : Takashi Kajinami * Quality Assurance : Martin Kopec * Solum : Rong Zhu * Storlets : Takashi Kajinami * Swift : Tim Burke * Tacker : Yasufumi Ogawa * Telemetry : Erno Kuvaja * Vitrage : Dmitriy Rabotyagov * Watcher : chen ke * Zaqar : Hao Wang * Zun : Hongbin Lu Elections: * OpenStack_Helm: https://civs1.civs.us/cgi-bin/results.pl?id=E_3cf498bb10adc5b8 Election process details and results are also available here: https://governance.openstack.org/election/ Thank you to all involved in the PTL election process, Yours Tony.

6 8

[kolla] change default domain
by Vivian Rook 10 Oct '23

10 Oct '23

I'm looking for a setting to change the default domain name from "default" to "kolla" Does such an option exist in globals.yml? Thank you! -- *Vivian Rook (They/Them)* Site Reliability Engineer Wikimedia Foundation <https://wikimediafoundation.org/>

3 3

Wsgi in openstack
by Gk Gk 30 Sep '23

30 Sep '23

Hi All, I am new to python coding and trying to understand how wsgi is implemented in nova or glance api servers. What would be the best way to understand the design here ? Are there any useful documents available over the internet to understand these ? I understand the WSGI standard. I can't figure out where the start_response functions are coded in nova-api wsgi implementation. Any useful tips are highly appreciated. Thanks so much for your time. Thanks Y.G

3 3

[release] Release countdown for week R+0, Oct 02-06
by Elõd Illés 29 Sep '23

29 Sep '23

Development Focus ----------------- We will be releasing the coordinated OpenStack 2023.2 Bobcat release next week, on Wednesday, October 4th, 2023. Thanks to everyone involved in the 2023.2 Bobcat cycle! We are now in pre-release freeze, so no new deliverable will be created until final release, unless a release-critical regression is spotted. Otherwise, teams attending the virtual PTG should start to plan what they will be discussing there, by creating and filling team etherpads. You can access the list of PTG etherpads at: http://ptg.openstack.org/etherpads.html General Information ------------------- On release day, the release team will produce final versions of deliverables following the cycle-with-rc release model, by re-tagging the commit used for the last RC. A patch doing just that will be proposed. PTLs and release liaisons should watch for that final release patch from the release team. While not required, we would appreciate having an ack from each team before we approve it on the 4th, so that their approval is included in the metadata that goes onto the signed tag. Upcoming Deadlines & Dates -------------------------- Final 2023.2 Bobcat release: October 4th, 2023 2024.1 Caracal Virtual PTG: October 23-27, 2023 Előd Illés irc: elodilles @ #openstack-release

1 0

[neutron] Neutron drivers meeting
by Rodolfo Alonso Hernandez 29 Sep '23

29 Sep '23

Hello neutrinos: Please remember today we have the Neutron drivers meeting. This is the agenda: https://wiki.openstack.org/wiki/Meetings/NeutronDrivers There are two topics: * [fwaas][rfe] support list type of port_range for firewall rule (I can't find this in the meeting logs) * [OVN] Allow scheduling external ports on non-gateway nodes See you in 2,5 hours!

1 0

[all][debian] First VM running under Bobcat pings: Debian Bobcat repository is ready!
by Thomas Goirand 29 Sep '23

29 Sep '23

Hi, As on each release, my first test is to pop a new OpenStack cluster on my visualized environment (a single machine running 38 VMs with nested virt). Today, my first VM running under Bobcat was ACTIVE and pinged. All of this is available the usual way, especially with extrepo if you want to use the unofficial Bookworm backport: apt-get install extrepo extrepo enable openstack_bobcat apt-get update The following days, I'll be uploading all of this form Debian Experimental to Unstable, trying to add as many SQLAlchemy 2.x support as I can, since the package needs to be uploaded to Unstable soon. There's also a few packages that I didn't have time to finish yet (like the last version of Horizon that has many unit test failures that I didn't have time to look on...). Thanks to everyone that contributed to Bobcat, or that was involved in this release, Cheers, Thomas Goirand (zigo)

2 1

Masakari Issue
by Karera Tony 29 Sep '23

29 Sep '23

Hello Team,, I have a cluster of 4 Compute nodes but when shutdown the VMs so that I can upgrade the kernel for the Hosts, The compute nodes in instance-ha and stuck in Maintenance mode. When I try to change them, I get the error below. Error: Failed to update host. Details ConflictException: 409: Client Error for url: http://10.10.13.31:15868/v1/segments/8d042245-5610-4b84-b611-b633f8f8367c/h…, Host 066c5654-dd1a-4fa9-a664-dad10b89e202 can't be updated as it is in-use to process notifications Regards Tony Karera

1 0

Masakari Issue
by Karera Tony 29 Sep '23

29 Sep '23

Hell oTeam, I have an openstack Wallaby Environment deployed using kolla-ansible. So, I shutdown all the VMs on the compute nodes becauseI wanted to upgrade the nodes kernel. After I rebooted them and all was fine apart from the fact that all the hosts in instance-ha segment are in maitenance mode. When I try to turn to false. I get the error below. *Error: *Failed to update host. Details <http://10.10.13.31/masakaridashboard/hosts/#message_details> ConflictException: 409: Client Error for url: http://x.x.x.x:15868/v1/segments/8d042245-5610-4b84-b611-b633f8f8367c/hosts…, Host 066c5654-dd1a-4fa9-a664-dad10b89e202 can't be updated as it is in-use to process notifications. Regards Tony Karera

1 0

[Glance] Weekly Meeting Cancelled for Today
by Pranali Deore 28 Sep '23

28 Sep '23

Hello Everyone, Cancelling today's weekly meeting, since there is nothing much on agenda for the week & few team members are also not around. See you next week ! Thanks & Regards, Pranali Deore

1 0

[all][tc] python 3.11 testing plan
by Ghanshyam Mann 27 Sep '23

27 Sep '23

Hi All, Some of you are part of discussion for python 3.11 testing but If you are not aware of it, below is the plan for python 3.11 testing in OpenStack. Non voting in 2023.2 ------------------------- You might have seen that python 3.11 job is now running as non voting in all projects[1]. Idea is to run it as non voting for this (2023.2) cycle which will give projects to fix the issue and make it green. As it is running on debian (frickler mentioned the reason of running it in debian in gerrit[2]), it need some changes in bindep.txt file to pass. Here is the example of fix[3] which you can do in your project also. Voting in 2024.1 -------------------- In next cycle (2024.1), I am proposing to make py3.11 testing mandatory [4] and voting (automatically via common python job template). You need to fix the failure in this cycle otherwise it will block the gate once the next cycle development start (basically once 891238 is merged). [1] https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/891227/5 [2] https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/891146/1 [3] https://review.opendev.org/c/openstack/nova/+/891256 [4] https://review.opendev.org/c/openstack/governance/+/891225 [5] https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/891238 -gmann

12 27

[kolla-ansible][neutron] Kolla-ansible deployment where not all network nodes have access to the same external networks
by Moonpiedumplings TheSecond 27 Sep '23

27 Sep '23

I'm currently prototyping for what will be a two node openstack install via kolla-ansible, with one part on a public vps as a network node, and another on my home server as the main compute/controller node, and an allinone node. I want to use the VPS to give virtual machines public ipv6 addresses. However, the VPS only has one ipv4 address (and I am not willing to buy more). I'm thinking the simplest way to have both ipv4 and ipv6 connectivity is just to have my "home" node be a network node as well, and then just virtual machines a ipv6 floating ip, an access to a normal, internal ipv4 subnet. However, I can't get networking working properly, as I don't know how to force openstack to create a network on a specific node. I have it halfway working, I can create a network on the equivalent of the main node in my prototypes, but not on the "vps" node. I can create a network, but not create a virtual machine on that network, or give a virtual machine a floating ip from that network. In my blog, I have documented what I've tried, but it's somewhat disorganized, as I've documented my entire process, not just my networking troubles: https://moonpiedumplings.github.io/projects/build-server-2/#asking-for-help Thanks in advance for any help you have to offer,

1 0

Cinder Bug Report 2023-09-27
by Jon Bernard 27 Sep '23

27 Sep '23

Hello Argonauts, Cinder Bug Meeting Etherpad <https://etherpad.opendev.org/p/cinder-bug-squad-meeting> Undecided - cinder reimage failure leaves the volume in downloading state <https://bugs.launchpad.net/cinder/+bug/2036994> - Status: In Progress - Backup using Swift driver fails when TLS is enabled on internal network <https://bugs.launchpad.net/cinder/+bug/2037113> - Status: New Thanks, -- Jon

1 0

[kolla-ansible] Deploying an kolla-ansible all-in-one to test enforce_scope
by Kyle Burall 27 Sep '23

27 Sep '23

I'm trying to test enabling enforce_scope on a kolla-ansible deployment in prep for rolling it out to an actual deployment and I am having issues with services registering themselves. The playbook fails at registering nova services and the keytsone log says the nova user doesn't exist, as well as the service project and the admin role. I've added enforce_scope and enforce_new_defaults to global.conf, which causes the referenced error. I assume that I'm missing something in the configuration that is required beyond just setting those two oslo_policy options, but I haven't been able to find anything beyond these two settings. Can anyone let me know if there's something specific that I'm missing? Thanks! This email and its attachments, if any, may contain information that is proprietary, business-confidential, and/or legally protected. If you received this communication in error, please notify me by reply message and delete the email and any attachments without copying, distributing, or otherwise using.

1 0

[openstack][nova][cinder] how can we know instance disk usage
by Nguyễn Hữu Khôi 27 Sep '23

27 Sep '23

Hello guys. I am try to get instance disk usage but I cannot. I try to use virsh domblkinfo instance-00000df8 vda but it did not show exactly disk allocation How we get this metric.I used SAN and Ceph as backend storages. Thank you so much, . Nguyen Huu Khoi

1 0

[magnum] Weekly meeting today (27th Sep 2023) cancelled
by Michał Nasiadka 27 Sep '23

27 Sep '23

Hello, Since key members are absent - let’s cancel this weeks (27th Sep 2023) meeting. Best regards, Michal

1 0

RE: openstack- Deployment through kolla ansible
by Asma Naz Shariq 27 Sep '23

27 Sep '23

Hi all, I have deployed openstack through kolla ansible deployment tool and i want to know that how to add firewall as a service in openstack release 2023.1 (antelope) and how to add bgp router plugins in openstack release 2023.1? please anyone guide. -----Original Message----- From: openstack-discuss-request(a)lists.openstack.org <openstack-discuss-request(a)lists.openstack.org> Sent: Tuesday, September 26, 2023 9:08 PM To: openstack-discuss(a)lists.openstack.org Subject: openstack-discuss Digest, Vol 59, Issue 97 Send openstack-discuss mailing list submissions to openstack-discuss(a)lists.openstack.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss or, via email, send a message with subject or body 'help' to openstack-discuss-request(a)lists.openstack.org You can reach the person managing the list at openstack-discuss-owner(a)lists.openstack.org When replying, please edit your Subject line so it is more specific than "Re: Contents of openstack-discuss digest..." Today's Topics: 1. Re: [openstack][neutron] OpenStack deployment (Satish Patel) 2. [tc][all] Please welcome Jay Faulkner as new Chair of Technical Committee (Nikolla, Kristi) 3. Re: [tc][all] Please welcome Jay Faulkner as new Chair of Technical Committee (Rodolfo Alonso Hernandez) 4. Re: [release][requirements][TC][oslo] how to manage divergence between runtime and doc? (Ghanshyam Mann) ---------------------------------------------------------------------- Message: 1 Date: Tue, 26 Sep 2023 08:06:47 -0400 From: Satish Patel <satish.txt(a)gmail.com> To: Dhanasekar Kandasamy <dhana.sys(a)gmail.com> Cc: openstack-discuss(a)lists.openstack.org, skaplons(a)redhat.com Subject: Re: [openstack][neutron] OpenStack deployment Message-ID: <0695D533-7F6D-4B95-8148-E5A2336C8649(a)gmail.com> Content-Type: text/plain; charset="utf-8" Hi, I would never deploy dedicated network node until I?m going to pumps gigs and gigs of traffic in/out. Anyway in DVR ( you can run network node function on each compute nodes) I would go with OVN deployment and skip network nodes. Just controller and compute. If I?m future you can easily add dedicated network node and distribute traffic. If you go with VLAN based provider then you really don?t need network node. Sent from my iPhone > On Sep 25, 2023, at 9:43 AM, Dhanasekar Kandasamy <dhana.sys(a)gmail.com> wrote: > > ? > Hi, > > I planning to deploy OpenStack for production zed release, OVS with > DVR when I went through the reference architectures I can see > 3 controller, 3 network and x compute nodes or > 3 controllers + network and x computes nodes I want to understand what > happens if I go with the below configuration OVS with DVR > 3 controller node > x compute + network nodes > For example I have 13 nodes, 3 controllers, 10 compute + network nodes > what is the advantage/disadvantage for this configuration > > Thanks & Regards, > Dhana

1 0

[tc] Technical Committee next weekly meeting Today on September 26, 2023
by Nikolla, Kristi 26 Sep '23

26 Sep '23

Hi all, This is a reminder that the next weekly Technical Committee meeting is to be held Today on Tuesday, 26 September 2023 at 1800 UTC on #openstack-tc on OFTC IRC. Please find below the agenda for the meeting: • Roll call • Follow up on past action items • No action items from Sept 19, 2023 meeting. • Gate health check • Leaderless projects • Call for volunteers for Vice-Chair • Open Discussion and Reviews • Register for the PTG • #link https://openinfra.dev/ptg/ • #link https://review.opendev.org/q/projects:openstack/governance+is:open Thank you, Kristi Nikolla

1 0

[release][requirements][TC][oslo] how to manage divergence between runtime and doc?
by Herve Beraud 26 Sep '23

26 Sep '23

We currently face a big issue. An issue which could have huge impacts on the whole Openstack and on our customers. By customers I mean all the users outside the upstream community, operators, distros maintainers, IT vendors, etc. Our problem is that, for a given series, Bobcat in our case, there is divergence between the versions that we announce as supported to our customers, and the versions really supported in our runtime. Let me describe the problem. The oslo.db's versions supported within Bobcat's runtime [1] doesn't reflect the reality of the versions really generated during Bobcat [2]. In Bobcat's upper-constraints, oslo.db 12.3.2 [1] is the supported version. This version corresponds in reality to the last version generated during 2023.1/antelope [3]. All the versions of oslo.db generated during Bobcat are for now ignored by our runtime. However all these generated versions are all listed in our technical documentation as supported by Bobcat. In fact, the problem is that these oslo.db versions are all stuck in their upper-constraints upgrade, because some cross-jobs failed and so the upper-constraints update can't be made. These cross-job are owned by different services (heat, manila, masakari, etc). We update our technical documentation each time we produce a new version of a deliverable, so before upgrading the upper-constraints. This is why the listed versions diverge from the versions really supported at runtime. We also face a similar issue with Castellan, but in the sake of clarity of description of this problem I'll focus on oslo.db's case during the rest of this thread. >From a quantitative point of view, we face this kind of problem, from a consecutive manner, since 2 series. It seems now that this becomes our daily life with each new series of openstack. . At this rate it is very likely that we will still be faced with this same problem during the next series. Indeed, during antelope, the same issue was thrown but only within one deliverable [4][5][6]. With Bobcat this scenario reappears again but now within two deliverables. The more the changes made in libraries are important, the more we will face this kind of issues again, and as everybody knows our libraries are all based on external libraries who could evolve toward new major releases with breaking changes. That was the case oslo.db where our goal was to migrate toward sqlalchemy 2.x. Leading to stuck upper-constraints. This problem could also impact all the downstream distros. Some distros already started facing issues [7] with oslo.db's case. We can't exclude that a similar issue will start to appear soon within all the Openstack deliverables listed in upper-constraints. Oslo's case is the first fruit. >From a quality point of view, we also face a real issue. As customers can establish their choices and their decisions on our technical documentation, a divergence between officially supported versions and runtime supported versions can have huge impacts for them. Imagine they decide to install a specific series led by imposed requirements requested by a government, that can be really problematic. By reading our technical documentation and our release notes, they can think that we fulfill those prerequisites. This kind of government requirement often arrives. It can be requested for a vendor who wants to be allowed to sell to a government, or to be allowed to respect some specific IT laws in a given country. This last point can completely undermine the quality of the work carried out upstream within the Openstack community. So, now, we have to find the root causes of this problem. In the current case, we would think that the root cause lives in the complexity of oslo.db migration, yet this is not the case. Even if this migration represents a major change in Openstack, it has been announced two year ago [8] - the equivalent of 4 series -, leaving a lot of time for every team to adopt the latest versions of oslo.db and sqlalchemy 2.x. Stephen Finucane and Mike Bayer have spent a lot of time on this topic. Stephen even contributed well beyond the oslo world, by proposing several patches to migrate services [9]. Unfortunately a lot of these patches remain yet unmerged and unreviewed [10], which has led us to this situation. This migration is therefore by no means the root cause of this problem. The root cause of this problem lurks in the volume of maintenance of services. Indeed the main cause of this issue is that some services are not able to follow the cadence, and therefore they slow down libraries' evolutions and maintenance. Hence, their requirements cross job reflect this fact [11]. This lack of activity is often due to the lack of maintainers. Fortunately Bobcat has been rescued by Stephen's recent fixes [12][13]. Stephen's elegant solution allowed us to solve failing cross jobs [14] and hence, allowed us to resync our technical documentation and our runtime. However, we can't ignore that the lack of maintainers is a growing trend within Openstack. As evidenced by the constant decrease in the number of contributors from series to series [15][16][17][18]. This phenomenon therefore risks becoming more and more amplified. So, we must provide a lasting response. A response more based on team process than on isolated human resources. A first solution could be to modify our workflow a little. We could update our technical documentation by triggering a job with the upper-constraints update rather than with a new release patch. Hence, the documentation and the runtime will be well aligned. However, we should notice that not all deliverables are listed in upper-constraints, hence this is a partial solution that won't work for our services. A second solution would be to monitor teams activity by monitoring the upper-constraints updates with failing cross-job. That would be a new task for the requirements team. The goal of this monitoring would be to inform the TC that some deliverables are not active enough. This monitoring would be to analyze, at defined milestones, which upper-constraints update remains blocked for a while, and then look at the cross-job failing to see if it is due to a lack of activity from the service side. For example by analyzing if patches, like those proposed by Stephen on services, remain unmerged. Then the TC would be informed. It would be a kind of signal addressed to the TC. Then the TC would be free to make a decision (abandoning this deliverable, removing cross-job, put-your-idea-here). The requirements team already provides such great job and expertise. Without them we wouldn't have solved the oslo.db and castellan case in time. However, I think we lack of aTC involvement a little bit earlier in the series to avoid fire fighter moments. The monitoring would officialize problems with deliverables sooners in the life cycle and would trigger a TC involvement. Here is the opportunity for us to act to better anticipate the growing phenomenon of lack of maintainers. Here is the opportunity for us to better anticipate our available human resources. Here is the opportunity for us to better handle this kind of incident in the future. Thus, we could integrate substantive actions in terms of human resources management into the life cycle of Openstack. It is time to manage this pain point, because in the long term, if nothing is done now, this problem will repeat itself again and again. Concerning the feasibility of this solution, the release team already created some similar monitoring. This monitoring is made during each series at specific milestones. The requirements team could trigger its monitoring at specific milestones targets, not too close to the series deadline. Hence we would be able to anticipate decisions. The requirements team could inspire from the release management process [19] to create their own monitoring. We already own almost the things we need to create a new process dedicated to this monitoring. Hence, this solution is feasible. The usefulness of this solution is obvious. Indeed, thus the TC would have better governance monitoring. A monitoring not based on people elected as TC members but based on process and so transmissible from a college to another. Therefore, three teams would then work together on the topic of decreasing activity inside teams. >From a global point of view, this will allow Openstack to more efficiently keep pace with the resources available from series to series. I would now like to special thank Stephen for his investment throughout these two years dedicated to the oslo.db migration. I would especially like to congratulate Stephen for the quality of the work carried out. Stephen helped us to solve the problem in an elegant manner. Without his expertise, delivering Bobcat would have been really painful. However, we should not forget that Stephen remains a human resource of Openstack and we should not forget that his expertise could go away from Openstack one day or one other. Solving this type of problem cannot only rest on the shoulders of one person. Let's take collective initiatives now and put in place safeguards. Thanks for your reading and thanks to all the people who helped with this topic and that I have not cited here. I think other solutions surely coexist and I'll be happy to discuss this topic with you. [1] https://opendev.org/openstack/requirements/src/branch/master/upper-constrai… [2] https://releases.openstack.org/bobcat/index.html#bobcat-oslo-db [3] https://opendev.org/openstack/releases/src/branch/master/deliverables/antel… [4] https://review.opendev.org/c/openstack/requirements/+/873390 [5] https://review.opendev.org/c/openstack/requirements/+/878130 [6] https://opendev.org/openstack/oslo.log/compare/5.1.0...5.2.0 [7] https://lists.openstack.org/pipermail/openstack-discuss/2023-September/0351… [8] https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.… [9] https://review.opendev.org/q/topic:sqlalchemy-20 [10] https://review.opendev.org/q/topic:sqlalchemy-20+status:open [11] https://review.opendev.org/c/openstack/requirements/+/887261 [12] https://opendev.org/openstack/oslo.db/commit/115c3247b486c71317613942264714… [13] https://opendev.org/openstack/oslo.db/commit/4ee79141e601482fcde02f0cecfb56… [14] https://review.opendev.org/c/openstack/requirements/+/896053 [15] https://www.openstack.org/software/ussuri [16] https://www.openstack.org/software/victoria [17] https://www.openstack.org/software/xena [18] https://www.openstack.org/software/antelope/ [19] https://releases.openstack.org/reference/process.html#between-milestone-2-a… -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

2 2

[tc][all] Please welcome Jay Faulkner as new Chair of Technical Committee
by Nikolla, Kristi 26 Sep '23

26 Sep '23

Hi all, Please join me in congratulating Jay Faulkner (JayF) as the new Chair of the OpenStack Technical Committee. Thank you for stepping up and thank you for your amazing contributions on the Technical Committee as a valued member and as my vice-chair for the past cycle. Kristi Nikolla

2 1

[openstack][neutron] OpenStack deployment
by Dhanasekar Kandasamy 26 Sep '23

26 Sep '23

Hi, I planning to deploy OpenStack for production zed release, OVS with DVR when I went through the reference architectures I can see - 3 controller, 3 network and x compute nodes or - 3 controllers + network and x computes nodes I want to understand what happens if I go with the below configuration OVS with DVR - 3 controller node - x compute + network nodes For example I have 13 nodes, 3 controllers, 10 compute + network nodes what is the advantage/disadvantage for this configuration Thanks & Regards, Dhana

2 1

[oslo][heat][requirements] RFE requested for oslo.messaging
by Herve Beraud 26 Sep '23

26 Sep '23

Hey Requirements Team, The previous bump of oslo.messaging (14.4.0) remains blocked [1] by a recent oslo.messaging change [2] that was blocked by heat's cross testing jobs. This issue is now fixed on the oslo.messaging side [3], and in the way to be released [4]. Hence a new requirements update will arrive soon on the requirements side. This new version should fix the heat compatibility issue, thus, this will allow us to resync upper-constraints [5] and listed releases [6] before bobcat's final release. [1] https://review.opendev.org/c/openstack/requirements/+/892919 [2] https://review.opendev.org/c/openstack/oslo.messaging/+/891096 [3] https://review.opendev.org/c/openstack/oslo.messaging/+/896422 [4] https://review.opendev.org/c/openstack/releases/+/896498 [5] https://opendev.org/openstack/requirements/src/branch/master/upper-constrai… [6] https://releases.openstack.org/bobcat/index.html#bobcat-oslo-messaging Thank you for considering this request. -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

1 0

[openstack-ansible][manila deployment] error: No valid host was found
by Danish Khan 25 Sep '23

25 Sep '23

Dear Team, I have deployed Manila in openstack-ansible and I have created share type and share network. But I am still unable to create a share in it. Below is the journalctl error from manila container : Sep 24 16:45:13 aio1-manila-container-5cce4ee7 manila-scheduler[69]: 2023-09-24 16:45:13.689 69 ERROR manila.scheduler.manager [req-13a8ec7b-2b66-452b-964b-f531b75b699b c0806f7a8b534149a72931b84f7d60af 61c6349cbba74db88d14d3b7348de585 - - -] Failed to schedule create_share: No valid host was found. Failed to find a weighted host, the last executed filter was AvailabilityZoneFilter.: manila.exception.NoValidHost: No valid host was found. Failed to find a weighted host, the last executed filter was AvailabilityZoneFilter. And below is the error of manila-share.service from controller node: Sep 24 16:44:24 aio1 manila-share[250518]: 2023-09-24 16:44:24.016 250518 CRITICAL manila [req-3c0e428b-226b-4bff-ade1-3d4ac5befb99 - - - - -] Unhandled error: manila.exception.ManilaException: Config opt 'driver_handles_share_servers' has improper value - 'None'. Please define it as boolean. Note: manila-share service is continueously giving this error even if there is no resources of manila(even in the setup). I am not sure where I am supposed to define driver_handles_share_servers defautl value if there is no share type. Thanks in advance. Regards, Danish

2 1

[neutron] Bug deputy report (week starting on September 18)
by Bernard Cafarelli 25 Sep '23

25 Sep '23

Hey neutrinos, time for a new bug deputy rotation again! Here are the bugs reported between 2023-09-18 and 2023-09-24 Quite a few bugs have patches in progress. For the rest, some failures on neutron-tempest-plugin-openvswitch tempest jobs in the first 2 bugs listed below, possibly a design question for https://bugs.launchpad.net/neutron/+bug/2036705 <https://bugs.launchpad.net/neutron/+bug/2036705> (different behaviour in ML2/OVN), one RBAC bug on tags in https://bugs.launchpad.net/neutron/+bug/2037002 and 2 other unassigned bugs https://bugs.launchpad.net/neutron/+bug/2037107 and https://bugs.launchpad.net/neutron/+bug/2036877 Critical * neutron-tempest-plugin-openvswitch-* jobs randomly failing in gate - https://bugs.launchpad.net/neutron/+bug/2037239 Just reported by haleyb - tests failing with "x is not active on any of the L3 agents" High * [tempest] Test "test_multicast_between_vms_on_same_network" radonmly failing - https://bugs.launchpad.net/neutron/+bug/2036603 Unassigned. Random recent failure in that test * Neutron tempest plugin zuul job definitions uses deprecated regex syntax - https://bugs.launchpad.net/neutron/+bug/2037034 Fix by ykarel: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/896188 * neutron-ovn-metadata-agent dies on broken namespace - https://bugs.launchpad.net/neutron/+bug/2037102 Not a common case but it can keep the agent down until fixed Patch by felix.huettner: https://review.opendev.org/c/openstack/neutron/+/896251 New * Reader can update object tag - https://bugs.launchpad.net/neutron/+bug/2037002 *_reader roles were not cooperating when I tried to reproduce this in devstack - needs another set of eyes Medium * Neutron fails to respawn radvd due to corrupt pid file - https://bugs.launchpad.net/neutron/+bug/2033980 When radvd crashes (it can happen like in cases described by users in LP), neutron does not respawn it properly - it works fine when we are restarting/killing it Patch by haleyb: https://review.opendev.org/c/openstack/neutron/+/895832 * [OVN] The API worker fails during "post_fork_initialize" call - https://bugs.launchpad.net/neutron/+bug/2036607 Enhancement to better handle restarts (as tested by tobiko). Patches by ralonsoh: https://review.opendev.org/c/openstack/neutron-lib/+/895940 https://review.opendev.org/c/openstack/neutron/+/895946 https://review.opendev.org/c/openstack/neutron/+/896009 * [ovn-octavia-provider] Fix issue when LRP has more than one address - https://bugs.launchpad.net/neutron/+bug/2036620 Fix by froyo - https://review.opendev.org/c/openstack/ovn-octavia-provider/+/895826 * A port that is disabled and bound is still ACTIVE with ML2/OVN - https://bugs.launchpad.net/neutron/+bug/2036705 Unassigned. Different behaviour compared to ML2/OVS (where status is down) * [pep8] Pylint error W0105 (pointless-string-statement) in random CI executions - https://bugs.launchpad.net/neutron/+bug/2036763 Random failure, fix by ralonsoh: https://review.opendev.org/c/openstack/neutron/+/895975 * radvd seems to crash when ipv4 addresses are supplied as nameservers to ipv6 subnets - https://bugs.launchpad.net/neutron/+bug/2036877 Unassigned. Follow-up enhancement to https://bugs.launchpad.net/neutron/+bug/2033980 to prevent adding IPv4 servers to radvd config * slow port creation with a large amount of networkrbacs - https://bugs.launchpad.net/neutron/+bug/2037107 Unassigned. We pushed an enhancement for similar slowness in https://bugs.launchpad.net/neutron/+bug/1918145 but not fixing this case. sqlalchemy/SQL experts welcome there Incomplete * subnet's gateway ip can be unset while attached to router - https://bugs.launchpad.net/neutron/+bug/2036423 Regards, -- Bernard Cafarelli

1 0

[kolla-ansible][keystone][skyline] Keystone use SSO for authentication
by Nguyễn Hữu Khôi 24 Sep '23

24 Sep '23

Hello guys. I want to share a lab to help newbies understand how setup Keystone and Skyline to use Google SSO for authentication, https://github.com/ngyenhuukhoi/openstack-note/blob/main/skyline%20with%20s… I will try to contribute what I can for newbies and community. Nguyen Huu Khoi

3 2

[aodh][mistral][solum][release][requirements] Feature freeze exception for wsme, sphinxcontrib-pecanwsme
by Stephen Finucane 22 Sep '23

22 Sep '23

o/ I'd like to request a feature freeze exception for the aforementioned packages to include fixes we recently merged: * WSME 0.12.1 includes two fixes to the Sphinx extension to fix compatibility with recent Sphinx versions. There are also additional fixes to address compatibility with Python 3.11 (by switching from nose to pytest, by replacing use of pkg_utils with importlib.metadata, and by replacing use of inspect.getargspec with inspect.getfullargspec). * sphinxcontrib-pecanwsme 0.11.0 includes a fix for Python 3.11 compatibility (replacing use of inspect.getargspec with inspect.getfullargspec), drops support for Python < 3.8, and removes the use of six. I've included a summary of the git logs below. I would argue that both version bumps are low-risk and address issues we're seeing in the gates of multiple projects. I would also argue that bumping these versions is a lower risk strategy than the alternative proposal, namely the reintroduction of a cap on the Sphinx version used to a version released nearly 18 months ago [2] (!!). Finally, I would argue that this would be the preferred approach by our packagers (zigo, are you there?). The change is available [3] and waiting for reviews. Cheers, Stephen PS: Why WSME 0.12.1 rather than 0.12.0, you ask? You can blame a typo [4] for that 🙈 [1] https://lists.openstack.org/pipermail/openstack-discuss/2023-September/0351… [2] https://pypi.org/project/Sphinx/4.5.0/ [3] https://review.opendev.org/c/openstack/requirements/+/896195 [4] https://review.opendev.org/c/x/wsme/+/896153

2 1

[release] Release countdown for week R-1 Sept 25-29
by Herve Beraud 22 Sep '23

22 Sep '23

Development Focus ----------------- We are on the final mile of the 2023.2 Bobcat development cycle! Remember that the 2023.2 Bobcat final release will include the latest release candidate (for cycle-with-rc deliverables) or the latest intermediary release (for cycle-with-intermediary deliverables) available. September 28, 2023 is the deadline for final 2023.2 Bobcat release candidates as well as any last cycle-with-intermediary deliverables. We will then enter a quiet period until we tag the final release on October 4, 2023. Teams should be prioritizing fixing release-critical bugs, before that deadline. Otherwise it's time to start planning the 2024.1 Caracal development cycle, including discussing Forum and PTG sessions content, in preparation of the 2024.1 Caracal Virtual PTG (October 23-27, 2023). Actions ------- Watch for any translation patches coming through on the stable/2023.2 branch and merge them quickly. If you discover a release-critical issue, please make sure to fix it on the master branch first, then backport the bugfix to the stable/2023.2 branch before triggering a new release. Please drop by #openstack-release with any questions or concerns about the upcoming release ! Upcoming Deadlines & Dates -------------------------- Final 2023.2 Bobcat release: October 4, 2023 2024.1 Caracal Virtual PTG: October 23-27, 2023 -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

1 0

[Nova][Neutron] Help troubleshooting metadata service error 404 (Ussuri)
by Jérôme BECOT 22 Sep '23

22 Sep '23

Hello, I am operating several OpenStack Ussuri clouds and we lately have random troubles with the metadata service. We run all our networks as provider networks. The metadata service works fine for months some hosts can't get their metadata with a 404 error I restarted nova-api, neutron-dhcp-agent and neutron-metadata-agent on all controllers but we still experience 404 on random nodes: ssh 10.60.51.130 $ ip r s ... 169.254.169.254 via 10.60.51.121 dev eth0 $ curl -s http://169.254.169.254 <html> <head> <title>404 Not Found</title> </head> <body> <h1>404 Not Found</h1> The resource could not be found.<br /><br /> </body> </html> ssh 10.60.51.131 $ ip r s ... 169.254.169.254 via 10.60.51.121 dev eth0 $ curl -s http://169.254.169.254 {"uuid": "ee6e8dea-796b-4a56-bb1e-1c38f4ac9030", "meta": {"Created By": "terraform"}...} $ grep "10.60.51.131" /var/log/neutron/neutron-metadata-agent.log 2023-09-22 12:41:10.687 21954 INFO eventlet.wsgi.server [-] 10.60.51.131,<local> "GET /openstack/latest/meta_data.json HTTP/1.1" status: 200 len: 2729 time: 0.3817778 $ grep "10.60.51.130" /var/log/neutron/neutron-metadata-agent.log $ grep "10.60.51.130" /var/log/nova/nova-api.log # on all controllers $ grep "10.60.51.131" /var/log/nova/nova-api.log # on all controllers 2023-09-22 12:41:10.682 57433 INFO nova.metadata.wsgi.server [req-1d222a75-ccb4-49bc-8b17-2e2e81dc5f9b - - - - -] 10.60.51.131,1.2.3.4 "GET /openstack/latest/meta_data.json HTTP/1.1" status: 200 len: 2710 time: 0.2587621 Can you help me to troubleshoot this behaviour for some hosts ? Regards -- *Jérôme B*<https://www.deveryware.com>

2 5

Problem with Barbican
by Derek O keeffe 22 Sep '23

22 Sep '23

Hi all, Hopefully someone may be able to give me some info/advice on this one. I've been asking in the IRC chat but unfortunately we couldn't get to the bottom of it, I'm hoping someone here may not be in the chat but able to help :) So I have an Openstack AIO that I'm trying to test Barbican on with a Thales Luna HSM. I get through the instructions to the point where I successfully create the HMAK and MKEK keys. After this I try the following commands: openstack secret store --name mysecret1 --payload testPayload openstack volume create --size 1 --type LUKS 'encrypted volume' Both of those fail and the logs from the Barbican container are here: Paste #bU1E0joDdf4eWiV8wjlj | LodgeIt! | | | | Paste #bU1E0joDdf4eWiV8wjlj | LodgeIt! | | | I have made sure that "library_path: /opt/barbican/libs/libCryptoki2.so in Barbican.conf actually exists so I have no idea really where to look after this, I can't see any info as to where Barbican is trying to find the crypto plugin or maybe do I need to install something else manually?? Anyway I hope someone might have some useful info or advice to help and thanks all for reading. Regards,Derek

2 1

[neutron] Thanks for the Bobcat release (and now let's continue working in Caracal)
by Rodolfo Alonso Hernandez 22 Sep '23

22 Sep '23

Hello Neutrinos: Before stepping down as Neutron PTL, I would like to thank you for your commitment and work done during the last two releases. Let's keep Neutron, the stadium projects and OpenStack in general healthy and growing every day. We are now in the RC phase and we should be focused on delivering the best possible code for Bobcat cycle. So far I think we are going to deliver a good quality release. And in advance for the next cycle, let me bring here the hot topics we have right now in gerrit. I would ask you, as I've done during the last year, to spend some time reviewing them in order to continue the development of these features. The sooner we merge them in the Caracal cycle, the better we'll test it during the release development period. Please remember this list is just a brief summary of the most relevant features under review. ** Specs ** https://review.opendev.org/c/openstack/neutron-specs/+/891204: Add spec for partial support for OVN Interconnect RFE ** RFEs and important bugs ** https://review.opendev.org/q/topic:neutron_ovn_l3_scheduler: a series of patches related to LP#2023993, that will improve the OVN L3 scheduler. https://review.opendev.org/c/openstack/neutron/+/894026: support for IPv6 in the OVN metadata, closing a present gap between ML2/OVS and ML2/OVN. https://review.opendev.org/c/openstack/neutron/+/882832: provide a new parameter to define the port HW offload capability, instead of writing directly on the port binding profile. https://review.opendev.org/q/topic:distributed_metadata_data_path: a series of patches for ML2/OVS, to provide distributed metadata capabilities. https://review.opendev.org/q/topic:2023-aa-l3-gw-multihoming: a series of patches to provide to ML2/OVN multihoming (active-active) L3 GW capabilities. Regards.

1 0

[neutron] Neutron drivers meeting cancelled
by Rodolfo Alonso Hernandez 22 Sep '23

22 Sep '23

Hello Neutrinos: Due to the lack of agenda [1], today's meeting is cancelled. Have a nice weekend. [1]https://wiki.openstack.org/wiki/Meetings/NeutronDrivers

1 0

Requirements FFE to return Sphinx to known working version
by Erno Kuvaja 21 Sep '23

21 Sep '23

Hi all, I'm writing to request FFE for https://review.opendev.org/c/openstack/requirements/+/896018 Revert "Bump Sphinx to the latest version" as it broke the gate of at least following projects: Aodh https://zuul.opendev.org/t/openstack/build/5df111c602944396bf6e334e1bec4e42 Solum https://zuul.opendev.org/t/openstack/build/e4b170cc25f2428caadce8870d27c62e Mistral https://zuul.opendev.org/t/openstack/build/a3e00a6312cf46059fa81c56fe9b5e11 Stephen kindly provided monkeypathing for Aodh as alternative https://review.opendev.org/c/openstack/aodh/+/896125 With kind regards that we shouldn't revert this breaking patch due to it being breaking via wsme dependency that is apparently not really maintained anymore. I'd be easily agreeing with that if Aodh was the only project suffering the breakage or using it at all, but the lib is in use for about dozen openstack projects and this dependency bump broke the gates for three of them. Other alternative for this is that wsme would get released (the fix is merged already) and we would bump that in the upper constraints. I would personally prefer any other option than needing to monkeypatch/vendor the wsme code in the projects long enough that we can find a solution to remove it as dependency across. Thanks for looking into this, Erno "jokke" Kuvaja

2 1

[ironic] [ptl] Announcement of non-candidacy for D / 2024.2
by Jay Faulkner 21 Sep '23

21 Sep '23

Hi all, I was just re-elected (thank you!) for my third team as Ironic PTL. By the time the D series starts, it'll be 18 months serving as PTL of the Ironic team. I believe it may cause harm to our project to have the PTL position stay on one individual for too long. For that reason, I'm announcing my non-candidacy for the next PTL election, for the "D" cycle, 2024.2. Ironic contributors who have an interest in being PTL, please reach out to me -- I'd be happy to work with you to ensure there is a good transition. Thanks, Jay Faulkner Ironic PTL

1 0

[all][elections][tc] Technical Committee Election Results
by Tony Breeds 21 Sep '23

21 Sep '23

Please join me in congratulating the 4 newly elected members of the Technical Committe (TC). Dan Smith (dansmith) Jens Harbott (frickler) Ghanshyam Mann (gmann) Jay Faulkner (JayF) Full results: https://civs1.civs.us/cgi-bin/results.pl?id=E_36ebf2eba022aded Election process details and results are also available here: https://governance.openstack.org/election/ Thank you to all of the candidates, having a good group of candidates helps engage the community in our democratic process. Thank you to all who voted and who encouraged others to vote. We need to ensure your voice is heard. Thank you for another great round. Yours Tony.

3 2

Cyborg nova reports mdev-capable resource is not available
by Karl Kloppenborg 21 Sep '23

21 Sep '23

Hi Cyborg Team! Karl from Helm Team. When creating a VM with the correct flavor, the mdev gets created by cyborg agent and I can see it in the nodedev-list --cap mdev. However Nova then fails with: nova.virt.libvirt.driver [<removed>- - default default] Searching for available mdevs... _get_existing_mdevs_not_assigned /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driver.py :8357 2023-09-21 14:34:47.808 1901814 INFO nova.virt.libvirt.driver [<removed> - - default default] Available mdevs at: set(). 2023-09-21 14:34:47.809 1901814 DEBUG nova.virt.libvirt.driver [<removed> - - default default] No available mdevs where found. Creating an new one... _allocate_mdevs /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driv er.py:8496 2023-09-21 14:34:47.809 1901814 DEBUG nova.virt.libvirt.driver [<removed> - - default default] Attempting to create new mdev... _create_new_mediated_device /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driver.py:8385 2023-09-21 14:34:48.455 1901814 INFO nova.virt.libvirt.driver [<removed> - - default default] Failed to create mdev. No free space found among the following devices: ['pci_0000_4b_03_1', … <truncated list>]. 2023-09-21 14:34:48.456 1901814 ERROR nova.compute.manager [<removed> - - default default] [instance: 2026e2a2-b17a-43ab-adcb-62a907f58b51] Instance failed to spawn: nova.exception.ComputeResourcesUnavailable: Insufficient compute resources: mdev-capable resource is not available. Once this happened, ARQ removes the mdev and cleans up. I’ve got Cyborg 2023.2 running and have a device profile like so: karl@Karls-Air ~ % openstack accelerator device profile show e2b07e11-fe69-4f33-83fc-0f9e38adb7ae +-------------+---------------------------------------------------------------------------+ | Field | Value | +-------------+---------------------------------------------------------------------------+ | created_at | 2023-09-21 13:30:05+00:00 | | updated_at | None | | uuid | e2b07e11-fe69-4f33-83fc-0f9e38adb7ae | | name | VGPU_A40-Q48 | | groups | [{'resources:VGPU': '1', 'trait:CUSTOM_NVIDIA_2235_A40_48Q': 'required'}] | | description | None | +-------------+---------------------------------------------------------------------------+ karl@Karls-Air ~ % I can see the allocation candidate: karl@Karls-Air ~ % openstack allocation candidate list --resource VGPU=1 | grep A40 | 41 | VGPU=1 | 229bf15f-5689-3d2c-b37b-5c8439ea6a71 | VGPU=0/1 | OWNER_CYBORG,CUSTOM_NVIDIA_2235_A40_48Q | karl@Karls-Air ~ % Am I missing something critical here? Because I cannot seem to figure this out… have I got a PCI address wrong, or something? Any help from the Cyborg or Nova teams would be fantastic. Thanks, Karl.

1 0

[octavia] state of taskflow jobboards
by Maximilian Stinsky 21 Sep '23

21 Sep '23

Greetings, I would be interested in the state of the taskflow jobboard feature in octavia. [1] states it still is an experimental feature, but in the config section [2] it is not marked as such. If I look into the code I see that the feature is implemented since the U release. So my question would be is the taskflow jobboard feature still in an experimental state or is it considered production ready? If its still experimental is there any ETA when its expected to be production ready? Thanks in advance Maximilian [1] https://docs.openstack.org/octavia/latest/install/install-amphorav2.html [2] https://docs.openstack.org/octavia/latest/configuration/configref.html#task…

2 1

[ptls][Bobcat] OpenInfra Live: OpenStack Bobcat
by Kristin Barrientos 21 Sep '23

21 Sep '23

Hi everyone, As we get closer to the OpenStack release, I wanted to reach out to see if any PTL’s were interested in providing their Bobcat cycle highlights in an OpenInfra Live[1] episode on Thursday, October 5 at 1400 UTC. Ideally, we would get 4-6 projects represented. Previous examples of OpenStack release episodes can be found here[2]and here[3]. Please let me know if you’re interested by Monday, September 25 and I can provide the next steps. If you would like to provide a project update but that time doesn’t work for you, please share a recording with me and I can get it added to the project navigator. Thanks, Kristin Barrientos Marketing Coordinator OpenInfra Foundation [1] openinfra.dev/live [2] https://www.youtube.com/watch?v=MSbB3L9_MeY [3] https://www.youtube.com/watch?v=YdLTUTyJ1eU Thanks, Kristin Barrientos Marketing Coordinator OpenInfra Foundation

6 5

[all] [diversity] Call for Outreachy mentors & projects
by Dmitry Tantsur 21 Sep '23

21 Sep '23

Hi teams, Your humble Outreachy coordinators would like to invite you to submit projects for the winter 23/24 cohort! Outreachy provides internships to people subject to systemic bias and impacted by underrepresentation in the technical industry where they are living. See https://www.outreachy.org/ for more details. The deadline for submission is Sep 29th, but please don't wait for the last moment since we need time to review your submissions and suggest potential improvements. Follow https://www.outreachy.org/communities/cfp/openstack/ or reach out to Mahati or myself if you have any questions. NOTE: this is not a call for interns yet. We will notify you separately. Thank you and good luck! Dmitry OpenStack community coordinator in Outreachy P.S. The Metal3 community, adjacent to Ironic, is also looking for projects and mentors: https://www.outreachy.org/communities/cfp/metal3/ Reach out to me if you're interested.

1 0

OpenInfra Live - Sept. 21 at 9 a.m. CT / 1400 UTC
by Kristin Barrientos 20 Sep '23

20 Sep '23

Hi everyone, This week’s OpenInfra Live episode is brought to you by the OpenStack Large Scale SIG. Episode: Large Scale Ops Deep Dive: NIPA Cloud In the "Large Scale Ops Deep Dive" series, a panel of OpenStack operators invites special guests to talk about their deployment and discuss their operations. For this episode, our guests will be Dr. Abhisak Chulya and Charnsilp Chinprasert from NIPA Cloud, Thailand's largest OpenStack public cloud provider. Speakers: Dr. Abhisak Chulya, Charnsilp Chinprasert, Arnaud Morin, Felix Huettner, Thierry Carrez, Kristin Barrientos Date and time: Sept. 21 at 9 a.m. CT / 1400 UTC You can watch us live on: YouTube: https://www.youtube.com/watch?v=ir9v_UK_MIA LinkedIn: https://www.linkedin.com/events/7085718563113603072/comments/ WeChat: recording will be posted on OpenStack WeChat after the live stream Have an idea for a future episode? Share it now at ideas.openinfra.live. Thanks, Kristin Barrientos Marketing Coordinator OpenInfra Foundation

1 0

Cinder Bug Report 2023-09-20
by Jon Bernard 20 Sep '23

20 Sep '23

Hello Argonauts, Cinder Bug Meeting Etherpad <https://etherpad.opendev.org/p/cinder-bug-squad-meeting> Cinder / Undecided - [Pure Storage] Missing replication pod can cause driver failure on restart <https://bugs.launchpad.net/cinder/+bug/2035404> - Status: In Progress - NetApp NFS driver for Cinder may over-provision pools in an active-active configuration <https://bugs.launchpad.net/cinder/+bug/2036457> - Status: New OS-Brick / Undecided - hostnqn file not created automatically <https://bugs.launchpad.net/os-brick/+bug/2035606> - Status: In Progress - NVMe-oF cannot connect <https://bugs.launchpad.net/os-brick/+bug/2035695> - Status: In Progress - NVMe-oF cannot connect on newer CentOS 9 stream <https://bugs.launchpad.net/os-brick/+bug/2035811> - Status: In Progress Thanks, -- Jon

1 0

[keystone] Meeting cancelled this week
by Dave Wilde 20 Sep '23

20 Sep '23

Hi folks, Due to some unforeseen circumstances I need to cancel the Keystone meeting this week. Please reach out on IRC or via the mailing list if you need anything. Thanks! /Dave

1 0

[ptls][URGENT] Bobcat Release: DPUs, GPUs, and FGPAs, oh my!
by Kendall Nelson 20 Sep '23

20 Sep '23

Hello folks! So, as the OpenInfra Foundation has begun putting together the release marking materials for the Bobcat release, we noticed a trend focused on enabling hardware and increasing utilization. Ironic in particular had an excellent cycle highlight about DPUs[1]. We wanted to quickly reach out and see if there were any cycle highlights that projects maybe haven't published in a review, but are related to this topic. Please respond ASAP as we are moving fast on finalizing the press release. Also, if there is anything related coming in Caracal, we might be able to call that out as well, though I know it's quite early to know what might be landing in 2024.1. If there are other cycle highlights that should get called out as well, definitely push the patches and add me as a reviewer! -Kendall Nelson [1] https://releases.openstack.org/bobcat/highlights.html#ironic

4 3

[neutron] Static ip address of subnet dhcp server
by Kamil Madac 20 Sep '23

20 Sep '23

Hello OpenStack community, I'd like to ask whether it's possible to set a static fixed IP address for a DHCP server within a subnet, especially when using the OVN driver. While I've come across a blueprint that addresses this topic, I haven't been able to find any pertinent details in the documentation. Our primary use case is to maintain IP addresses during the migration of VMs between two OpenStack clouds. Any guidance or suggestions would be greatly appreciated. Thank you! -- Kamil Madac

2 3

[largescale-sig] Next meeting: September 20, 8utc
by Thierry Carrez 20 Sep '23

20 Sep '23

Hi everyone, The Large Scale SIG will be meeting this Wednesday in #openstack-operators on OFTC IRC, at 8UTC, our APAC+EU-friendly time. I will be chairing. You can doublecheck how that UTC time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20230920T08 Feel free to add topics to the agenda: https://etherpad.opendev.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez

1 1

Enquiry
by Gk Gk 20 Sep '23

20 Sep '23

Hi All, Is there any discussion going on among the developers or architects of this community to change the coding language from Python to Go in the near future by any chance ? Or, are there any such suggestions from the community so far, given the better speed of Go over Python in execution times ? Thanks Y.G,

5 6

[elections] Deadline to vote approaching!
by Jay Faulkner 19 Sep '23

19 Sep '23

Hey all, There is a little over 24 hours remaining in the election for OpenStack Technical Committee and for OpenStack-Helm PTL. Eligible voters who have opted-in to election email should have a ballot in their inbox from civs at cornell dot edu. Ballots are documented to close Sep 20, 2023 23:45 UTC. Thanks for participating! -- Jay Faulkner TC Vice-Chair

1 0

[tc] Technical Committee next weekly meeting Today, September 19, 2023
by Jay Faulkner 19 Sep '23

19 Sep '23

Hi all, This is a reminder that the next weekly Technical Committee meeting is to be held today on Tuesday, September 12 2023 at 1800 UTC on #openstack-tc on OFTC IRC. Please find the agenda below: - Roll call - Follow up on past action items - No action items last meeting - Gate health check - Open Discussion and Reviews - Register for the PTG - #link https://openinfra.dev/ptg/ - #link https://review.opendev.org/q/projects:openstack/governance+is:open Thank you, Jay Faulkner TC Vice-Chair

1 0

[barbican][cinder] Business Software License in new Vault
by Damian Bulira 19 Sep '23

19 Sep '23

Hi Guys, Recently Hashicorp changed their product licensing from MPL to BSL. Did any of you carry out research on the impact of this change in regard to using Vault as a backend in Barbican and/or Cinder for both private and public clouds? Any thoughts about that? Cheers, Damian

4 4

Regarding issues in openstack magnum deployment
by Naman Jain 19 Sep '23

19 Sep '23

HI I am creating an openstack cluster where i have deployed, i am followinf victoria release of installation https://docs.openstack.org/victoria/install/ keystone placement glance nova neutron i have created an vm instance inside my openstack cluster which is working fine for both inbound and outbound connectivity now i want to deploy the kubernetes service for which i have installed heat and magnum when i am creating the kubernetes service using magnum it is getting stuck in kube_master resource to create for an hour and then gets failed attaching the SS for reference what could be the reason for this i am expecting the kubernetes creation process by magnum in openstack -- This email and any attachments may contain confidential, proprietary, or legally privileged information and are for the intended recipient only. Any other use of emails, such as unauthorized access, disclosure, copying, distribution, or reliance on any contents by anyone else, is prohibited and unlawful. If you are not the intended recipient, please notify the sender immediately and delete the email

2 1

Virtual hardware Watchdog for Windows VMs
by pi3.14＠tuta.io 19 Sep '23

19 Sep '23

Hello, Is it possible to use virtual hardware watchdog in Windows VMs? I am emulating the "i6300esb" with QEMU in Linux instances and everything works as expected. But when creating Windows instances, this device does not show up in the device manager. - Has anyone tried/tested this before? - Is this supported? - Are additional drivers needed? - I found the following old bug. Has anything changed since then or are there still no suitable drivers? https://bugzilla.redhat.com/show_bug.cgi?id=610063

2 1

Advice on Openstack installer
by Felipe Mogollon 19 Sep '23

19 Sep '23

Hello to everyone, I have some experience in installing Openstack but I am having some weird problems lately. I have installed Openstack Victoria previously using packstack and everything was working fine. Some days ago I tried to deploy a more updated version of Openstack and previous configuration that I was using with packstack is not working. I have tried to deploy a new Openstack with packstack and devstack and none of them were working. I would like to ask you if you kown a running installer because I have run out of ideas about what to do. Regards Felipe -- [image: Vicomtech] <https://www.vicomtech.org> Industry Division Juan Felipe Mogollón Rodríguez Researcher Digital Media fmogollon(a)vicomtech.org +(34) 943 30 92 30 The information contained in this electronic message is intended only for the personal and confidential use of the recipients. If you have received this e-mail by mistake, please, notify us and delete it. Avoid printing this message if it is not strictly necessary.

5 4

[barbican] PTO - Canceling meetings
by Grzegorz Grasza 19 Sep '23

19 Sep '23

Hi all, I'm canceling today's barbican meeting, since I'm still on unscheduled PTO. I also won't be able to attend next week's meeting (September 26). So see y'all next month. In the meantime, if something needs urgent attention, please contact dmendiza or alee. / Greg

1 0

[openstack][skyline]Menu route problem.
by Nguyễn Hữu Khôi 19 Sep '23

19 Sep '23

Hello guys. I am working on Skyline and I followed https://github.com/openstack/skyline-console/blob/master/docs/en/develop/3-… https://github.com/openstack/skyline-console/blob/master/docs/en/develop/3-… I add layouts/menu.jsx ##### { path: '/konitor', name: t('Konitor'), key: /konitor', icon: <DatabaseOutlined />, children: [ { path: '/konitor/konitork', name: t('Konitork'), key: 'konitork', level: 1, } ], } I put my component in pages/konitor/containers/Konitork/index.jsx ####### import React, { Component } from 'react'; import { inject, observer } from 'mobx-react'; export class Konitork extends Component { render() { return ( <div>nhk oi</div> ); } } export default observer(Konitork); Here is my routes/index.js ##### import BaseLayout from '../../../layouts/Basic'; import Konitork from '../containers/Konitork'; import Instance from '../containers/Instance'; const PATH = '/konitor'; export default [ { path: PATH, component: BaseLayout, routes: [ { path: `${PATH}/konitork`, component: Konitork, exact: true }, ], }, ]; When click on Menu. Konitor>>Konitork. It return 404. although I have x.x.x.x:9999/konitor/konitork. Pls correct me. Thank you much. Nguyen Huu Khoi

2 4

[Ceilometer] neither gnocchi nor file publisher receive any data or metric
by Mostafa Salari 19 Sep '23

19 Sep '23

Hi. how can i find a simple, correct, ceilometer configuration for "yoga" version of openstack? 1) I followed this instruction guide <https://docs.openstack.org/ceilometer/latest/install/> step by step! 2) I installed devstack with enabled ceilometer (and gnocchi as backend) but after both tries mentioned above, i did not see any metric. I also changed the publisher from gnocchi:// into file:// but the result did not change. cat /etc/ceilometer/ceilometer.conf: [DEFAULT] transport_url = rabbit://stackrabbit:bsvbr5pzLpN0szN84v3m@127.0.0.1:5672/ use_syslog = True [oslo_messaging_notifications] topics = notifications driver = messagingv2 [coordination] backend_url = redis://localhost:6379 [notification] workers = 4 [cache] backend_argument = url:redis://localhost:6379 backend_argument = distributed_lock:True backend_argument = db:0 backend_argument = redis_expiration_time:600 backend = dogpile.cache.redis enabled = True [service_credentials] auth_url = http://127.0.0.1/identity region_name = RegionOne password = bsvbr5pzLpN0szN84v3m username = ceilometer project_name = service project_domain_id = default user_domain_id = default auth_type = password [keystone_authtoken] memcached_servers = localhost:11211 cafile = /opt/stack/data/ca-bundle.pem project_domain_name = Default project_name = service user_domain_name = Default password = bsvbr5pzLpN0szN84v3m username = ceilometer auth_url = http://127.0.0.1/identity interface = public auth_type = password _________________________________________ cat /etc/ceilometer/pipeline.yaml: --- sources: - name: meter_source meters: - "*" sinks: - meter_sink sinks: - name: meter_sink publishers: - file:///tmp/ceilofile?json # - gnocchi://?archive_policy=low&filter_project=service ____________________________ cat /etc/ceilometer/polling.yaml --- sources: - name: all_pollsters interval: 300 meters: - "*" _____________________________ root@os-aio:/# cat /etc/gnocchi/gnocchi.conf [DEFAULT] debug = True coordination_url = redis://localhost:6379 [indexer] url = mysql+pymysql:// root:bsvbr5pzLpN0szN84v3m@127.0.0.1/gnocchi?charset=utf8 [storage] redis_url = redis://localhost:6379 driver = redis [metricd] metric_processing_delay = 5 [api] auth_mode = keystone [keystone_authtoken] memcached_servers = localhost:11211 cafile = /opt/stack/data/ca-bundle.pem project_domain_name = Default project_name = service user_domain_name = Default password = bsvbr5pzLpN0szN84v3m username = gnocchi auth_url = http://127.0.0.1/identity interface = public auth_type = password

2 1

[manila] - Enforcing CephFS pool data placement via backend config. options deprecated?
by Paul Browne 18 Sep '23

18 Sep '23

Hello to the list, In OpenStack releases prior to Wallaby, in CephFS native driver Manila backend config there existed a configuration directive cephfs_volume_path_prefix <https://docs.openstack.org/manila/wallaby/configuration/shared-file-systems…> ; # DEPRECATED: The prefix of the cephfs volume path. (string value) > # This option is deprecated for removal since Wallaby. > # Its value may be silently ignored in the future. > # Reason: This option is not used starting with the Nautilus release > # of Ceph. > #cephfs_volume_path_prefix = /volumes This volume path prefix was usable (and very useful) to set different prefixes in different backends, pointing to different pools in the same backend Ceph cluster e.g. pools backed by storage devices of different characteristics/technology or Ceph CRUSH rule, etc The pool selection would be done in CephFS via the use of file layout fattributes <https://docs.ceph.com/en/quincy/cephfs/file-layouts/>, file layout inheritance ensuring that data created in sub-directories ends up in the correct Ceph pool. e.g. two backends using the same Ceph cluster but two different path prefixes hitting different Ceph pools [cephfsnative1] > driver_handles_share_servers = False > share_backend_name = CEPHFS1 > share_driver = manila.share.drivers.cephfs.driver.CephFSDriver > cephfs_conf_path = /etc/ceph/ceph.conf > cephfs_auth_id = manila > cephfs_cluster_name = ceph > cephfs_volume_path_prefix = /volumes-staging > cephfs_volume_mode = 777 > cephfs_enable_snapshots = False > [cephfsnative1_ec] > driver_handles_share_servers = False > share_backend_name = CEPHFS1_EC > share_driver = manila.share.drivers.cephfs.driver.CephFSDriver > cephfs_conf_path = /etc/ceph/ceph.conf > cephfs_auth_id = manila > cephfs_cluster_name = ceph > cephfs_volume_path_prefix = /volumes-ec-staging > cephfs_volume_mode = 777 > cephfs_enable_snapshots = False However, since Wallaby this config directive looks to have been deprecated and the default of using a path prefix of only /volumes is possible.Trying to use any other prefix in backend configs is ignored. Would anyone on the list know why this option was deprecated in Manila code, or was this forced on Manila by upstream Ceph as of Nautilus? Is there a way to get back to an equivalent functionality? Currently using only a default path of /volumes means we have lost all flexibility in defining Manila CephFS share data placement using the native CephFS driver. Possibly using share group types+share groups and some pre-created paths in the root CephFS could get to something like equivalency? But these paths would need to correspond to the share group UUID, which will only be known after the share group has been created. So not all that flexible a path, since it requires an interaction between users to communicate the share group ID and Ceph admins to set the correct file layout policy. Putting the path prefix in the backend type removed all of that in a nicely transparent way. Having just prototyped this, it will work for setting a desired file layout on a pre-defined share group UUID path in the root CephFS, though it's not really ideal or sustainable to be able to do this for dynamically created share groups by users or automation... Thanks in advance for any advice, Paul Browne

2 1

[neutron] Bug Deputy Report September 11 - 17
by Miro Tomaska 18 Sep '23

18 Sep '23

Hello All, I was the neutron bug deputy last week (Sept 11-17). Here is what is new: High: -https://bugs.launchpad.net/neutron/+bug/2035325 FDB entries grows indefinitely Assigned: Luis Tomas - https://bugs.launchpad.net/neutron/+bug/2035382 "TestMonitorDaemon._run_monitor" failing radomly, initial message not written Assigned: Miro Tomaska - https://bugs.launchpad.net/neutron/+bug/2035578 [stable branches] devstack-tobiko-neutron job Fails with InvocationError('could not find executable python', None) Unassigned - https://bugs.launchpad.net/neutron/+bug/2036118 VM fails to contact metadata during live-migration Assigned: Jakub Libosvar Medium: - https://bugs.launchpad.net/neutron/+bug/2035168 Remaining db migrations for unmaintained Nuage plugin Unassigned - https://bugs.launchpad.net/neutron/+bug/2035281 [ML2/OVN] DGP/Floating IP issue - no flows for chassis gateway port Assigned: Roberto Bartzen Acosta Undecided: - https://bugs.launchpad.net/neutron/+bug/2035230 Port on creation is returned without the fixed_ips field populated - https://bugs.launchpad.net/neutron/+bug/2035573 Insertion of a duplicated ProviderResourceAssociation entry while creating a HA router Asked for more info - https://bugs.launchpad.net/neutron/+bug/2035332 [OVN] VLAN networks for North / South Traffic Broken Best Regards, -- Miro Tomaska Senior Software Engineer

1 0

[ironic] Mass retirement of old branches due to config errs
by Jay Faulkner 18 Sep '23

18 Sep '23

Hi all, Ironic is one of the major offenders remaining in the work to cleanup Zuul config errors. To resolve these issues, it's my plan to retire any impacted branches unless a contributor volunteers to clean them up. I will give folks until at least September 1, 2023, to object before I begin to take action. These branches have been broken for at least one year, and have gotten no patches or releases in that time. It's clear they have little need to continue existing. Please look at https://zuul.opendev.org/t/openstack/config-errors for the full list of potentially impacted branches; I will list here as best I can: openstack/ironic-prometheus-exporter - bugfix/2.1 - stable/train openstack/networking-generic-switch - stable/pike - stable/train - stable/ussuri - stable/victoria - stable/wallaby - stable/xena openstack/python-ironicclient - stable/train - stable/ussuri - stable/victoria - stable/wallaby - stable/xena - Note: stable/yoga is also in the config errors list, but is considered maintained and cannot be retired at this time. I will work to remove failing jobs and get this zuul config fixed manuallt. As an additional note, the following branches were retired months ago, but due to a bug were recreated. They have been fully deleted (again): openstack/ironic-python-agent - bugfix/8.0 openstack/ironic - bugfix/18.0

3 4

[manila] Manila Bugsquash Sep 19th - Sep 21st
by Carlos Silva 18 Sep '23

18 Sep '23

Hello, Zorillas! As agreed in the previous weekly meeting, our bugsquash will run from September 19th to September 21st. The kick off call will be held on Tuesday, Sep 19th at 14:30 UTC using meetpad [1]. We will use the Manila weekly meeting on Thursday at 15 UTC as a checkpoint. If you plan to work on a bug that is not on the list [2], feel free to add it. Looking forward to seeing you there! [1] https://meetpad.opendev.org/manilabobcatbugsquash [2] https://etherpad.opendev.org/p/6VZ5-sbYiSy0hygRZZ6c Regards, carloss

1 0

[cinder] Map Availability Zone to Volume Type
by Jahns, Jay 18 Sep '23

18 Sep '23

Hi, We have a multi-AZ environment where we launch instances with bootable volumes. When we specify AZ, we need to have a specific volume type in that AZ selected to launch to. Right now, the __DEFAULT__ volume type is used. Normally, that would work, since there is 1 backend in each AZ; however, we need to be able to use the key/value RESKEY:availability_zones to pin a volume type to an AZ, so we can use specific key/values in creating the volume. It is 2 separate backends in the environment. I see this bug: https://bugs.launchpad.net/cinder/+bug/1999706 And this change: https://review.opendev.org/c/openstack/cinder/+/868539 Is there anything we can do to help expedite adding this functionality? It seems that it was supposed to already be in. Thanks, Jay

2 1

[neutron] CI meeting on 18.09.2023 cancelled
by Yatin Karel 18 Sep '23

18 Sep '23

Hello Neutron Team !! I will be out tomorrow for a public holiday, so cancelling the CI meeting. Sorry for the short notice. See You at the next meeting on 25th of September. Thanks and Regards Yatin Karel

1 1

[kolla] [ptg] Kolla 2024.1 (Caracal) PTG planning
by Michał Nasiadka 18 Sep '23

18 Sep '23

Hello All, The 2024.1 (Caracal) virtual PTG is approaching and will be held between 23-27 October, 2023. I’ve created a usual etherpad [1] where you can post your topic proposals and we’ll discuss them For dates - we have been usually meeting on two/three days: There were requests on the Bobcat PTG to make these time slots more Europe friendly (e.g. earlier in the day and not finish at 8pm) and host a slot that is Americas/Asia friendly (if needed) - so I associated each day with a Doodle Poll - let’s try to work out best times for most of us. 1. Kolla/Kolla-Ansible topics day 1 - 4 hours Doodle: https://doodle.com/meeting/participate/id/dyXZN9nb 2. Kolla/Kolla-Ansible topics day 2 - 4 hours (including Kolla Operator Hour) Doodle: https://doodle.com/meeting/participate/id/aO8WxqLb 3. Kayobe topics day - 2 hours Doodle: https://doodle.com/meeting/participate/id/e5LmZExd NOTE: If there are participants interested from other timezones that proposed times in the Doodle poll would not fit - please reply to this email and we’ll work out a dedicated slot. Please add your topics timely so they can be arranged as per your preferences, see you at the PTG! [1] https://etherpad.opendev.org/p/kolla-caracal-ptg Thanks, Michal

1 0

[nova] Live migration of a RAM intensive instance failed
by Rafa 18 Sep '23

18 Sep '23

Hello, I have a volume-backed instance with 16 vCPU and 64GB RAM. The instance uses 60GB RAM out of 64GB (used: 22GB; buff/cache 38GB). When I do a live migration of this instance, it fails without any timeouts. It copies almost all the RAM (within 150 - 250 seconds) to the target compute host without any problems according to the logs. Then the instance is paused to copy the rest of the RAM. Everything seems to be working correctly up to this point, but then the instance resumes and the following error message appears: Live Migration failure: operation failed: migration out job: unexpectedly failed: libvirt.libvirtError: operation failed: migration out job: unexpectedly failed Unfortunately, this error message does not say much. It doesn't look like it's due to any timeouts or short downtimes, but I still tested different (higher) values for the following configurations. Unfortunately without success. - live_migration_completion_timeout - live_migration_timeout_action: abort / force_complete (pause) - live_migration_downtime - live_migration_downtime_steps - live_migration_downtime_delay - live_migration_permit_auto_converge: True / False All other instances on the same source and destination hosts can be live migrated without any issues. This instance can also be successfully live migrated after a restart, as it is probably not yet heavily loaded. After a few hours, however, the live migration no longer works. Any ideas what the problem could be? Logs: - nova-compute.log from source compute host: https://paste.openstack.org/show/bJMFxnPKQBEVaPakud61/ - i found this Traceback using journalctrl: https://paste.openstack.org/show/bIS4GFAd2RJ5fHVN9I8d/ - there was also an error in /var/log/libvirt/qemu/: https://paste.openstack.org/show/bImT89IelDcXXBPSgTCO/ Enviroment: - Libvirt: 8.0.0 - QEMU: 4.2.1 - Nova: 25.1.1 - OpenStack: Yoga - Compute operating system: Ubuntu 20.04

4 7

[release] Release countdown for week R-2, Sep 18-22
by Elõd Illés 15 Sep '23

15 Sep '23

Development Focus ----------------- At this point we should have release candidates (RC1 or recent intermediary release) for all the 2023.2 Bobcat deliverables. Teams should be working on any release-critical bugs that would require another RC or intermediary release before the final release. Warning ------- This is a special note to teams about issues regarding oslo.db and castellan deliverables. Please be aware that new releases are about to be sent out. For details, see the mail thread: https://lists.openstack.org/pipermail/openstack-discuss/2023-September/0351… Actions ------- Early in the week, the release team will be proposing stable/2023.2 branch creation for all deliverables that have not branched yet, using the latest available 2023.2 Bobcat release as the branch point. If your team is ready to go for creating that branch, please let us know by leaving a +1 on these patches. If you would like to wait for another release before branching, you can -1 the patch and update it later in the week with the new release you would like to use. By the end of the week the release team will merge those patches though, unless an exception is granted. Once stable/2023.2 branches are created, if a release-critical bug is detected, you will need to fix the issue in the master branch first, then backport the fix to the stable/2023.2 branch before releasing out of the stable/2023.2 branch. After all of the cycle-with-rc projects have branched we will branch devstack, grenade, and the requirements repos. This will effectively open them up for 2024.1 Caracal development, though the focus should still be on finishing up 2023.2 Bobcat until the final release. For projects with translations, watch for any translation patches coming through and merge them quickly. A new release should be produced so that translations are included in the final 2023.2 Bobcat release. Finally, now is a good time to finalize release notes. In particular, consider adding any relevant "prelude" content. Release notes are targeted for the downstream consumers of your project, so it would be great to include any useful information for those that are going to pick up and use or deploy the 2023.2 Bobcat version of your project. Upcoming Deadlines & Dates -------------------------- Final RC deadline: September 28th, 2023 (R-1 week) Final 2023.2 Bobcat release: October 4th, 2023 2024.1 Caracal Virtual PTG - October 23-27, 2023

1 0

[neutron] Neutron drivers meeting cancelled
by Rodolfo Alonso Hernandez 15 Sep '23

15 Sep '23

Hello Neutrinos: Due to the lack of agenda [1], today's meeting is cancelled. Have a nice weekend. [1]https://wiki.openstack.org/wiki/Meetings/NeutronDrivers

1 0

[manila] - Enforcing CephFS pool data placement via backend config. options deprecated?
by Paul Browne 15 Sep '23

15 Sep '23

Hello to the list, In OpenStack releases prior to Wallaby, in CephFS native driver Manila backend config there existed a configuration directive cephfs_volume_path_prefix<https://docs.openstack.org/manila/wallaby/configuration/shared-file-systems…> ; # DEPRECATED: The prefix of the cephfs volume path. (string value) # This option is deprecated for removal since Wallaby. # Its value may be silently ignored in the future. # Reason: This option is not used starting with the Nautilus release # of Ceph. #cephfs_volume_path_prefix = /volumes This volume path prefix was usable (and very useful) to set different prefixes in different backends, pointing to different pools in the same backend Ceph cluster e.g. pools backed by storage devices of different characteristics/technology or Ceph CRUSH rule, etc The pool selection would be done in CephFS via the use of file layout fattributes<https://docs.ceph.com/en/quincy/cephfs/file-layouts/>, file layout inheritance ensuring that data created in prefix child sub-directories ends up in the correct Ceph pool. e.g. two backends using the same Ceph cluster but two different path prefixes hitting different Ceph pools [cephfsnative1] driver_handles_share_servers = False share_backend_name = CEPHFS1 share_driver = manila.share.drivers.cephfs.driver.CephFSDriver cephfs_conf_path = /etc/ceph/ceph.conf cephfs_auth_id = manila cephfs_cluster_name = ceph cephfs_volume_path_prefix = /volumes-staging cephfs_volume_mode = 777 cephfs_enable_snapshots = False [cephfsnative1_ec] driver_handles_share_servers = False share_backend_name = CEPHFS1_EC share_driver = manila.share.drivers.cephfs.driver.CephFSDriver cephfs_conf_path = /etc/ceph/ceph.conf cephfs_auth_id = manila cephfs_cluster_name = ceph cephfs_volume_path_prefix = /volumes-ec-staging cephfs_volume_mode = 777 cephfs_enable_snapshots = False However, since Wallaby this config directive looks to have been deprecated and the default of using a path prefix of only /volumes is possible. Trying to use any other prefix in backend configs is ignored. Would anyone on the list know why this option was deprecated in Manila code, or was this forced on Manila by upstream Ceph as of Nautilus? Is there a way to get back to an equivalent functionality? Currently using only a default path of /volumes means we have lost all flexibility in defining Manila CephFS share data placement using the native CephFS driver. Possibly using share group types+share groups and some pre-created paths in the root CephFS could get to something like equivalency? But these paths would need to correspond to the share group UUID, which will only be known after the share group has been created. So not all that flexible a path, since it requires an interaction between users/tenant to communicate the share group ID and Ceph admins to set the correct file layout policy. Putting the path prefix in the backend+share-type removed all of that in a nicely transparent way. Having just prototyped this, it will work for setting a desired file layout on a pre-defined share group UUID path in the root CephFS, though it's not really ideal or sustainable to be able to do this for dynamically created share groups by users or automation, which could exist in large numbers... Thanks in advance for any advice, Paul Browne ******************* Paul Browne Research Computing Platforms University Information Services Roger Needham Building JJ Thompson Avenue University of Cambridge Cambridge United Kingdom E-Mail: pfb29(a)cam.ac.uk<mailto:pfb29@cam.ac.uk> Tel: 0044-1223-746548 *******************

1 0

[kolla-ansible]change prometheus data volume path
by Satish Patel 15 Sep '23

15 Sep '23

Folks, I have deployed prometheus with koll-ansible and now I want to change the path of prometheus volume to store matrices on bigger partitions. How do I change the volume path of prometheus to store matrix data on separate filesystem? This is what I did in globals.yml but it doesn't work. Is the following variable correct? prometheus_server_default_volumes: "/mnt/prometheus"

3 2

[cinder][kolla-ansible] cinder-volume active/active usage
by 韩光宇 15 Sep '23

15 Sep '23

Hello, I saw that kolla-ansible had allow cinder coordination backend to be configured. And it can be set to "redis" or "etcd" in /etc/kolla/gloabls.yml [1]. Based on my current understanding, this option enables cinder-volume active/active. 1. I would like to ask, between Redis and Etcd, which one is more recommended or what are their pros and cons? 2. And I have saw the spec of cinder-volume a/a[2]. Is it sufficient to configure [coordination]backend_url and [DEFAULT]cluster to use Cinder volume active/active, or are there any other considerations? Could any reference for Cinder volume active/active configuration and usage documentation be recommended ? I would appreciate for any help. Best wishes. Han Guangyu [1] https://github.com/openstack/kolla-ansible/blob/master/etc/kolla/globals.ym… [2] https://specs.openstack.org/openstack/cinder-specs/specs/ocata/ha-aa-job-di…

3 2

[adjutant][barbican][blazar][ec2-api][freezer][masakari][mistral][monasca][murano][sahara][solum][telemetry][vitrage][zun] broken gate on master branch
by Elõd Illés 14 Sep '23

14 Sep '23

Hi $teams_in_subject, I've generated test patches to see gate health (this time for cycle-with-rc deliverables, that have not merged any patches recently) and the patches show [1] that you have broken CI jobs on your gate. Could the teams please check what the issue is / issues are and fix as soon as possible? Thanks in advance! [1] https://review.opendev.org/q/topic:release-health-check-cwr-bobcat+is:open Thanks, Előd Illés irc: elodilles @ #openstack-release

2 1

Octavia - open source scalable load balancer for use in large OpenStack deployments.
by Ajay Kushal 14 Sep '23

14 Sep '23

Hello this is Ajay, final year engineering student from karnataka,India. I have come across this project and feels really interesting and very important for part my course project. Can you please help me in executing this complete project successfully or sharing a demo video of project could be more clear. Please help me in this regards..

2 1

[puppet] Transitioning train, ussuri and victoria to EOL
by Takashi Kajinami 14 Sep '23

14 Sep '23

Hello, As we agreed in the PTG discussions at Vancouver, we'll EOL the old stable branches of Puppet OpenStack projects. As the first step I'll start transitioning stable/train, ussuri and victoria to EOL early next week. Please let me know in case anyone has any concerns about it. Thank you, Takashi

1 1

[cinder] festival of feature reviews on 15th September
by Rajat Dhasmana 14 Sep '23

14 Sep '23

Hello Argonauts, We will be having our monthly festival of reviews tomorrow i.e. 15th September (Friday) from 1400-1600 UTC. Following are some additional details: Date: 15th September, 2023 Time: 1400-1600 UTC Meeting link: https://meet.google.com/jqg-eigw-rku etherpad: https://etherpad.opendev.org/p/cinder-festival-of-reviews We are planning to prioritize patches for RC2[1]. [1] https://etherpad.opendev.org/p/cinder-bobcat-rc-patches Thanks Rajat Dhasmana

1 0

Re: [openstack][skyline] Some works with skyline
by Nguyễn Hữu Khôi 14 Sep '23

14 Sep '23

Hello. The case is here, https://github.com/openstack/skyline-console/blob/master/src/pages/compute/… but you need to understand how skyline construct page. This project uses nodejs, I use the chart.js library to code. it will need some knowledge about api and javascript. Let's read first. Nguyen Huu Khoi On Wed, Sep 13, 2023 at 7:29 PM Satish Patel <satish.txt(a)gmail.com> wrote: > Sure that is ok. Can you give me a small hit about what you actually > did? I can see you created a new tab 'monitor' but behind that did you > create a new css theme etc to pull metrics from prometheus? > > All I want to do is just to see if it's worth it because we don't have > prometheus but we have different data sources which I can integrate. Thanks > for the work. very appreciated. > > On Wed, Sep 13, 2023 at 7:27 AM Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> > wrote: > >> Hello. Bc i am just working. And I am not dev so I still learn. If it is >> ok then i will do. >> >> On Wed, Sep 13, 2023, 6:17 PM Satish Patel <satish.txt(a)gmail.com> wrote: >> >>> That is great!! >>> >>> I’m not a developer but will try to read the code but it would be good >>> if you just give me file name or clue about where to look. >>> >>> If you have patch or diff would be great. Why don’t you contribute this >>> feature and submit a patch >>> >>> Sent from my iPhone >>> >>> On Sep 12, 2023, at 8:08 PM, Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com> >>> wrote: >>> >>> >>> >>> Nguyen Huu Khoi >>> Hello. I just read skyline source code and followed it about data >>> source. This is >>> https://github.com/zhangjianweibj/prometheus-libvirt-exporter. >>> >>>

2 4

[cinder][PTG] Cinder 2024.1 (Caracal) PTG Planning
by Rajat Dhasmana 14 Sep '23

14 Sep '23

Hello All, The 2024.1 (Caracal) virtual PTG is approaching and will be held between 23-27 October, 2023. I've created a planning etherpad[1] and a PTG etherpad[2] to gather topics for the PTG. Note that you only need to add topics in the planning etherpad[1] and those will be arranged in the PTG etherpad[2] later. Date: Tuesday (24th October) to Friday (27th October) 2023 Time: 1300-1700 UTC Etherpad: https://etherpad.opendev.org/p/caracal-ptg-cinder-planning Please add your topics timely so they can be arranged as per your preferences, see you at the PTG! [1] https://etherpad.opendev.org/p/caracal-ptg-cinder-planning [2] https://etherpad.opendev.org/p/caracal-ptg-cinder Thanks Rajat Dhasmana

1 0

[kolla-ansible][ovn] availability zone for OVN
by Satish Patel 13 Sep '23

13 Sep '23

Folks, I am trying to set an availability zone for OVN because skyline UI has a mandatory requirement to have an availability zone otherwise you are not allowed to create a network from skyline GUI. for OVN based deployment only option to set AZ is in ovn-cms-options # ovs-vsctl set open_vswitch . external_ids:ovn-cms-options="enable-chassis-as-gw,availability-zones=az-0" Question: 1. How do I configure in kolla-ansible to override or set ovn-cms-options on only gateway chassis? 2. How does AZ work in OVN because OVN is anyway distributed as per documents, What if I just give foobar AZ name to meet Skyline requirement? Is it going to break anything?

3 3

Cinder Bug Report 2023-09-13
by Jon Bernard 13 Sep '23

13 Sep '23

Hello Argonauts, Cinder Bug Meeting Etherpad <https://etherpad.opendev.org/p/cinder-bug-squad-meeting> Undecided - Fail to migrate, resize, evacuate VM with volume state backing-up <https://bugs.launchpad.net/cinder/+bug/2034677> - Status: New - Dell PowerMax Live Migration Fails Without a Pool Name <https://bugs.launchpad.net/cinder/+bug/2034937> - Status: New - Cinder Backups Fail with cinder.exception.ServiceNotFound with multiple availibility zones <https://bugs.launchpad.net/cinder/+bug/2035007> - Status: New - Dell PowerFlex (Scaleio) connector doesn't handle volume disconnection/unmapping properly <https://bugs.launchpad.net/os-brick/+bug/2034685> - Status: New Thanks, -- Jon

1 0

[kolla-ansible] haproxy tls key location
by Jonathan Proulx 12 Sep '23

12 Sep '23

Hi All, Reading through https://docs.openstack.org/kolla-ansible/latest/admin/tls.html and global.yml / passwords.yml in my deploy I see configuration for certificates but not where to set the key (though there is a key location configuration for backend tls in globals.yml). Unsurprisingly when I put the certs where they are expected and enable TLS the haproxy containers fail because they don't have a key. What am I missing here? Thanks, -Jon -- Jonathan Proulx (he/him) Sr. Technical Architect The Infrastructure Group MIT CSAIL

3 3

Nova conductor errors
by Gk Gk 12 Sep '23

12 Sep '23

Hi All, We have a Yoga OSA setup. We have been observing the following errors in the nova conductor service intermittently. After a restart of the service, they seem to disappear. The logs are as follows: --- nova-conductor[3199409]: ERROR oslo.messaging._drivers.impl_rabbit [-] Connection failed: [Errno 24] Too many open files (retrying in 0 seconds): OSError: [Errno 24] Too many open files --- Is it a known issue with yoga ? Please let me know. Thanks Y.G

3 6

[kolla-ansible][horizon][policy]
by James Leong 12 Sep '23

12 Sep '23

Hi all, I am currently having a yoga version openstack. I noticed that user from a domain are able to view other domain leases if they are having admin role. Is there any possible way to change anything in the policy file? I have tried to add rule:owner but it didn't work out the way I wanted. Any recommendations would be appreciated. Best, James

2 1

[tc] Technical Committee next weekly meeting Today, September 12, 2023
by Nikolla, Kristi 12 Sep '23

12 Sep '23

Hi all, This is a reminder that the next weekly Technical Committee meeting is to be held today on Tuesday, September 12 2023 at 1800 UTC on #openstack-tc on OFTC IRC. Please find the agenda below: • Roll call • Follow up on past action items • No action items • Gate health check • OpenStack TC Charter Updates for OIF Foundation Simplification • As discussed in the previous TC+Board syncup, the OpenInfra Foundation bylaws contain a lot of OpenStack specific language. • Kristi, Gmann, and Brian have been meeting with Allison from the board to study the necessary changes to move that language to the charter. • Bylaws changes and what all to go in the TC charter have been shared in an email. TC needs to provide the feedback before Sept 19. • OpenStack Elections • #link https://governance.openstack.org/election/ • #link https://review.opendev.org/c/openstack/election/+/893810 • Feedback from election officials regarding template for extra ACs. • Open Discussion and Reviews • Register for the PTG • #link https://openinfra.dev/ptg/ • #link https://review.opendev.org/q/projects:openstack/governance+is:open Thank you, Kristi Nikolla

1 0

[Kolla] Transition Xena to EOL
by Bartosz Bezak 12 Sep '23

12 Sep '23

Hello, As aggreed on IRC meeting, Kolla Xena deliverables (Kolla, Kolla-Ansible, Kayobe, Ansible-Collection-Kolla) are going EOL. [1] Yoga, Zed and Antelope are our current stable releases. Xena was in Extended Maintenance for a some time, Kolla community does not have resources to support it actively. All changes for Kolla project deliverables in Xena branches have been abandoned. [1] https://review.opendev.org/c/openstack/releases/+/894630 Best regards, Bartosz Bezak

1 0

[openstack-ansible] Meeting on 12.09.2023 is cancelled
by Dmitriy Rabotyagov 12 Sep '23

12 Sep '23

Hi everyone, Due to the absence of multiple core members of the group during this week, it has been decided to cancel our regular community meeting on 12.09.2023.

1 0

[Diversity] Diversity and Inclusion WG Meeting reminder
by Amy Marrich 11 Sep '23

11 Sep '23

This is a reminder that the Diversity and Inclusion WG will be meeting tomorrow at 14:00 UTC in the #openinfra-diversity channel on OFTC. We hope members of all OpenInfra projects join us as we look at the progress on responses to the Foundation-wide diversity survey. Thanks, Amy (spotz) 0 - https://etherpad.opendev.org/p/diversity-wg-agenda

1 0

Re: [openstack-ansible] Dedicated gateway hosts not working with OVN
by Dmitriy Rabotyagov 11 Sep '23

11 Sep '23

Hey, 1. Sorry, my bad, was copying you from my phone, so the extra section ( container_skel) that is required has slipped my paste. So /etc/openstack_deploy/env.d/nova.yml should look like this: container_skel: nova_compute_container: belongs_to: - compute_containers - kvm-compute_containers - qemu-compute_containers contains: - neutron_sriov_nic_agent - neutron_ovn_controller - nova_compute properties: is_metal: true 2. Now I actually see more issues in defined openstack_user_config. I'm not sure if that is the reason of the error or not, but it still must be adjusted: a) replace network_hosts with network-infra_hosts. Defining network_hosts also adds infra servers to neutron_l3_agent (and other agents) which has in fact no effect, but triggers a bug, where run_once is treated wrongly. But this will cause failure down the line and I assume that's not it yet. You might need to clean up inventory as a result. b) also define network-northd_hosts - this usually is usually set to infra nodes, and spawns inside LXC. I would also suggest to check out doc on OVN configuration: https://docs.openstack.org/openstack-ansible-os_neutron/latest/app-ovn.html c) For the issue itself. Most likely, it is looking for `container_bridge` or `host_bind_override` key for some network. As one of these keys are expected in order to create a mapping and ovs bridges for you. It does combine net_name and one of these keys. So it would be interesting to see adjusted openstack_user_config once the above issues are sorted out. I can also suggest defining mappings in neutron_provider_networks directly, like mentioned in the documentation above.

4 12

[publiccloud-sig] Reminder - next meeting September 13th - 0700 UTC
by Felix Kronlage-Dammers 11 Sep '23

11 Sep '23

Hi everyone, this Wednesday the next meeting of the public cloud sig is going to happen. As announced a few weeks ago, we want to try out to have a videocall format every other meeting of the public cloud SIG, so this time we will meet in a video call. Part of the meeting will be a lightning talk. For the first lightning talk Artem (gtema@) will talk on his work on “OpenAPI support for OpenStack and making a new faster CLI on Rust”. (There is a detailled mail about this work that Artem sent to this list End of August[1]). Thanks a lot Artem for giving the first lightning talk in the new format. We will meet at 0700 UTC here: https://conf.scs.koeln:8443/OIF-public-cloud-sig In parallel we will be on irc on #openstack-operators as well of course. Am very much looking forward to wednesday morning! :) felix [1] <url: https://lists.openstack.org/pipermail/openstack-discuss/2023-August/034934.…> -- Felix Kronlage-Dammers Product Owner IaaS & Operations Sovereign Cloud Stack Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr(a)osb-alliance.com

1 0

[ptl][tc][winstackers] Final call for Winstackers PTL and maintainers
by James Page 11 Sep '23

11 Sep '23

Hi All As announced by Lucian last November (see [0]) Cloudbase Solutions are no longer in a position to maintain support for running OpenStack on Windows and have also ceased operation of their 3rd party CI for the windows support across a number of OpenStack projects. This situation has resulted in the Winstackers project becoming PTL-less for the 2023.2 cycle with no volunteers responding to the TC's call to fill this role and take this feature in OpenStack forward (see [1]). This is the final call for any maintainers to step forward if this feature is important to them in OpenStack. The last user survey in 2022 indicated that 2% of respondents were running on Hyper-V so this might be important enough to warrant a commitment from someone operating OpenStack on Windows to maintain these features going forward. Here is a reminder from Lucian's original email on the full list of projects which are impacted in some way: * nova hyper-v driver - in-tree plus out-of-tree compute-hyperv driver * os-win - common Windows library for Openstack * neutron hyperv ml2 plugin and agent * ovs on Windows and neutron ovs agent support * cinder drivers - SMB and Windows iSCSI * os-brick Windows connectors - iSCSI, FC, SMB, RBD * ceilometer Windows poller * manila Windows driver * glance Windows support * freerdp gateway The lack of 3rd party CI for testing all of this really needs to be addressed as well. If no maintainers are forthcoming between now and the next PTG in June the TC will need to officially retire the project and start the process of removing support for Windows across the various projects that support this operating system in some way - either directly or through the use of os-win. For clarity this call refers to the use of the Hyper-V virtualisation driver and associated Windows server components to provide WIndows based OpenStack Hypervisors and does not relate to the ability to run Windows images as guests on OpenStack. Regards James [0] https://lists.openstack.org/pipermail/openstack-discuss/2022-November/03104… [1] https://lists.openstack.org/pipermail/openstack-discuss/2023-March/032888.h…

5 9

[neutron] Bug Deputy Report September 4 - September 10
by Yatin Karel 11 Sep '23

11 Sep '23

Hi Team, Below is the bug deputy report for last week:- High ------ - https://bugs.launchpad.net/neutron/+bug/2033887 [OVN][Trunk] The cold migration process is broken since patch 882581 Fix Proposed: https://review.opendev.org/q/topic:bug%252F2018289 Assigned to Rodolfo - https://bugs.launchpad.net/neutron/+bug/2033932 Add support for OVN MAC_Binding aging Fix Proposed: https://review.opendev.org/c/openstack/neutron/+/893575 Assigned to Terry - https://bugs.launchpad.net/neutron/+bug/2034522 Fake members operating_status ONLINE Fix Proposed: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/893839 Assigned to Fernando Medium ---------- - https://bugs.launchpad.net/neutron/+bug/2034589 [FT][OVN] "ovsdb_connection.stop()" failing during the test cleanup process Unassigned Undecided -------------- - https://bugs.launchpad.net/neutron/+bug/2034761 test discovery fail with latest neutron-lib Marked as Invalid as it was due to version mismatch of neutron-lib and neutron - https://bugs.launchpad.net/neutron/+bug/2034684 UEFI (edk2/ovmf) network boot with OVN fail because no DHCP release reply Looks like issue only against Core OVN, asked reporter for it Thanks and Regards Yatin Karel

1 0

[Skyline] error username or password incorrect
by Satish Patel 11 Sep '23

11 Sep '23

Folks, Try to install skyline UI to replace horizon using doc: https://docs.openstack.org/skyline-apiserver/latest/install/docker-install-… Everything went well and I got a login page on http://x.x.x.x:9999 also it pulled Region/Domains. When I am trying to login with my account, I get an error: Username or Password is incorrect. I am using sqlite DB for skyline as per documents. No errors in logs command $ docker logs skyline When I use Chrome Developer Tools then it was indicating an error in these URLs. http://openstack.example.com:9999/api/openstack/skyline/api/v1/profile http://openstack.example.com:9999/api/openstack/skyline/api/v1/policies 401 Unauthorized ( {"detail":"no such table: revoked_token"} )

2 16

No valid host was found. There are not enough hosts available
by - 10 Sep '23

10 Sep '23

I added a new node to the OpenStack cluster and when creating a new instance I have this error. Maybe someone can tell me what else I can check? No valid host was found. There are not enough hosts available. No valid host was found. There are not enough hosts available. Kod 500 Traceback (most recent call last): File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 1548, in schedule_and_build_instances host_lists = self._schedule_instances(context, request_specs[0], File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 908, in _schedule_instances host_lists = self.query_client.select_destinations( File "/usr/lib/python3/dist-packages/nova/scheduler/client/query.py", line 41, in select_destinations return self.scheduler_rpcapi.select_destinations(context, spec_obj, File "/usr/lib/python3/dist-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations return cctxt.call(ctxt, 'select_destinations', **msg_args) File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/client.py", line 189, in call result = self.transport._send( File "/usr/lib/python3/dist-packages/oslo_messaging/transport.py", line 123, in _send return self._driver.send(target, ctxt, message, File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send return self._send(target, ctxt, message, wait_for_reply, timeout, File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send raise result nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. Traceback (most recent call last): File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 241, in inner return func(*args, **kwargs) File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 223, in select_destinations selections = self._select_destinations( File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 250, in _select_destinations selections = self._schedule( File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 416, in _schedule self._ensure_sufficient_hosts( File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 455, in _ensure_sufficient_hosts raise exception.NoValidHost(reason=reason) nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. Utworzono

2 1

Re: [openstack][neutron][openvswitch] Openvswitch Packet loss when high throughput (pps)
by Ha Noi 10 Sep '23

10 Sep '23

Hi Satish, Actually, the guess interface is not using tap anymore. <interface type='vhostuser'> <mac address='fa:16:3e:76:77:dd'/> <source type='unix' path='/var/run/openvswitch/vhu3766ee8a-86' mode='server'/> <target dev='vhu3766ee8a-86'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> It's totally bypass the kernel stack ? On Fri, Sep 8, 2023 at 5:02 AM Satish Patel <satish.txt(a)gmail.com> wrote: > I did test OVS-DPDK and it helps offload the packet process on compute > nodes, But what about VMs it will still use a tap interface to attach from > compute to vm and bottleneck will be in vm. I strongly believe that we have > to run DPDK based guest to pass through the kernel stack. > > I love to hear from other people if I am missing something here. > > On Thu, Sep 7, 2023 at 5:27 PM Ha Noi <hanoi952022(a)gmail.com> wrote: > >> Oh. I heard from someone on the reddit said that Ovs-dpdk is transparent >> with user? >> >> So It’s not correct? >> >> On Thu, 7 Sep 2023 at 22:13 Satish Patel <satish.txt(a)gmail.com> wrote: >> >>> Because DPDK required DPDK support inside guest VM. It's not >>> suitable for general purpose workload. You need your guest VM network to >>> support DPDK to get 100% throughput. >>> >>> On Thu, Sep 7, 2023 at 8:06 AM Ha Noi <hanoi952022(a)gmail.com> wrote: >>> >>>> Hi Satish, >>>> >>>> Why dont you use DPDK? >>>> >>>> Thanks >>>> >>>> On Thu, 7 Sep 2023 at 19:03 Satish Patel <satish.txt(a)gmail.com> wrote: >>>> >>>>> I totally agreed with Sean on all his points but trust me, I have >>>>> tried everything possible to tune OS, Network stack, multi-queue, NUMA, CPU >>>>> pinning and name it.. but I didn't get any significant improvement. You may >>>>> gain 2 to 5% gain with all those tweek. I am running the entire workload on >>>>> sriov and life is happy except no LACP bonding. >>>>> >>>>> I am very interesting is this project >>>>> https://docs.openvswitch.org/en/latest/intro/install/afxdp/ >>>>> >>>>> On Thu, Sep 7, 2023 at 6:07 AM Ha Noi <hanoi952022(a)gmail.com> wrote: >>>>> >>>>>> Dear Smoney, >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Sep 7, 2023 at 12:41 AM <smooney(a)redhat.com> wrote: >>>>>> >>>>>>> On Wed, 2023-09-06 at 11:43 -0400, Satish Patel wrote: >>>>>>> > Damn! We have noticed the same issue around 40k to 55k PPS. Trust >>>>>>> me >>>>>>> > nothing is wrong in your config. This is just a limitation of the >>>>>>> software >>>>>>> > stack and kernel itself. >>>>>>> its partly determined by your cpu frequency. >>>>>>> kernel ovs of yesteryear could handel about 1mpps total on a ~4GHZ >>>>>>> cpu. with per port troughpuyt being lower dependin on what >>>>>>> qos/firewall >>>>>>> rules that were apllied. >>>>>>> >>>>>>> >>>>>> >>>>>> My CPU frequency is 3Ghz and using CPU Intel Gold 2nd generation. I >>>>>> think the problem is tuning in the compute node inside. But I cannot find >>>>>> any guide or best practices for it. >>>>>> >>>>>> >>>>>> >>>>>>> moving form iptables firewall to ovs firewall can help to some degree >>>>>>> but your partly trading connection setup time for statead state >>>>>>> troughput >>>>>>> with the overhead of the connection tracker in ovs. >>>>>>> >>>>>>> using stateless security groups can help >>>>>>> >>>>>>> we also recently fixed a regression cause by changes in newer >>>>>>> versions of ovs. >>>>>>> this was notable in goign form rhel 8 to rhel 9 where litrally it >>>>>>> reduced >>>>>>> small packet performce to 1/10th and jumboframes to about 1/2 >>>>>>> on master we have a config option that will set the default qos on a >>>>>>> port to linux-noop >>>>>>> >>>>>>> https://github.com/openstack/os-vif/blob/master/vif_plug_ovs/ovs.py#L106-L1… >>>>>>> >>>>>>> the backports are propsoed upstream >>>>>>> https://review.opendev.org/q/Id9ef7074634a0f23d67a4401fa8fca363b51bb43 >>>>>>> and we have backported this downstream to adress that performance >>>>>>> regression. >>>>>>> the upstram backport is semi stalled just ebcasue we wanted to >>>>>>> disucss if we shoudl make ti opt in >>>>>>> by default upstream while backporting but it might be helpful for >>>>>>> you if this is related to yoru current >>>>>>> issues. >>>>>>> >>>>>>> 40-55 kpps is kind of low for kernel ovs but if you have a low >>>>>>> clockrate cpu, hybrid_plug + incorrect qos >>>>>>> then i could see you hitting such a bottelneck. >>>>>>> >>>>>>> one workaround by the way without the os-vif workaround backported >>>>>>> is to set >>>>>>> /proc/sys/net/core/default_qdisc to not apply any qos or a low >>>>>>> overhead qos type >>>>>>> i.e. sudo sysctl -w net.core.default_qdisc=pfifo_fast >>>>>>> >>>>>>> >>>>>> >>>>>>> that may or may not help but i would ensure that your are not usign >>>>>>> somting like fqdel or cake >>>>>>> for net.core.default_qdisc and if you are try changing it to >>>>>>> pfifo_fast and see if that helps. >>>>>>> >>>>>>> there isnet much you can do about the cpu clock rate but ^ is >>>>>>> somethign you can try for free >>>>>>> note it wont actully take effect on an exsitng vm if you jsut change >>>>>>> the default but you can use >>>>>>> tc to also chagne the qdisk for testing. hard rebooting the vm >>>>>>> shoudl also make the default take effect. >>>>>>> >>>>>>> the only other advice i can give assuming kernel ovs is the only >>>>>>> option you have is >>>>>>> >>>>>>> to look at >>>>>>> >>>>>>> https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.rx… >>>>>>> >>>>>>> https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.tx… >>>>>>> and >>>>>>> >>>>>>> https://docs.openstack.org/nova/latest/configuration/extra-specs.html#hw:vi… >>>>>>> >>>>>>> if the bottelneck is actully in qemu or the guest kernel rather then >>>>>>> ovs adjusting the rx/tx queue size and >>>>>>> using multi queue can help. it will have no effect if ovs is the >>>>>>> bottel neck. >>>>>>> >>>>>>> >>>>>>> >>>>>> I have set this option to 1024, and enable multiqueue as well. But it >>>>>> did not help. >>>>>> >>>>>> >>>>>>> > >>>>>>> > On Wed, Sep 6, 2023 at 9:21 AM Ha Noi <hanoi952022(a)gmail.com> >>>>>>> wrote: >>>>>>> > >>>>>>> > > Hi Satish, >>>>>>> > > >>>>>>> > > Actually, our customer get this issue when the tx/rx above only >>>>>>> 40k pps. >>>>>>> > > So what is the threshold of this throughput for OvS? >>>>>>> > > >>>>>>> > > >>>>>>> > > Thanks and regards >>>>>>> > > >>>>>>> > > On Wed, 6 Sep 2023 at 20:19 Satish Patel <satish.txt(a)gmail.com> >>>>>>> wrote: >>>>>>> > > >>>>>>> > > > Hi, >>>>>>> > > > >>>>>>> > > > This is normal because OVS or LinuxBridge wire up VMs using >>>>>>> TAP interface >>>>>>> > > > which runs on kernel space and that drives higher interrupt >>>>>>> and that makes >>>>>>> > > > the kernel so busy working on handling packets. Standard >>>>>>> OVS/LinuxBridge >>>>>>> > > > are not meant for higher PPS. >>>>>>> > > > >>>>>>> > > > If you want to handle higher PPS then look for DPDK or SRIOV >>>>>>> deployment. >>>>>>> > > > ( We are running everything in SRIOV because of high PPS >>>>>>> requirement) >>>>>>> > > > >>>>>>> > > > On Tue, Sep 5, 2023 at 11:11 AM Ha Noi <hanoi952022(a)gmail.com> >>>>>>> wrote: >>>>>>> > > > >>>>>>> > > > > Hi everyone, >>>>>>> > > > > >>>>>>> > > > > I'm using Openstack Train and Openvswitch for ML2 driver and >>>>>>> GRE for >>>>>>> > > > > tunnel type. I tested our network performance between two >>>>>>> VMs and suffer >>>>>>> > > > > packet loss as below. >>>>>>> > > > > >>>>>>> > > > > VM1: IP: 10.20.1.206 >>>>>>> > > > > >>>>>>> > > > > VM2: IP: 10.20.1.154 <https://10.20.1.154/24> >>>>>>> > > > > >>>>>>> > > > > VM3: IP: 10.20.1.72 >>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > Using iperf3 to testing performance between VM1 and VM2. >>>>>>> > > > > >>>>>>> > > > > Run iperf3 client and server on both VMs. >>>>>>> > > > > >>>>>>> > > > > On VM2: iperf3 -t 10000 -b 130M -l 442 -P 6 -u -c 10.20.1.206 >>>>>>> > > > > >>>>>>> > > > > On VM1: iperf3 -t 10000 -b 130M -l 442 -P 6 -u -c 10.20.1.154 >>>>>>> > > > > <https://10.20.1.154/24> >>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > Using VM3 ping into VM1, then the packet is lost and the >>>>>>> latency is >>>>>>> > > > > quite high. >>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > ping -i 0.1 10.20.1.206 >>>>>>> > > > > >>>>>>> > > > > PING 10.20.1.206 (10.20.1.206) 56(84) bytes of data. >>>>>>> > > > > >>>>>>> > > > > 64 bytes from 10.20.1.206: icmp_seq=1 ttl=64 time=7.70 ms >>>>>>> > > > > >>>>>>> > > > > 64 bytes from 10.20.1.206: icmp_seq=2 ttl=64 time=6.90 ms >>>>>>> > > > > >>>>>>> > > > > 64 bytes from 10.20.1.206: icmp_seq=3 ttl=64 time=7.71 ms >>>>>>> > > > > >>>>>>> > > > > 64 bytes from 10.20.1.206: icmp_seq=4 ttl=64 time=7.98 ms >>>>>>> > > > > >>>>>>> > > > > 64 bytes from 10.20.1.206: icmp_seq=6 ttl=64 time=8.58 ms >>>>>>> > > > > >>>>>>> > > > > 64 bytes from 10.20.1.206: icmp_seq=7 ttl=64 time=8.34 ms >>>>>>> > > > > >>>>>>> > > > > 64 bytes from 10.20.1.206: icmp_seq=8 ttl=64 time=8.09 ms >>>>>>> > > > > >>>>>>> > > > > 64 bytes from 10.20.1.206: icmp_seq=10 ttl=64 time=4.57 ms >>>>>>> > > > > >>>>>>> > > > > 64 bytes from 10.20.1.206: icmp_seq=11 ttl=64 time=8.74 ms >>>>>>> > > > > >>>>>>> > > > > 64 bytes from 10.20.1.206: icmp_seq=12 ttl=64 time=9.37 ms >>>>>>> > > > > >>>>>>> > > > > 64 bytes from 10.20.1.206: icmp_seq=14 ttl=64 time=9.59 ms >>>>>>> > > > > >>>>>>> > > > > 64 bytes from 10.20.1.206: icmp_seq=15 ttl=64 time=7.97 ms >>>>>>> > > > > >>>>>>> > > > > 64 bytes from 10.20.1.206: icmp_seq=16 ttl=64 time=8.72 ms >>>>>>> > > > > >>>>>>> > > > > 64 bytes from 10.20.1.206: icmp_seq=17 ttl=64 time=9.23 ms >>>>>>> > > > > >>>>>>> > > > > ^C >>>>>>> > > > > >>>>>>> > > > > --- 10.20.1.206 ping statistics --- >>>>>>> > > > > >>>>>>> > > > > 34 packets transmitted, 28 received, 17.6471% packet loss, >>>>>>> time 3328ms >>>>>>> > > > > >>>>>>> > > > > rtt min/avg/max/mdev = 1.396/6.266/9.590/2.805 ms >>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > Does any one get this issue ? >>>>>>> > > > > >>>>>>> > > > > Please help me. Thanks >>>>>>> > > > > >>>>>>> > > > >>>>>>> >>>>>>>

3 5

[release] Release countdown for week R-3, Sep 11-15
by Herve Beraud 08 Sep '23

08 Sep '23

Development Focus ----------------- The Release Candidate (RC) deadline is next Thursday, September 14. Work should be focused on fixing any release-critical bugs. General Information ------------------- All deliverables released under a cycle-with-rc model should have a first release candidate by the end of the week, from which a stable/2023.2 branch will be cut. This branch will track the 2023.2 Bobcat release. Once stable/2023.2 has been created, the master branch will be ready to switch to 2024.1 Caracal development. While the master branch will no longer be feature-frozen, please prioritize any work necessary for completing 2023.2 Bobcat plans. Release-critical bugfixes will need to be merged in the master branch first, then backported to the stable/2023.2 branch before a new release candidate can be proposed. Actions ------- Early in the week, the release team will be proposing RC1 patches for all cycle-with-rc projects, using the latest commit from the master branch. If your team is ready to go for cutting RC1, please let us know by leaving a +1 on these patches. If there are still a few more patches needed before RC1, you can -1 the patch and update it later in the week with the new commit hash you would like to use. Remember, stable/2023.2 branches will be created with this, so you will want to make sure you have what you need included to avoid needing to backport changes from the master branch (which will technically then be 2024.1 Bobcat) to this stable branch for any additional RCs before the final release. The release team will also be proposing releases for any deliverable following a cycle-with-intermediary model that has not produced any 2023.2 Bobcat release so far. Finally, now is a good time to finalize release highlights. Release highlights help shape the messaging around the release and make sure that your work is properly represented. Upcoming Deadlines & Dates -------------------------- RC1 deadline: September 14 (R-3 week) Final RC deadline: September 28 (R-1 week) Final 2023.2 Bobcat release: October 4 -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

1 0

[neutron] Neutron drivers meeting cancelled
by Rodolfo Alonso Hernandez 08 Sep '23

08 Sep '23

Hello Neutrinos: Due to the lack of agenda [1], today's meeting is cancelled. Have a nice weekend. [1]https://wiki.openstack.org/wiki/Meetings/NeutronDrivers

1 0

[openstack][neutron][openvswitch] Openvswitch Packet loss when high throughput (pps)
by Ha Noi 07 Sep '23

07 Sep '23

Hi everyone, I'm using Openstack Train and Openvswitch for ML2 driver and GRE for tunnel type. I tested our network performance between two VMs and suffer packet loss as below. VM1: IP: 10.20.1.206 VM2: IP: 10.20.1.154 <https://10.20.1.154/24> VM3: IP: 10.20.1.72 Using iperf3 to testing performance between VM1 and VM2. Run iperf3 client and server on both VMs. On VM2: iperf3 -t 10000 -b 130M -l 442 -P 6 -u -c 10.20.1.206 On VM1: iperf3 -t 10000 -b 130M -l 442 -P 6 -u -c 10.20.1.154 <https://10.20.1.154/24> Using VM3 ping into VM1, then the packet is lost and the latency is quite high. ping -i 0.1 10.20.1.206 PING 10.20.1.206 (10.20.1.206) 56(84) bytes of data. 64 bytes from 10.20.1.206: icmp_seq=1 ttl=64 time=7.70 ms 64 bytes from 10.20.1.206: icmp_seq=2 ttl=64 time=6.90 ms 64 bytes from 10.20.1.206: icmp_seq=3 ttl=64 time=7.71 ms 64 bytes from 10.20.1.206: icmp_seq=4 ttl=64 time=7.98 ms 64 bytes from 10.20.1.206: icmp_seq=6 ttl=64 time=8.58 ms 64 bytes from 10.20.1.206: icmp_seq=7 ttl=64 time=8.34 ms 64 bytes from 10.20.1.206: icmp_seq=8 ttl=64 time=8.09 ms 64 bytes from 10.20.1.206: icmp_seq=10 ttl=64 time=4.57 ms 64 bytes from 10.20.1.206: icmp_seq=11 ttl=64 time=8.74 ms 64 bytes from 10.20.1.206: icmp_seq=12 ttl=64 time=9.37 ms 64 bytes from 10.20.1.206: icmp_seq=14 ttl=64 time=9.59 ms 64 bytes from 10.20.1.206: icmp_seq=15 ttl=64 time=7.97 ms 64 bytes from 10.20.1.206: icmp_seq=16 ttl=64 time=8.72 ms 64 bytes from 10.20.1.206: icmp_seq=17 ttl=64 time=9.23 ms ^C --- 10.20.1.206 ping statistics --- 34 packets transmitted, 28 received, 17.6471% packet loss, time 3328ms rtt min/avg/max/mdev = 1.396/6.266/9.590/2.805 ms Does any one get this issue ? Please help me. Thanks

3 9

[ops]
by Chris Morgan 07 Sep '23

07 Sep '23

I'm trying to reanimate the openstack ops meetups. I moved the social media account used for notifications from X to mastodon (using the fosstodon instance). Please follow there if interested (link below). I also posted a similar announcement to the openstack group on linkedin. One of the first tasks (besides reforming the team) is finding which communications channels actually work. https://fosstodon.org/@osopsmeetup The old account was here https://twitter.com/osopsmeetup now "sepia-tinted" because it's just historical. Cheers, Chris -- Chris Morgan <mihalis68(a)gmail.com>

1 0

[i18n] New I18n SIG Core
by Ian Y. Choi 07 Sep '23

07 Sep '23

Hello, I would like to happily announce a new i18n core: Seongsoo Cho, considering his notable contributions on I18n with volunteering: 1. PDF Docs contribution with I18n issues (font issues, latex with Asian languages) [1] 2. Zanata to Weblate migration volunteering effort [2] 3. Participation of I18n SIG PTG on March [3] and Forum on May [4] Also, he is active on #openstack-i18n IRC channel (nick name: "seongsoocho"). All, please welcome Seongsoo as I18n-core. I am looking forward to more active contributions from him and interactions with other wonderful I18n members and OpenStack team! Thank you, /Ian [1] https://review.opendev.org/q/topic:bp%252Fbuild-pdf-from-rst-guides [2] https://lists.openstack.org/pipermail/openstack-i18n/2022-October/003568.ht… [3] https://etherpad.opendev.org/p/march2023-ptg-i18n [4] https://etherpad.opendev.org/p/vancouver-2023-i18n-forum

4 3

[Glance] Cancelling Weekly Meeting 7th Sept
by Pranali Deore 07 Sep '23

07 Sep '23

Hello, I won't be able to chair today's meeting, my kid is not well, going to take him to the doctor. so cancelling the meeting for this week. Let's meet next week ! Thanks, Pranali D

1 0

Windows Image failed to access network
by Richard Buregeya 07 Sep '23

07 Sep '23

Hello Team, I created the windows image and added the N/w drivers, but it can't be able to get from DHCP. Any idea? Regards Richard.

2 1

[neutron][release] Proposing transition to EOL Train (Neutron and neutron-lib)
by Rodolfo Alonso Hernandez 07 Sep '23

07 Sep '23

Hello: I'm sending this mail in advance to propose transitioning the Neutron and the neutron-lib Train branch to EOL. This topic was proposed and approved during the last Neutron team meeting [1]. These are the only two Neutron related projects still in EM. The announcement is the first step [2] to transition a stable branch to EOL. The patch to mark these branches as EOL will be pushed in two weeks. If you have any inconvenience, please let me know in this mail chain or in IRC (ralonsoh, #openstack-neutron channel). You can also contact any Neutron core reviewer in the IRC channel. Regards. [2] https://meetings.opendev.org/meetings/networking/2023/networking.2023-09-05… [1] https://docs.openstack.org/project-team-guide/stable-branches.html#end-of-l…

1 0

[all][elections][ptl][tc] Conbined PTL/TC 2024.1 cycle Election Voting Kickoff
by Ian Y. Choi 06 Sep '23

06 Sep '23

Polls for PTL and TC elections are now open and will remain open for you to cast your vote until Sep 20, 2023 23:45 UTC. We are selecting 4 TC members, and are having PTL elections for OpenStack_Helm. Please rank all candidates in your order of preference. You are eligible to vote in the TC election if you are a Foundation individual member[0] that also has committed to any official project team's deliverable repositories[1] over the Sep 16, 2022 00:00 UTC - Aug 30, 2023 00:00 UTC timeframe (2023.1 to 2023.2) or if you are in the list of extra-atcs[2] for any official project team. You are eligible to vote in a PTL election if you are a Foundation individual member[0] and had a commit in one of that team's deliverable repositories[1] over the Sep 16, 2022 00:00 UTC - Aug 30, 2023 00:00 UTC timeframe (2023.1 to 2023.2) or if you are in that team's list of extra-atcs[2]. If you are eligible to vote in an election, you should find your email with a link to the Condorcet Internet Voting Service (CIVS) page to cast your vote in the inbox of your Gerrit preferred email[3]. What to do if you don't see the email and have a commit in at least one of the projects having an election: * due to a new CIVS policy, to get the email from poll supervisors, you must opt in to email communication opt in with your Gerrit email address at: https://civs1.civs.us/cgi-bin/opt_in.pl * eligible voters who opt in to email communication late can see a pending poll invitation and vote until Sep 20, 2023 23:45 UTC. * check the trash or spam folders of your Gerrit Preferred Email address, in case it went into trash or spam * wait a bit and check again, in case your email server is a bit slow * find the sha of at least one commit from the project's deliverable repos[0] and email the election officials[4]. If we can confirm that you are entitled to vote, we will add you to the voters list for the appropriate election. Our democratic process is important to the health of OpenStack, please exercise your right to vote! Candidate statements/platforms can be found linked to Candidate names on this page: https://governance.openstack.org/election/ Happy voting, [0] https://www.openstack.org/community/members/ [1] The list of the repositories eligible for electoral status: https://opendev.org/openstack/governance/src/tag/0.15.0/reference/projects.… [2] Look for the extra-atcs element in [1] [3] Sign into review.openstack.org: Go to Settings > Contact Information. Look at the email listed as your preferred email. That is where the ballot has been sent. [4] https://governance.openstack.org/election/#election-officials

1 0

[manila] Cancelling Sep 7 weekly meeting
by Carlos Silva 06 Sep '23

06 Sep '23

Hello, Zorillas! As discussed in the previous meeting, we are cancelling tomorrow's weekly meeting, as a big part of the audience is on PTO tomorrow/this week. The next Manila weekly meeting will be on Sep 14th. If you have something urgent to bring up, please let me know on the #openstack-manila IRC channel. Regards, carloss

1 0

[largescale-sig] Next meeting: Sept 6, 15utc
by Thierry Carrez 06 Sep '23

06 Sep '23

Hi everyone, The Large Scale SIG is back after the northern hemisphere summer break and will be meeting this Wednesday in #openstack-operators on OFTC IRC, at 15UTC, our EU+US-friendly time. I will be chairing. You can doublecheck how that UTC time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20230906T15 Feel free to add topics to the agenda: https://etherpad.opendev.org/p/large-scale-sig-meeting Regards, -- Thierry Carrez

1 1

[kolla-ansible][octavia] octavia management network setup for vlan provider
by Satish Patel 06 Sep '23

06 Sep '23

Folks, I have setup kolla-ansible and configured octavia using the o-hm0 interface with the tenant and it works. For production I think I should use VLAN based provider for octavia management network so this is what I did I have created a bond0.41 dedicated interface on all 3 controller nodes and created vlan 41 on all network switches. This is what my global.yml looks like ## Octivia enable_octavia: "yes" octavia_network_interface: "bond0.41" octavia_amp_flavor: name: "amphora" is_public: no vcpus: 2 ram: 2048 disk: 5 octavia_amp_network: name: lb-mgmt-net provider_network_type: vlan provider_segmentation_id: 41 provider_physical_network: physnet1 external: false shared: false subnet: name: lb-mgmt-subnet cidr: "192.168.41.0/24" allocation_pool_start: "192.168.41.100" allocation_pool_end: "192.168.41.200" enable_dhcp: yes After running the playbook all get setup as per document. When I create loadbalancer it just get stuck in PENDING status. [1] Document saying make sure your octavia_network_interface is connected to openvswitch. Do I need to connect manually or will kolla-ansible do that for me? If I am going to do that then on which bridge I should attach br-ex or br-int ? [1] https://docs.openstack.org/kolla-ansible/latest/reference/networking/octavi…

3 3

Cinder Bug Report 2023-09-06
by Jon Bernard 06 Sep '23

06 Sep '23

Hello Argonauts, Cinder Bug Meeting Etherpad <https://etherpad.opendev.org/p/cinder-bug-squad-meeting> Undecided - Weak Cryptographic Algorithm (MD5) used <https://bugs.launchpad.net/cinder/+bug/2033612> - Status: New - group list doesn't parse 'all_tenants' parameter correctly <https://bugs.launchpad.net/cinder/+bug/2034248> - Status: In Progress - Documentation check and correction for PowerStore NFS driver <https://bugs.launchpad.net/cinder/+bug/2034482> - Status: New Thanks -- Jon

1 0

[all][elections][ptl][tc] Opt in to CIVS voting system
by Ian Y. Choi 05 Sep '23

05 Sep '23

The election uses the Condorcet Internet Voting Service (CIVS). Due to CIVS policy, to vote in private CIVS polls, you must opt in to email communication. To opt in, please enter your Gerrit email address in the following page before the start of the election Sep 06, 2023 23:45 UTC, and confirm with the code that will sent to you via email. https://civs1.civs.us/cgi-bin/opt_in.pl If you have any question, please contact the election officials. https://governance.openstack.org/election/#election-officials

1 0

[election] [tc] Running for a second year on the TC
by Jay Faulkner 05 Sep '23

05 Sep '23

Hey all, Around a year ago, I announced my candidacy for the TC after a call for more contributors from a diverse employer base was made. I think a year later, it's more obvious than ever how important it is to have a diverse, independent technical committee. We have made huge progress on this front. We have six candidates for the four open TC positions, representing different independent companies utilizing OpenStack. This is a major improvement year over year. If you've been in the community a while, you've probably worked with me and have an idea if you want to vote for me or not in this position. Whether elected or not, I'll continue my vocal support of OpenStack technologies in private meetings and on social media, I'll continue to invite people I meet interested in cloud to join our community -- not because it's my job (it is), but also because I'm proud of what we've built. When I have those conversations with potential users or contributors, I highlight two things I admire about the OpenStack community: longevity -- there are not many large, distributed open source projects that have survived as long as we have, and the four opens -- putting openness out front as our face to the world is why I'll continue to be proud to represent OpenStack in the future -- as a TC member or not (you decide). If someone does want to talk about a particular policy or thought process of mine, I'm happy to respond to comments here or via private email/IRC. Thanks, Jay Faulkner

1 0

[QA][release] Cycle With Intermediary without recent deliverables
by Herve Beraud 05 Sep '23

05 Sep '23

Quick reminder that for deliverables following the cycle-with-intermediary model, the release team will use the latest $series release available on release week. The following deliverables have done a 2023.2 bobcat release, but it was not refreshed in the last two months: tempest You should consider making a new one very soon, so that we don't use an outdated version for the final release. Thanks -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

2 1

Host aggregates and placement aggregates
by Gk Gk 05 Sep '23

05 Sep '23

Hi All, We have a yoga OSA setup. We have upgraded the setup from Xena to Yoga. We have found that some of the host aggregates have not synced to the placement aggregates. As a result , the vm creation on those aggregate nodes is failing even though they have the full capacity. I have tried the placement sync aggregates command but it did not help. Is this a known bug in Yoga ? Thanks Y.G

1 0

[openstack-ansible] Recommended branch for a production environment?
by Roger Rivera 05 Sep '23

05 Sep '23

Hello, We have deployed Openstack-Ansible in a test environment and we were wondering what the recommended repository branch is to deploy in a production environment that will be integrated with a separate Ceph cluster. 1. We are thinking about pinning to the stable/zed. Is that recommended over the master branch? 2. Are bugs ironed out on both master and stable/zed with the same cadence? 3. Additionally, is Debian 11 a better alternative than Ubuntu 22 for target hosts? We noticed Ubuntu 22.04 support was added recently, whereas Debian 11 has been supported for quite some time now which leads us to believe its stability/integration could be more mature at this point? Any suggestions will be appreciated. Thank you, -- *Roger *

3 7

What is the difference between 4 and 8 virtual sockets to physical sockets?
by Jorge Visentini 05 Sep '23

05 Sep '23

Hello Team, What is the difference between creating an instance with *4* or *8 virtual sockets*, since the hypervisor has only *4 physical sockets*. My question is where do sockets, cores and virtual threads fit into the physical hardware. I think this question is not just related to Openstack, but with any virtualization. My hypervisor configuration is as follows: CPU(s): 192 Online CPU(s) list: 0-191 Thread(s) per core: 2 Core(s) per socket: 24 Socket(s): 4 NUMA node(s): 4 Do you have any documentation that I can read and understand better? That we have a nice week! -- Att, Jorge Visentini +55 55 98432-9868

2 3

[tc] Technical Committee next weekly meeting Today on September 5
by Nikolla, Kristi 05 Sep '23

05 Sep '23

Hi all, This is a reminder that the next weekly Technical Committee meeting is to be held on Tuesday, September 5, 2023 at 1800 UTC on Zoom https://us06web.zoom.us/j/87108541765?pwd=emlXVXg4QUxrUTlLNDZ2TTllWUM3Zz09 The agenda can be found at the link https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Agenda_Suggesti… and attached below: • Roll call • Follow up on past action items • No action items • Gate health check • OpenStack Elections • #link https://governance.openstack.org/election/ • Ballots go out tomorrow and remain open until September 20. • We have 6 candidates out of 4 seats for TC :) • Open Discussion and Reviews • Register for the PTG • #link https://openinfra.dev/ptg/ • #link https://review.opendev.org/q/projects:openstack/governance+is:open Thank you, Kristi Nikolla

1 0

RE: ***UNCHECKED*** openstack-discuss Digest, Vol 59, Issue 10 | Openstack Web Interface Issue
by Asma Naz Shariq 05 Sep '23

05 Sep '23

Hi Team, I have deployed Openstack through manual installation-Yoga release. I encountered the attached error and can't have access to Openstack dashboard. When I entered the login credentials it displays something went wrong and you don't have permission to use /horizon/. Can anyone have a solution for this error. -----Original Message----- From: openstack-discuss-request(a)lists.openstack.org <openstack-discuss-request(a)lists.openstack.org> Sent: Monday, September 4, 2023 8:27 PM To: openstack-discuss(a)lists.openstack.org Subject: ***UNCHECKED*** openstack-discuss Digest, Vol 59, Issue 10 Send openstack-discuss mailing list submissions to openstack-discuss(a)lists.openstack.org To subscribe or unsubscribe via the World Wide Web, visit https://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss or, via email, send a message with subject or body 'help' to openstack-discuss-request(a)lists.openstack.org You can reach the person managing the list at openstack-discuss-owner(a)lists.openstack.org When replying, please edit your Subject line so it is more specific than "Re: Contents of openstack-discuss digest..." Today's Topics: 1. Re: Live migration error (Karera Tony) 2. What is the difference between 4 and 8 virtual sockets to physical sockets? (Jorge Visentini) 3. [all][stable][ptl] Propose to EOL Stein series (El?d Ill?s) ---------------------------------------------------------------------- Message: 1 Date: Mon, 4 Sep 2023 14:11:37 +0200 From: Karera Tony <tonykarera(a)gmail.com> To: smooney(a)redhat.com Cc: Danny Webb <Danny.Webb(a)thehutgroup.com>, openstack-discuss <openstack-discuss(a)lists.openstack.org> Subject: Re: Live migration error Message-ID: <CA+69TL3=7GwfgRgAjfxvx5q4aZCstXxtBeB2mGb9ofa0JCZmbA(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hello Smooney, I can confirm that it is disabled on all the servers. Regards Tony Karera On Mon, Sep 4, 2023 at 1:25?PM <smooney(a)redhat.com> wrote: > On Mon, 2023-09-04 at 12:59 +0200, Karera Tony wrote: > > Hello Danny, > > > > I am actually using Broadwell-noTSX-IBRS on all the servers but I > > have > seen > > that some flags are not there in the older servers. > > > > The flags below to be specific. > > > > hwp hwp_act_window hwp_pkg_req > so hwp is apprentlly intels pstate contol based on > https://unix.stackexchange.com/a/43540 > > i dont belive this is normally exposed to a guest but if you are > seeign differnce between hosts then that likely means you have > disabled pstates also know as "intel speedstep" on some of the host > and not others. > > > > > Regards > > > > Tony Karera > > > > > > > > > > On Fri, Sep 1, 2023 at 4:05?PM <smooney(a)redhat.com> wrote: > > > > > On Fri, 2023-09-01 at 14:56 +0200, Karera Tony wrote: > > > > Hello Danny, > > > > > > > > Thanks for the feedback. > > > > > > > > use a lowest common denominator cpu_model : Does this mean > > > > changing > the > > > > servers ? > > > > > > it means tha tif you have a mix of cpus in the deployment you > > > shoudl hardcod to the older of the diffent cpu models i.e skylake > > > or whatever it may > be > > > in your case. > > > > > > > > > > https://docs.openstack.org/nova/latest/configuration/config.html#libvi > rt.cpu_models > > > > > > [libvirt] > > > cpu_mode=custom > > > cpu_models=skylake > > > > > > in more recent release we replaced the older cpu_model with > > > cpu_models with is a comma sperated list of models in prefernce > > > order. i.e. nova will use the first > cpu_modle > > > in the list that work > > > on the current host. not that while this give extra flexiblity it > > > vs > just > > > using the oldest cpu model > > > it does limit the set of hosts you can migrate too but means you > > > can > have > > > better perfromance so its > > > a tradeoff. performance vs portablity. > > > > > > the error you mentioned can also be caused by micorocode updates. > > > intel remove TSX and MPX i beilve in the last 2 years form come of > there > > > cpus > > > that breaks live migration if a vm was create with access to that > > > cpu instruction > > > > > > the only way to resolve that is to cold migrate the guest. > > > i.e. if a vm is currently booted with cpu_model X it cannot be > > > modifed while the guest is running so you either need to update > > > the config option and reboot all the vms > or > > > more particlaly update the config > > > and then cold migrate them which will allow you to move the > instnace(your > > > orginal goal) while allowing > > > the new cpu model to take effect. > > > > > > novas default for the cpu model when not set is with our default > cpu_mode > > > is host_model > > > meaning whatever model best matches the current hosts cpu. > > > > > > that is the correct default in general but if you have mixed cpu > > > generatiosn in your cloud its not ideal. > > > > > > hopefuly that helps > > > > > > > > > > > Regards > > > > > > > > Tony Karera > > > > > > > > > > > > > > > > > > > > On Fri, Sep 1, 2023 at 12:37?PM Danny Webb < > Danny.Webb(a)thehutgroup.com> > > > > wrote: > > > > > > > > > It means that you have CPUs with incompatible flags or you've > > > > > got differing kernel versions that expose different flags or > > > > > you've got differing libvirt versions that expose different > > > > > flags. You either > > > need to > > > > > use a lowest common denominator cpu_model or do a cold migration. > > > > > ------------------------------ > > > > > *From:* Karera Tony <tonykarera(a)gmail.com> > > > > > *Sent:* 01 September 2023 10:13 > > > > > *To:* openstack-discuss > > > > > <openstack-discuss(a)lists.openstack.org> > > > > > *Subject:* Live migration error > > > > > > > > > > > > > > > * CAUTION: This email originates from outside THG * > > > > > ------------------------------ Hello Team, > > > > > > > > > > I am trying to migrate instances from one host to another but > > > > > I am > > > getting > > > > > this error. > > > > > > > > > > *Error: *Failed to live migrate instance to host "compute1". > Details > > > > > <http://10.10.13.31/admin/instances/#message_details> > > > > > Migration pre-check error: Unacceptable CPU info: CPU doesn't > > > > > have compatibility. 0 Refer to > > > > > > http://libvirt.org/html/libvirt-libvirt-host.html#virCPUCompareResult > > > > > (HTTP 400) (Request-ID: > > > > > req-9dfdfe2e-70fa-481a-bd3d-c2c76a6e93da) > > > > > > > > > > Any idea on how to fix this? > > > > > > > > > > Regards > > > > > > > > > > Tony Karera > > > > > > > > > > > > > > > > > > > > > > >

1 0

Need assistance
by Gk Gk 04 Sep '23

04 Sep '23

Hi, We have Yoga version setup of OSA. When a user is trying to use a certain flavor which has some extra specs, same as that which is mentioned for an aggregate, the vm creation is failing for the user. But I created a vm using that same flavor and it succeeded for a test user account. When I analyzed the logs, the nova scheduler is not filtering or checking the individual hosts from that target aggregate in the case of that user. But it is validating the target aggregate hosts in the case of a test user and able to select a host for vm creation. So what could be the reason that the scheduler is unable to find the hosts from that aggregate in the case of that user as opposed to that of the test user ? Is there any connection between a project and an aggregate ? Thanks Kumar

2 1

[all][qa] Devstack dropping support for Ubuntu Focal
by Ghanshyam Mann 04 Sep '23

04 Sep '23

Hello Everyone, As you know, this release 2023.2 (bobcat) does not mandate testing the ubuntu Focal[1] and with Nova bumped the libvirt version, focal jobs are failing. Devstack was planning to drop the Focal support in the next cycle but we have to do it now only[2]. This email is a heads-up if you are still wondering about the Focal job failure. If your project still running any of the jobs on focal, please plan to move to Jammy. [1] https://governance.openstack.org/tc/reference/runtimes/2023.2.html [2] https://review.opendev.org/c/openstack/devstack/+/885468 -gmann

1 0

[all][stable][ptl] Propose to EOL Stein series
by Elõd Illés 04 Sep '23

04 Sep '23

Hi, Core projects announced and deleted their stable/stein branches for some time already. Furthermore, most of the projects, where stable/stein is still open, have broken gates. So, the same way as we transitioned Rocky series to End of Life [1][2], now is the time to do the same with Stein. I'll propose the stein-eol transition patches for the rest of the open projects. I ask again the teams to signal their decision (with a +1 if the team is ready for the transition). Thanks in advance! (Please note, that the Extended Maintenance process is phasing out / transforming soon, see details in Technical Committee's resolution [3], which was accepted and merged in August.) Thanks, Előd Illés irc: elodilles @ #openstack-stable / #openstack-release [1] https://lists.openstack.org/pipermail/openstack-discuss/2023-January/031922… [2] https://lists.openstack.org/pipermail/openstack-discuss/2023-April/033386.h… [3] https://governance.openstack.org/tc/resolutions/20230724-unmaintained-branc…

1 0

Live migration error
by Karera Tony 04 Sep '23

04 Sep '23

Hello Team, I am trying to migrate instances from one host to another but I am getting this error. *Error: *Failed to live migrate instance to host "compute1". Details <http://10.10.13.31/admin/instances/#message_details> Migration pre-check error: Unacceptable CPU info: CPU doesn't have compatibility. 0 Refer to http://libvirt.org/html/libvirt-libvirt-host.html#virCPUCompareResult (HTTP 400) (Request-ID: req-9dfdfe2e-70fa-481a-bd3d-c2c76a6e93da) Any idea on how to fix this? Regards Tony Karera

3 6

[PTL][release] 2023.2 Bobcat Cycle Highlights
by Herve Beraud 04 Sep '23

04 Sep '23

Hi, This is a reminder that Cycle highlights need to be added to deliverable yamls so that they can be included in release marketing preparations. (See the details about how to add them at the project team guide [1]) Note that the deadline was the Feature Freeze date in previous cycles, but due to requests of PTLs it was decided to postpone one week in the future (to R-4 week). This is better for PTLs as they are usually very busy around Feature Freeze, on the other hand gives less time to marketing team to process the highlights. [1] https://docs.openstack.org/project-team-guide/release-management.html# cycle-highlights Thanks, -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/

1 0

[neutron] Bug Deputy Report August 28 - September 3
by Elvira Garcia Ruiz 03 Sep '23

03 Sep '23

Hi! I was the bug deputy last week. Find the summary below: High ------ - https://bugs.launchpad.net/neutron/+bug/2033508 - [HA] "neutron-dynamic-routing" HA unittest failing Unassigned - https://bugs.launchpad.net/neutron/+bug/2033493 - [ndr] Unit tests failing due to a missing patch, still unreleased, in Neutron Fix proposed: https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/893189 Assigned to Rodolfo Alonso Hernandez - https://bugs.launchpad.net/neutron/+bug/2033305 - OVN metadata agent keeps restarting Assigned to Miro Tomaska Fix proposed: https://review.opendev.org/c/openstack/neutron/+/890986 - https://bugs.launchpad.net/neutron/+bug/2033556 - Documentation for DNS integration is incomplete Assigned to Dr. Jens Harbott - https://bugs.launchpad.net/neutron/+bug/2033887 - [OVN][Trunk] The cold migration process is broken since patch 882581 Assigned to Rodolfo Fix proposed: https://review.opendev.org/c/openstack/neutron/+/893447 + backports - https://bugs.launchpad.net/neutron/+bug/2033752 - test_reboot_server_hard fails with AssertionError: time.struct_time() not greater than time.struct_time() This is affecting us but the failure is on Nova. Yatin will monitor this. - https://bugs.launchpad.net/neutron/+bug/2033932 - Add support for OVN MAC_Binding aging Fix proposed: https://review.opendev.org/c/openstack/neutron/+/893575 Assigned to Terry Medium ---------- - https://bugs.launchpad.net/neutron/+bug/2033281 - [OVN] ovn_hash_ring table cleanup In progress. Fix merged on master: https://review.opendev.org/c/openstack/neutron/+/893030 Assigned to Lucas Low ------ - https://bugs.launchpad.net/neutron/+bug/2033651 - [fullstack] Reduce the CI job time Unassigned Incomplete --------------- - https://bugs.launchpad.net/neutron/+bug/2033293 - dns integration saying plugin does not match requirements Unassigned Left a comment since I think this might not be a bug. Also contacted mlavalle to get more information. Undecided --------------- https://bugs.launchpad.net/neutron/+bug/2033683 - openvswitch.agent.ovs_neutron_agent fails to Cmd: ['iptables-restore', '-n'] I’m not very sure about how to confirm this one. Unassigned Kind regards, Elvira

1 0

[SDK][CLI][API][OpenAPI] Machine readable OpenStack API spec - time to do a next step?
by Artem Goncharov 01 Sep '23

01 Sep '23

Hey, Some of you may know that for certain time I was doing experiments on the sdk/cli/ansible_mod side with regards to the code generation. Reason for that is simple: huge codebases and too few resources. While not being the primary goal of this initiative, but still a pointer into the same direction is that most of the tools we need to maintain do the same stuff with some adaptations necessary for the framework in question. I did a lot of diverse attempts trying to take current OpenStackSDK codebase as a “source of truth” regarding what exists in which service and use it to generate code (OSC, AnsibleModule, etc). Pretty quickly it became clear that we do not have enough information to cover all the different needs: - when query parameter has been added into or dropped from the API - what is the type of the query parameter and whether in the case it is list it should be rendered as "?tags=a,b" or "?tags=a&tags=b” - which fields are necessary to be send in the request, which are optional and how to deal when you send name but get back UUID in the same field - what are the parameter/resource fields types in detail with respect to strongly typed programming languages (“int” is not good to differentiate between u8 and f64) - how to determine list of supported “actions” on the resource which are above the plain CRUD Firstly I tried to extend our current model of things in SDK, but even that was not so helpful. And then I decided to give another look at OpenAPI and our desires since multiple years to move into that direction. First of all it provides all the required information to do what is necessary. And so I tried to gather all the things we said about OpenAPI at OpenStack and which challenges we have and compare it against the OpenAPI 3.1. Problems: 1) OpenAPI does not support model of microversions. To be more precise on that you can’t describe different request/response body based on the header differentiator 2) URL + Method are representing unique combination making it impossible to represent different actions based on the content 3) All relevant Headers are supposed to be described in the spec. For example Swift is using X-Object-Meta-* prefix to allow user customisations what is not currently supported by the spec Actually that’s it, everything else was doable. Have I missed something? So what can we do about it: 1) OpenAPI 3.1 is now based upon jsonschema, what allows huge flexibility in describing object schema including inheritance and polymorphism (with composition, discriminator, etc). So we can describe the necessary Microversion Header and schema using “oneOf” whenever necessary. Moreover it is possible to extend type schema with “x-“ prefixed custom properties. So I went and started adding “x-openstack-“ prefixed properties where necessary 2) I personally agree with OpenAPI that URL+method should be enough to uniquely identify what is expected (while not a big deal for the API consumer, it would have been easier on the server side to have URL uniquely tied with the data and operation instead of dynamically looking at the request to figure out what to do. This is actually a most complex thing to deal with and the only solution that comes to my mind would be not to try to put everything into single spec, but rather have a dedicated specs for different operations. It makes it anyway easier to maintain such spec. Maybe at some next OpenAPI iteration this gets resolved and we can adopt changes. 3) A tricky thing, but there is nothing preventing us specifying headers as “x-object-meta-*” and defining custom processors for things like that to for example compose/decompose this into dictionary on the client side OpenAPI now even allows defining spec extensions if necessary, but I think at the moment we can cover all issues without that. Demo time: - server live migration (in my eyes one of the trickiest things I have seen so far from the OpenStack APIs). Here you have an action requiring a dedicated spec, multiple revisions with micro versions and even parameter type bool which can be a string as well: [2] https://paste.opendev.org/show/bcxFi2CrNX1YNEuTzoEh/ More real-life examples in [4] - image create and upload: [3] https://paste.opendev.org/show/bNN1t0qVmHEdVHu0Mtrn/ Here it is possible to put few methods into the single spec. You can also see here samples of different media-types, capture headers, readOnly/writeOnly properties (You can actually take those specs and paste them into https://editor-next.swagger.io/ and see how it looks like) Closing notes: I have implemented a WIP change [1] in SDK that consumes list of specs and performs requested operation. Series of SDK commands are converted to the new way dealing with APIs. This is backwards compatible, but opens a road to heavily reduce amount of code in SDK (precisely by dropping all those customisations that are necessary now). Honestly speaking my drafts even show performance improvement on SDK side in the area of 100-300ms for diverse API operations, but I am still not really able to explain why. Maybe because of avoiding most black magic of SDK. I assume after completion we could get another performance boost by dropping other unnecessary logic and abstractions. The work is not completed as the change title states. Once we have specs for the commands code generators can be updated to consume this data and produce much better code. BTW, I have now also a prototype of the new CLI written with Rust and this is a hypersonic bullet compared to the current OSC. But please do not push me more on that, it is still in a very early stages and depends heavily on the available specs. I can only say that it is now designed in the way that every API call can have a thin CLI coverage just by providing a spec, when additional logic is desired - surely will require human implementation. Code generators in the pipe: OSC, AnsibleModules, RustSDK (sync/async), RustCLI. Next thing that are on the radar: gopher cloud, terraform modules, async python sdk, JS SDK(?) If all of that gets executed properly and with some community traction we can all have following things covered: - improve standardisation of OpenStack internals and externals: glance and nova (at least those 2) are already using jsonschema internally in different areas to describe requests/responses. Why not to make this standard reaching the service consumers? - getting rid of api-ref work by updating our sphinx machinery to consume our customised specs and produce nice docs matching the reality - sharing specs between teams to improve interface (not like currently we need to read the api-ref with tons of bugs plus source code to understand how to cover new feature in service X). Maybe even a central repo with the specs per release. - there are plenty of code generators and server bindings for OpenAPI specs so that we can potentially align frameworks used by different teams to maintain less - less work for all of us who needs services talking to each other (not immediately right now, but once the code is switched on consuming specs) - request verification already on the client side not waiting for the response - finally show something to customers often annoying asking “where are your openapi specs” (no offence here ;-))? I know it is a long message. But I am pretty excited with the progress and would like to hear community opinions. For the more detailed discussion consider this as a pre-announcement of the topic for PTG in sdk/cli slots. Huge invest but huge outcome P.S. it can result in a good chunk of relatively easy work for students Regards, Artem [1] https://review.opendev.org/c/openstack/openstacksdk/+/892161 [2] https://paste.opendev.org/show/bcxFi2CrNX1YNEuTzoEh/ [3] https://paste.opendev.org/show/bNN1t0qVmHEdVHu0Mtrn/ [4] https://review.opendev.org/c/openstack/openstacksdk/+/893365

4 5

[release] Release countdown for week R-4, Sep 04-08
by Elõd Illés 01 Sep '23

01 Sep '23

Development Focus ----------------- We just passed feature freeze! Until release branches are cut, you should stop accepting featureful changes to deliverables following the cycle-with-rc release model, or to libraries. Exceptions should be discussed on separate threads on the mailing-list, and feature freeze exceptions approved by the team's PTL. Focus should be on finding and fixing release-critical bugs, so that release candidates and final versions of the 2023.2 Bobcat deliverables can be proposed, well ahead of the final 2023.2 Bobcat release date. General Information ------------------- We are still finishing up processing a few release requests, but the 2023.2 Bobcat release requirements are now frozen. If new library releases are needed to fix release-critical bugs in 2023.2 Bobcat, you must request a Requirements Freeze Exception (RFE) from the requirements team before we can do a new release to avoid having something released in 2023.2 Bobcat that is not actually usable. This is done by posting to the openstack-discuss mailing list with a subject line similar to: [$PROJECT][requirements] RFE requested for $PROJECT_LIB Include justification/reasoning for why a RFE is needed for this lib. If/when the requirements team OKs the post-freeze update, we can then process a new release. A soft String freeze is now in effect, in order to let the I18N team do the translation work in good conditions. In Horizon and the various dashboard plugins, you should stop accepting changes that modify user-visible strings. Exceptions should be discussed on the mailing-list. By September 28th, 2023 this will become a hard string freeze, with no changes in user-visible strings allowed. Actions ------- stable/2023.2 branches should be created soon for all not-already-branched libraries. You should expect 2-3 changes to be proposed for each: a .gitreview update, a reno update (skipped for projects not using reno), and a tox.ini constraints URL update. Please review those in priority so that the branch can be functional ASAP. The Prelude section of reno release notes is rendered as the top level overview for the release. Any important overall messaging for 2023.2 Bobcat changes should be added there to make sure the consumers of your release notes see them. Upcoming Deadlines & Dates -------------------------- RC1 deadline: September 14th, 2023 (R-3 week) Final RC deadline: September 28th, 2023 (R-1 week) Final 2023.2 Bobcat release: October 4th, 2023 2024.1 Caracal Virtual PTG - October 23-27, 2023 Előd Illés irc: elodilles @ #openstack-release

1 0

[cinder][FFE] Granting FFE to features (8th September)
by Rajat Dhasmana 01 Sep '23

01 Sep '23

Hi All, We had 6 feature requests during the 2023.2 (Bobcat) development cycle[1]. Few of the features had other patch dependencies amounting to significantly more patches. With limited review bandwidth, we were not able to review all the features on time, hence I would like to grant FFE to the following listed features. The reason so many changes are granted FFE are due to the following reasons: * It has gone through iterations of updates and most of them has +2s so it should just require another core to merge * The response time for addressing review comments is fast which is a good thing * The changes are driver specific and isolated to a particular driver that have been tested by the vendor team and third party CI 1. Fujitsu Driver: Add QoS support Patch: https://review.opendev.org/c/openstack/cinder/+/847730 2. NetApp ONTAP: Added support to Active/Active mode in NFS driver Patch: https://review.opendev.org/c/openstack/cinder/+/886869 Dependent patches: https://review.opendev.org/c/openstack/cinder/+/798384 https://review.opendev.org/c/openstack/cinder/+/889826 3. [NetApp] LUN space-allocation support for iSCSI Patch: https://review.opendev.org/c/openstack/cinder/+/893106 4. [Pure Storage] Replication-Enabled and Snapshot Consistency Groups Patch: https://review.opendev.org/c/openstack/cinder/+/891234 5. [HPE XP] Support HA and data deduplication Patch: https://review.opendev.org/c/openstack/cinder/+/892608 Dependent patches: https://review.opendev.org/c/openstack/cinder/+/885986 https://review.opendev.org/c/openstack/cinder/+/879830 https://review.opendev.org/c/openstack/cinder/+/877672 https://review.opendev.org/c/openstack/cinder/+/891937 [1] https://etherpad.opendev.org/p/cinder-2023.2-bobcat-features The deadline after FFE is going to be 8th September, 2023 but we will try to get the changes reviewed and merged at the earliest. Let me know if there are any queries or concerns. Thanks Rajat Dhasmana

1 0

[neutron] Neutron drivers meeting cancelled
by Rodolfo Alonso Hernandez 01 Sep '23

01 Sep '23

Hello Neutrinos: Due to the lack of agenda [1], today's meeting is cancelled. Have a nice weekend. [1]https://wiki.openstack.org/wiki/Meetings/NeutronDrivers

1 0

[cinder] Cinder Midcycle - 2 (R-6) for 2023.2 (Bobcat) Summary
by Rajat Dhasmana 01 Sep '23

01 Sep '23

Hello Argonauts, The summary for mid cycle -2 (R-6) held on 23rd August, 2023 from 1400 to 1600 UTC is available at the wiki page[1]. Please go through the etherpad and the recording for the entire discussion. [1] https://wiki.openstack.org/wiki/CinderBobcatMidCycleSummary Thanks Rajat Dhasmana

1 0

[all] osprofiler requirements on opentelemetry: is this reasonable?
by Thomas Goirand 01 Sep '23

01 Sep '23

Hi, I just saw that the last version of osprofiler requires opentelemetry-exporter-otlp and opentelemetry-sdk. Since osprofiler is used in almost all projects, and that opentelemetry is really HUGE, I wonder if this is all reasonable. This forces downstream distros to do a lot of packaging only for that. Is this avoidable? I started doing the packaging, and then stopped to write this message, seeing how much work that would represent. Your thoughts? Cheers, Thomas Goirand (zigo)

3 3

[Telemetry] Adding Erno Kuvaja (jokke_) to telemetry cores
by Matthias Runge 01 Sep '23

01 Sep '23

Hi there, a bit premature, but in-line with[1] I've added Erno to telemetry cores. Erno is not a stranger in the OpenStack world and has been deeply involved in glance for quite a while. Thank you Erno for stepping up and for helping with Telemetry! Matthias [1] https://governance.openstack.org/election/#ptl-candidates -- Matthias Runge <mrunge(a)matthias-runge.de>

1 0