October 2023 - openstack-discuss - lists.openstack.org

[openstack][kolla-ansible]about group node config overwrite
by Nguyễn Hữu Khôi 30 Oct '25

30 Oct '25

Hello Koller. I want to ask if we can overwrite config for a group of nodes. For example, compute01 >> compute10 will have the same nova configuration compute11 >> compute20 will have the same nova configuration. It will be very nice if we support this way. Thank you. Regards Nguyen Huu Khoi

4 7

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

How to config DPDK compute in OVN
by dangerzone ar 22 Apr '25

22 Apr '25

Hi....as far I understand ovs does support dpdk compute to be deployed. I'm referring to the guideline ' *https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html <https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html>* ' ... It's too brief.. May I know how to config dpdk compute in OVN networking openstack? Hope someone could help what is the config to be done.... Currently compute is run without dpdk over openstack train ovn based. Please need some info on how I can proceed. Thank you

2 1

ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version
by Swogat Pradhan 24 Dec '24

24 Dec '24

Hi, I am currently trying to deploy openstack wallaby using tripleo arch. I created the network jinja templates, ran the following commands also: #openstack overcloud network provision --stack overcloud --output networks-deployed-environment.yaml custom_network_data.yaml # openstack overcloud network vip provision --stack overcloud --output vip-deployed-environment.yaml custom_vip_data.yaml # openstack overcloud node provision --stack overcloud --overcloud-ssh-key /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml and used the environment files in the openstack overcloud deploy command: (undercloud) [stack@hkg2director ~]$ cat deploy.sh #!/bin/bash THT=/usr/share/openstack-tripleo-heat-templates/ CNF=/home/stack/ openstack overcloud deploy --templates $THT \ -r $CNF/templates/roles_data.yaml \ -n $CNF/workplace/custom_network_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e $CNF/templates/node-info.yaml \ -e $CNF/templates/scheduler-hints.yaml \ -e $CNF/workplace/networks-deployed-environment.yaml \ -e $CNF/workplace/vip-deployed-environment.yaml \ -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ -e $CNF/workplace/custom-net-bond-with-vlans.yaml Now when i run the ./deploy.sh script i encounter an error stating: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command.: tripleoclient.exceptions.InvalidConfiguration: Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command. 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END return value: 1 Can someone tell me where the mistake is? With regards, Swogat Pradhan

4 4

[kolla] [openeuler] CI jobs status
by Michał Nasiadka 26 Nov '24

26 Nov '24

Hello Koalas, OpenEuler jobs in master (to be Antelope) are failing for a long time. If no one is interested in maintaining OpenEuler support in Kolla-Ansible - I will propose dropping it’s support on Wednesday Kolla meeting. Best regards, Michal

4 3

Kolla Ansible on Ubuntu 20.04 - cloud-init & other network issues
by Tobias McNulty 10 Oct '24

10 Oct '24

Hi, I'm attempting to use Kolla Ansible 14.6.0 to deploy OpenStack Yoga on a small 3-node Ubuntu 20.04 cluster. The nodes have 128 GB RAM each, dual Xeon processors, and dual 10G Intel NICs. The NICs are connected to access ports on a 10G switch with separate VLANs for the local and external networks. All the playbooks run cleanly, but cloud-init is failing in the Ubuntu 20.04 and 22.04 VMs I attempt to boot. The VM images are unmodified from https://cloud-images.ubuntu.com/, and cloud-init works fine if I mount a second volume with user-data. The error is a timeout attempting to reach 169.254.169.254. This occurs both when booting a VM in an internal routed network and directly in an external network. I tried various neutron plugin agents (ovn, linuxbridge, and openvswitch both with and without firewall_driver = openvswitch <https://docs.openstack.org/kolla-ansible/latest/reference/networking/neutro…>) first with a clean install of the entire OS each time, all with the same result. Running tcpdump looking for 169.254.169.254 shows nothing. As a possible clue, the virtual NICs are unable to pass any traffic (e.g., to reach an external DHCP server) unless I completely disable port security on the interface (even if the associated security group is wide open). But disabling port security does not fix cloud-init (not to mention I don't really want to disable port security). Are there any additional requirements related to deploying OpenStack with Kolla on Ubuntu 20.04? This is a fairly vanilla configuration using the multinode inventory as a starting point. I tried to follow the Quick Start <https://docs.openstack.org/kolla-ansible/yoga/user/quickstart.html> as closely as possible; the only material difference I see is that I'm using the same 3 nodes for control + compute. I am using MAAS so it's easy to get a clean OS install on all three nodes ahead of each attempt. I plan to try again with the standard (non-HWE) kernel just in case, but otherwise I am running out of ideas. In case of any additional clues, here are my globals.yml and inventory file, along with the playbook I'm using to configure the network, images, VMs, etc., after bootstrapping the cluster: https://gist.github.com/tobiasmcnulty/7dbbdbc67abc08cbb013bf5983852ed6 Thank you in advance for any advice! Cheers, Tobias

5 9

Live migration fails
by Szabo, Istvan (Agoda) 07 Aug '24

07 Aug '24

Hi, I have couple of compute nodes where the live migration fails with existing vms. When I quickly spawn a vm and try live migration it works so I assume shouldn't be a big problem with the compute node. However I have many existing vms where it fails with a servername not found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 ERROR nova.conductor.tasks.migrate [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Unable to find record for source node servername on servername: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Failed to compute_task_migrate_server: Compute host servername could not be found.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Setting instance to ACTIVE state.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.672 227612 ERROR oslo_messaging.rpc.server [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Exception during message handling: ComputeHostNotFound: Compute host am-osfecn-4025 Tried with this command: nova live-migration --block-migrate id. Any idea? Thank you. ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

5 16

[ptl][tc] OpenStack packages PyPi additional external maintainers audit & cleanup
by Ghanshyam Mann 17 May '24

17 May '24

Hi PTLs, As you might know or have seen for your project package on PyPi, OpenStack deliverables on PyPi have additional maintainers, For example, https://pypi.org/project/murano/, https://pypi.org/project/glance/ We should keep only 'openstackci' as a maintainer in PyPi so that releases of OpenStack deliverables can be managed in a single place. Otherwise, we might face the two sets of maintainers' places and packages might get released in PyPi by additional maintainers without the OpenStack project team knowing about it. One such case is in Horizon repo 'xstatic-font-awesome' where a new maintainer is added by an existing additional maintainer and this package was released without the Horizon team knowing about the changes and release. - https://github.com/openstack/xstatic-font-awesome/pull/2 To avoid the 'xstatic-font-awesome' case for other packages, TC discussed it in their weekly meetings[1] and agreed to audit all the OpenStack packages and then clean up the additional maintainers in PyPi (keep only 'openstackci' as maintainers). To help in this task, TC requests project PTL to perform the audit for their project's repo and add comments in the below etherpad. - https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup Thanks to knikolla to automate the listing of the OpenStack packages with additional maintainers in PyPi which you can find the result in output.txt at the bottom of this link. I have added the project list of who needs to check their repo in etherpad. - https://gist.github.com/knikolla/7303a65a5ddaa2be553fc6e54619a7a1 Please complete the audit for your project before March 15 so that TC can discuss the next step in vPTG. [1] https://meetings.opendev.org/meetings/tc/2023/tc.2023-01-11-16.00.log.html#… -gmann

18 27

[all][packaging][release][qa] pep-517 and pip 23
by smooney＠redhat.com 06 May '24

06 May '24

hi folks, TL;DR ----- this is a long email so the tl;dr is we should add pyproject.toml ``` [build-system] requires = ["pbr>=5.7.0", "setuptools>=36.6.0", "wheel"] build-backend = "pbr.build" ``` to all openstack projects to supprot pip 23.1+ Long version ------------ As some of you may know the ptg happend last week an done of hte topic raised there was a know limitation in devstack where GLOBAL_VENV=True does not work on centos/fedora derived distros. while i generally dont use centos/fedora derived distros for my daily work i have been using devstack for over 10 years at this point so when im told it broken i ocationlly will go fix it. In this case the issue is not with devstack (it often isnt) and is actully in our collective reliance on a deprecated functionality and how we package openstack today. Back story, for better or worse when openstack was created the landscape of python packaging was significantly different then it is today. To help corodinate our dependcies the openstack comunity went all in on requirements.txt and developed upper-constraits to manage our requirements and developed PBR (python build Reasonableness). both of those approches have served us well and likely will continue to do so for some time. Since then pyproject.toml has started to gain popularity and pep-517 (A build-system independent format for source trees) has been adotpted but so far openstack as a comunity has not embraced those changes. (although pbr already supprot pep-517 :) ). Until now this has not been a problem however with recent release of pip things are now breaking. Problem Context: - openstack does not provide pyporject.toml files - openstack project configure setuptools to use pbr via setup.py - setuptools support generating scripts form entry points(console_scripts) which we use for project binaries( nova- compute, neutron-server, ...) - openstack api services use a pbr extension know as wsgi_scripts which adds a similar functionality for generating wsgi application mains file - devstack due to coming changes in distro to block "sudo pip install" now installs openstack in a single by default on debian distros. on centos/fedora distros it currently almost functional (but not). - devstack installs in editable mode because that is how most developers use devstack for developemnt What changed: - pip 20.2 deprecated support for falling back to calling setup.py when pyproject.toml is not present - https://pip.pypa.io/en/stable/news/#v20-2 - https://github.com/pypa/pip/issues/8368 - at this point when pyproject.yaml is not called pip calls setup.py install [-e] <path to repo or package name> - setup.py invokes pbr to generate wsgi_scripts and other files - in pip 23.1 the fallback was removed https://pip.pypa.io/en/stable/news/#v23-1 - pip now builds a temp local wheel instead of calling setup.py install when pyproject.toml is not found - in editable mode (-e) the default way the wheel is built does not invoke pbr properly to generate the wsgi files - as a result the wsgi files are not installed to the venv bin dir and this causes devstack to fail to start keystone (or any other wsgi api) The fix, there are a few ways to approch this: - clamp pip (https://review.opendev.org/c/openstack/devstack/+/898845) this is a terible idea and it breaks debian - clamping pip below 23.1 works on centos for now however OVS on debian already requires pep 517 for installing - as such when that version of ovs (or other projects start enforcing that) hits centos it will break - add a minimal pyproject.toml to all openstack project - example for keystone https://review.opendev.org/c/openstack/keystone/+/899519 - this example jsut adds the minium pyproject.toml to enable pbr to be used as the build system - it does not remove supprot for setup.py but it will not be used by default going forward whit new version of pip - all config is still in setup.cfg - add pyproject.toml support but remove setup.py support https://review.opendev.org/c/opendev/bindep/+/816741 - this goes one step futrue and actuly nulls out setup.py - we could take this approach if we want to but its not required. The reason for this email is to get agreement on how to support pip 23.1+ My proposal would be to add pyproject.toml with the minimal config required to enable pip 23.2 but not drop the setup.py support in this cycle. i would suggest that we can deprecate installing via setup.py in caracal if that is desirable but not make that change until the next slurp to give time to projects and packages to catch up. if there are no objections to that i would like to ask what is the preferd way to create and submit the patches for the minimal pyproject.toml ``` [build-system] requires = ["pbr>=5.7.0", "setuptools>=36.6.0", "wheel"] build-backend = "pbr.build" ``` i can propose patches to the main project deployed by devstack by default. i.e. nova/neutron ectra but it might be better if someone form the core teams of each project could take on that task and chasing the reviews. if someone has a script to generate a patch for a set of repos that just add the above pyporject.toml then we can use that to keep a consistent topic/commit message across all patches. my proposal for testing this is two fold. in ci we can test the patches are functional by using a DNM patch to devstack to force enable the gloal venv on centos and depend on the set of patches, i will modify https://review.opendev.org/c/openstack/devstack/+/898845 to do that locally we can test that the patch is functional by doing the following substituting keystone for the relevant project. ``` git clone https://opendev.org/openstack/keystone cd keystone python3 -m venv .venv . .venv/bin/activate python3 -m pip install -U pip python3 -m pip install -e . ls .venv/bin/keystone* ``` the ls will not contain the wsgi script with the addition of the pyproject.toml it should. on debian sid i can reproduce the same issue the problem we are seeing on centos 9 now will start impacting other distos eventually so its good to fix this now. if i start proposing this to glance/nova/neutron/cinder/swift/horizon/placement (to get a default devstack working) can i get cores to review? can cores form each project nominate peopel to review or volenteer to do this work. i have created https://etherpad.opendev.org/p/pep-517-and-pip-23 to track this work i have some other work that i think we can do in parallel like addign pyporject.toml support to the pti requirements if we go this way but that i think we can discuss on a governacne patch to updated it. anyway this is a lot to digest so feedback is welcome. if someone else would like to drive this that is also welcome (i.e. tc/release team) regards sean.

10 27

Accessing virsh in openstack wallaby | tripleo
by Swogat Pradhan 24 Mar '24

24 Mar '24

Hi, Can someone please tell me how can i access the virsh utility on the compute node on openstack wallaby? Previously we used nova_libvirt but in wallaby there are multiple containers and i am unable to use virsh utility to edit my domain.xml file. List of containers: [root@dcn01-hci-2 ~]# podman ps | grep nova 10d981158842 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtlogd_wrapper e91667516afc 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtsecretd a4b07fe0e833 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtnodedevd d7e1db393e9c 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtstoraged 5308a171793e 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 2 months ago nova_virtqemud f490a6249ba1 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo kolla_start 2 months ago Up 6 weeks ago nova_virtproxyd ab0441e56957 172.25.201.68:8787/tripleomaster/openstack-nova-compute:current-tripleo kolla_start 2 months ago Up 2 months ago (healthy) nova_migration_target f35fbba5b690 172.25.201.68:8787/tripleomaster/openstack-nova-compute:current-tripleo kolla_start 2 months ago Up 6 weeks ago (unhealthy) nova_compute 82bc01993106 172.25.201.68:8787/tripleomaster/openstack-nova-libvirt:current-tripleo /usr/sbin/virtlog... 2 months ago Up 2 months ago Can someone please suggest how I can i edit an instance xml file in this setup? With regards, Swogat Pradhan

2 5

[Freezer] Is Freezer dead?
by pi3.14＠tuta.io 19 Mar '24

19 Mar '24

Hi folks, is Freezer still supported? I don't see any real development going on with it. Is it recommended to use in a production Cloud environment? Are there other free and open source alternatives to Freezer? Thanks Roby

8 19

[Watcher] How use Watcher to scheduler Openstack resource without cpu_usage
by Nguyễn Hữu Khôi 06 Mar '24

06 Mar '24

Hello guys. I want to use watcher to scheduler openstack resources but some openstack versions are missing cpu_usage metric. How do we use Watcher without it? Thank you. Nguyen Huu Khoi

3 5

[openstack][skyline] Some works with skyline
by Nguyễn Hữu Khôi 26 Feb '24

26 Feb '24

Hello guys. I am working on Skyline. I can integrate my monitor tab on it. I will try to do something about it. If it works properly, I can contribute. It is a very nice project. I hope it will grow and grow in the future. Thanks Skyline Team. [image: image.png] Nguyen Huu Khoi

3 7

[neutron][vpnaas][ovn] Pushing VPNaaS support through OVN
by Mohammed Naser 12 Feb '24

12 Feb '24

Hi all! One of the biggest show stoppers for us for OVN is VPNaaS but I think it’s time that we’d like to be involved in getting it over the line, I want to thank Bodo who has shown a lot of patience in getting the code necessary, and it seems that the code works for them in production as well. I’d like to ask if possible for folks to review these two outstanding changes: https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/847007 https://review.opendev.org/c/openstack/neutron-vpnaas/+/765353 I know Slawek had some concerns, so hoping that raising via the email list if Bodo can address those and over the next bit of time we can get this landed in, since it even has actual proper testing both in CI and with user feedback. Thanks! Mohammed

6 8

about magnum capi for production
by Nguyễn Hữu Khôi 07 Feb '24

07 Feb '24

Hello guys. Could you share with me some experience about using Magnum CAPI in production? https://github.com/vexxhost/magnum-cluster-api Thank you so much. Nguyen Huu Khoi

6 44

[all] SQLAlchemy 2.x support in Bobcat
by Thomas Goirand 02 Feb '24

02 Feb '24

Hi, As you may know, and to my great frustration, I'm not the maintainer of SQLAlchemy in Debian, even though OpenStack is the biggest consumer of it. The current maintainer insists that he wants to upload SQLA 2.x in Unstable, potentially breaking all of OpenStack. At the present moment, if I understand correctly, we're not there yet, and Bobcat doesn't have such a support. It would be ok for me, *IF* there are patches available on master, that I could backport to Bobcat and maintain in the debian/patches folder of each project. However, the biggest current annoyance, is that I have no idea where we are at. Are we close to such a support? Is there a list of patches to apply on top of Bobcat that is maintained somewhere? Please enlighten me... :) Cheers, Thomas Goirand (zigo)

5 10

[designate] Proposal to deprecate the agent framework and agent based backends
by Michael Johnson 09 Jan '24

09 Jan '24

TLDR: The Designate team would like to deprecate the backend agent framework and the agent based backends due to lack of development and design issues with the current implementation. The following backends would be deprecated: Bind9 (Agent), Denominator, Microsoft DNS (Agent), Djbdns (Agent), Gdnsd (Agent), and Knot2 (Agent). Designate includes many backend DNS server drivers[1], many of which are "native" (also known as xfr type backends) backend implementations. In addition to the "native" backends, Designate has an agent backend[2] that supports other backends via an agent process. To quote the agent backend documentation[2]: This backend uses an extension[3] of the DNS protocol itself to send management requests to the remote agent processes, where the requests will be actioned. The rpc traffic between designate and the agent is both unauthenticated and unencrypted. Do not run this traffic over unsecured networks. Here are the reasons we are proposing to deprecate the agent framework now: 1. The agent protocol used by Designate is using an "unassigned"[4][5] DNS opcode (14) that is causing problems with the dnspython library >= 2.3.0 which is now validating the opcode when building DNS messages. It is a bad practice to use "unassigned" values as they may be officially assigned at any time, likely with an incompatible message format. 2. The agent backends are not tested in the OpenStack jobs[1]. 3. Many of the agent backends have been marked as "Experimental" since 2016 with no additional contributions beyond general repository code maintenance. 4. The protocol between the Designate worker process and the agent process is unauthenticated and unencrypted. 5. We do not know of a development resource to rewrite the agent framework protocol to address issue #1 and #4 above. 6. The introduction of catalog zones[6] may eliminate the need for some of the agent based backend drivers. By marking the agent framework and agent based backends "deprecated" in the Antelope cycle, we would remove the code no earlier than the "C" release of OpenStack (per the OpenStack deprecation policy[7]). In the meantime, issue #1 has been worked around by overriding the dnspython opcode validation[7] as needed in the Designate code (similar to a monkey patch). This is not a sustainable long term solution. The following backend agent based drivers would be marked "deprecated" in addition to the agent framework itself: Bind9 (Agent) Denominator Microsoft DNS (Agent) Djbdns (Agent) Gdnsd (Agent) Knot2 (Agent) We plan to propose patches for the deprecation over the next week, but will not merge them until at least January 24th to allow time for comment from the community. If you have concerns about this deprecation plan or are interested in rewriting the agent framework protocol to address the above issues, please reply to this announcement. Michael [1] https://docs.openstack.org/designate/latest/admin/support-matrix.html [2] https://docs.openstack.org/designate/latest/admin/backends/agent.html [3] https://github.com/openstack/designate/blob/master/designate/backend/privat… [4] https://www.rfc-editor.org/rfc/rfc6895.html#section-2.2 [5] https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-pa… [6] https://www.ietf.org/archive/id/draft-ietf-dnsop-dns-catalog-zones-08.txt [7] https://docs.openstack.org/project-team-guide/deprecation.html [8] https://review.opendev.org/c/openstack/designate/+/870678

3 13

[TC][Monasca] Proposal to mark Monasca as an inactive project
by Sławek Kapłoński 05 Jan '24

05 Jan '24

Hi, I just proposed patch [1] to mark Monasca as inactive project. It was discussed during last TC meeting on 03.10.2023. Reasons for that are described in the commit message of the [1] but I will also put all of it here: previous cycle there wasn't almost any contributions to that project and most active contributors in last 180 days were Elod Illes and Dr. Jens Harbott who were fixing some Zuul configuration issues only. Here are detailed statistics about Monasca projects: Validating Gerrit... * There are 7 ready for review patches generated within 180 days * There are 2 not reviewed patches generated within 180 days * There are 5 merged patches generated within 180 days * Unreviewed patch rate for patches generated within 180 days is 28.0 % * Merged patch rate for patches generated within 180 days is 71.0 % * Here's top 10 owner for patches generated within 180 days (Name/Account_ID: Percentage): - Dr. Jens Harbott : 42.86% - Joel Capitao : 28.57% - Elod Illes : 28.57% Validate Zuul... Set buildsets fetch size to 500 * Repo: openstack/monasca-log-api gate job builds success rate: 82% * Repo: openstack/monasca-statsd gate job builds success rate: 95% * Repo: openstack/monasca-tempest-plugin gate job builds success rate: 81% * Repo: openstack/monasca-common gate job builds success rate: 84% * Repo: openstack/monasca-kibana-plugin gate job builds success rate: 83% * Repo: openstack/monasca-ceilometer gate job builds success rate: 100% * Repo: openstack/monasca-events-api gate job builds success rate: 83% * Repo: openstack/monasca-ui gate job builds success rate: 88% * Repo: openstack/monasca-specs gate job builds success rate: 100% * Repo: openstack/monasca-grafana-datasource gate job builds success rate: 100% * Repo: openstack/monasca-persister gate job builds success rate: 98% * Repo: openstack/monasca-notification gate job builds success rate: 93% * Repo: openstack/monasca-thresh gate job builds success rate: 100% * Repo: openstack/monasca-api gate job builds success rate: 76% * Repo: openstack/monasca-agent gate job builds success rate: 98% * Repo: openstack/python-monascaclient gate job builds success rate: 100% * Repo: openstack/monasca-transform gate job builds success rate: 100% What's next? According to the "Emerging and inactive projects" document [2] if there will be no volunteers who will step up and want to maintain this project before Milestone-2 of the 2024.1 cycle (week of Jan 08 2024) there will be no new release of Monasca in the 2024.1 cycle and TC will discuss if project should be retired. So if You are interested in keeping Monasca alive and active, please reach out to the TC to discuss that ASAP. Thx in advance. [1] https://review.opendev.org/c/openstack/governance/+/897520 [2] https://governance.openstack.org/tc/reference/emerging-technology-and-inact… -- Slawek Kaplonski Principal Software Engineer Red Hat

8 14

[kolla-ansible][zun] Intention to drop support for Zun for the 2023.1 and path forward
by Jan Gutter 21 Dec '23

21 Dec '23

Hi folks, In yesterday's Kolla IRC meeting we discussed the path forward for Zun [1] and came to the conclusion that it might be better for all involved to drop support for Zun for the 2023.1 release until it can be updated to support the necessary dependencies. More specific details at the end of the message. With this in mind, the plan is to start this process in a week from now, unless a better solution is presented. There is still some time for discussion, please don't hesitate! There are two paths forward after this: If Zun can be updated to support the dependencies during the maintenance timeframe of 2023.1, then support would be restored. If no support is available at the time Caracal releases, however, it will be considered for removal in kolla and kolla-ansible. What this means: Kolla-ansible operators currently running Zun should hold off migration to 2023.1. Any development help would of course be appreciated. Please coordinate with the Zun project to help with development. Details on the dependency problems: * Currently, Zun has an external dependency on Docker 20.x, and uses a feature that has been removed in later versions of Docker. * This Docker feature also relies on an older version of etcd (3.3) * The mitigation [2] is to pin to the old versions of Docker and etcd, but this is not possible with Debian Bookworm hosts. Why this is a problem in Kolla-ansible (and Kolla): * This can't easily be solved by vendoring in an older version of etcd: the migration path it introduces to operators would incur a significant amount of risk. * Host level involvement is also particularly painful, the host version of Docker needs to connect to the etcd service running in the containers. * Even though the containers can be kept buildable and the configuration and inventory settings could be maintained, a working setup is not testable in CI. Thanks very much for everyone's patience, contributions and time! [1]: https://meetings.opendev.org/meetings/kolla/2023/kolla.2023-09-13-13.00.log… [2]: https://bugs.launchpad.net/zun/+bug/2007142

2 2

Cyborg nova reports mdev-capable resource is not available
by Karl Kloppenborg 08 Dec '23

08 Dec '23

Hi Cyborg Team! Karl from Helm Team. When creating a VM with the correct flavor, the mdev gets created by cyborg agent and I can see it in the nodedev-list --cap mdev. However Nova then fails with: nova.virt.libvirt.driver [<removed>- - default default] Searching for available mdevs... _get_existing_mdevs_not_assigned /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driver.py :8357 2023-09-21 14:34:47.808 1901814 INFO nova.virt.libvirt.driver [<removed> - - default default] Available mdevs at: set(). 2023-09-21 14:34:47.809 1901814 DEBUG nova.virt.libvirt.driver [<removed> - - default default] No available mdevs where found. Creating an new one... _allocate_mdevs /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driv er.py:8496 2023-09-21 14:34:47.809 1901814 DEBUG nova.virt.libvirt.driver [<removed> - - default default] Attempting to create new mdev... _create_new_mediated_device /var/lib/openstack/lib/python3.10/site-packages/nova/virt/libvirt/driver.py:8385 2023-09-21 14:34:48.455 1901814 INFO nova.virt.libvirt.driver [<removed> - - default default] Failed to create mdev. No free space found among the following devices: ['pci_0000_4b_03_1', … <truncated list>]. 2023-09-21 14:34:48.456 1901814 ERROR nova.compute.manager [<removed> - - default default] [instance: 2026e2a2-b17a-43ab-adcb-62a907f58b51] Instance failed to spawn: nova.exception.ComputeResourcesUnavailable: Insufficient compute resources: mdev-capable resource is not available. Once this happened, ARQ removes the mdev and cleans up. I’ve got Cyborg 2023.2 running and have a device profile like so: karl@Karls-Air ~ % openstack accelerator device profile show e2b07e11-fe69-4f33-83fc-0f9e38adb7ae +-------------+---------------------------------------------------------------------------+ | Field | Value | +-------------+---------------------------------------------------------------------------+ | created_at | 2023-09-21 13:30:05+00:00 | | updated_at | None | | uuid | e2b07e11-fe69-4f33-83fc-0f9e38adb7ae | | name | VGPU_A40-Q48 | | groups | [{'resources:VGPU': '1', 'trait:CUSTOM_NVIDIA_2235_A40_48Q': 'required'}] | | description | None | +-------------+---------------------------------------------------------------------------+ karl@Karls-Air ~ % I can see the allocation candidate: karl@Karls-Air ~ % openstack allocation candidate list --resource VGPU=1 | grep A40 | 41 | VGPU=1 | 229bf15f-5689-3d2c-b37b-5c8439ea6a71 | VGPU=0/1 | OWNER_CYBORG,CUSTOM_NVIDIA_2235_A40_48Q | karl@Karls-Air ~ % Am I missing something critical here? Because I cannot seem to figure this out… have I got a PCI address wrong, or something? Any help from the Cyborg or Nova teams would be fantastic. Thanks, Karl.

4 9

openstack Vm shutoff by itself
by AJ_ sunny 30 Nov '23

30 Nov '23

Hi team, I am using openstack kolla ansible on wallaby version and currently I am facing issue with virtual machine, vm is shutoff by itself and and from log it seems libvirt lifecycle stop event triggering again and again Logs:- 2023-10-16 08:48:10.971 7 WARNING nova.compute.manager [req-c7b731db-2b61-400e-917f-8645c9984696 f226d81a45dd46488fb2e19515 848 316d215042914de190f5f9e1c8466bf0 default default] [instance: 4b04d3f1-1fbd-4b63-b693-a0ef316ecff3] Received unexpected - vent network-vif-plugged-f191f6c8-dff5-4c1b-94b3-8d91aa6ff5ac for instance with vm_state active and task_state None. 2023-10-21 22:42:44.589 7 INFO nova.compute.manager [-] [instance: 4b04d3f1-1fbd-4b63-b693-a0ef316ecff3] VM Stopped (Lifecyc Event) 2023-10-21 22:42:44.683 7 INFO nova.compute.manager [req-1d99b87b-7ff7-462d-ab18-fbdec6bda71d -] [instance: 4b04d3f1- fbd-4b63-b693-a0ef316ecff3] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from ti e hypervisor (4). Updating power_state in the DB to match the hypervisor. 2023-10-21 22:42:44.811 7 WARNING nova.compute.manager [req-1d99b87b-7ff7-462d-ab18-fbdec6bda71d ----] [instance: 4b04d3f 1-1fbd-4b63-b693-a0ef316ecff3] Instance shutdown by itself. Calling the stop API. Current vm_state: active, current task_state : None, original DB power_state: 1, current VM power_state: 4 2023-10-21 22:42:44.977 7 INFO nova.compute.manager [req-1d99b87b-7ff7-462d-ab18-fbdec6bda71d -] [instance: 4b04d3f1-1 fbd-4b63-b693-a0ef316ecff3] Instance is already powered off in the hypervisor when stop is called. Thanks & Regards Arihant Jain +91 8299719369

5 15

[swift][OpenStackSDK][keystone] Warnings about (my) swift endpoint URL by keystoneauth1 discovery
by Christian Rohmann 13 Nov '23

13 Nov '23

Hey openstack-discuss, we use Ceph RadosGW (Quincy release) to provide Swift as part of an OpenStack cloud. The endpoints are configured as such: > # openstack endpoint list > +----------------------------------+--------+--------------+----------------+---------+-----------+----------------------------------------------------------------------+ > | ID | Region | Service Name | Service > Type | Enabled | Interface | URL | > +----------------------------------+--------+--------------+----------------+---------+-----------+----------------------------------------------------------------------+ > | 1234567890 | region | swift | object-store | True | > public | > https://object-store.region.cloud.example.com/swift/v1/AUTH_%(tenant_id)s > | > | 2345678901 | region | swift | object-store | True | > internal | > https://object-store.region.cloud.example.com/swift/v1/AUTH_%(tenant_id)s > | > | 3456789012 | region | swift | object-store | True | > admin | > https://object-store.region.cloud.example.com/swift/v1/AUTH_%(tenant_id)s > | > so according to https://docs.ceph.com/en/latest/radosgw/keystone/#cross-project-tenant-acce…, if I am not mistaken. I can also use the openstack client to access containers: > # openstack container list > +---------------+ > | Name | > +---------------+ > | containera | > | containerb | > +---------------+ and there is no warnings or other apparent issues. But when using "project cleanup" via e.g. "openstack project cleanup --dry-run --project $PROJECT" I see multiple warnings like > Failed to contact the endpoint at > https://object-store.region.cloud.example.com/swift/v1/AUTH_f2bc4bd34567ddc… > for discovery. Fallback to using that endpoint as the base url. > but only mentioning the swift endpoint. Following the warning string this originates from the discovery method of keystoneauth1, see https://github.com/openstack/keystoneauth/blob/28048af9593740315df8d9027c3b…. Could anybody help me understand what is issue might be? Why does this warning not appear for every use of object storage / swift via openstackclient? Is there any way for me to "fix" this? Could this even be a bug? Thanks and with kind regards Christian

4 10

[ptls] [tc] 2023 OpenStack Deployment Data
by Allison Price 09 Nov '23

09 Nov '23

Hi everyone, Apologies for sending this during the PTG as I intended to send this prior, but below is a link to the anonymized data including responses to many project or TC contributed questions. I have removed a significant number of fields to preserve the anonymity of the organizations, but if there is a particular data point that would help your team, please let me know. Cheers, Allison https://docs.google.com/spreadsheets/d/1ehlabuqOnNK4M7xtN1GXvhLoeHn4JBaISih…

2 1

[nova] Nov 7 is a spec review day and Nov 14 is implementation review day
by Sylvain Bauza 07 Nov '23

07 Nov '23

Hey folks, I feel a bit sad to had no time yet to provide you a summary for our PTG, but please wait a bit more :-) That said, something you should already know, we will have two spec review days this cycle and two implementation review days as well, so please mark your calendars for the first to come : - Nov 7 will be a spec review day. Please ensure your specs are ready for reviews - Nov 14 will be an implementation review day. Mark your patches up ! As a reminder, the idea behind those review days is to synchronize our efforts between owners and reviewers so people can discuss at the same time. Thanks, -Sylvain

1 1

[charms] Nominating Utkarsh Bhatt for charms-ceph-core
by Peter Sabaini 03 Nov '23

03 Nov '23

Hello all, I would like to nominate Utkarsh Bhatt for charms-ceph-core. Utkarsh has worked for the Ceph charms team for 1.5 years now and has provided many contributions to the charms, especially to the Ceph charms[0]. I know him as a diligent and enthusiastic contributor and hope you will support his nomination. Thanks, peter. [0] https://review.opendev.org/q/owner:utkarsh.bhatt%2540canonical.com

4 3

[ops] Seeking Advice on Service Deployment and Layout for an 8-Node Cluster
by Forrest Fuqua 02 Nov '23

02 Nov '23

Hi there, I'm currently exploring some strategies and seeking advice on the deployment and layout of services on an 8-node cluster. Each node is equipped with dual AMD EYPC 7702s, two 100G Links, and 2TB of RAM. Additionally, I've set up two 13TB NVME drives in a Ceph configuration for each node. At present, the setup involves one management node, while the others are dedicated to running Nova, Neutron, Glance, and Cinder. The workload, while fairly basic, involves handling substantial numbers. It's primarily used for one-off research projects and a few classes, serving around 60 end users. The most demanding scenario occurs during cybersecurity competitions when we need to deploy 3100 VMs at once. When this is spread out over a few days, it's manageable. However, there's a risk of overloading the system, leading to DiskIO issues and Database inconsistencies. Currently, I'm utilizing Kolla for deployment, and I've shared my configurations on this GitHub repository: Link to Configs<https://github.com/RIT-GCI-CyberRange/openstack-configs>. I'd greatly appreciate any insights or advice on optimizing this setup to handle the peak workload more efficiently and prevent potential performance issues. Thanks in advance for your help!

3 7

[ironic] Announcement: Future of Metalsmith
by Jay Faulkner 01 Nov '23

01 Nov '23

Hi all, Just announcing that at the vPTG, the Ironic core team came to a consensus about the future of the metalsmith project. It was originally created to improve the user experience around provisioning Ironic nodes when using Ironic's API directly. At this, it was mostly successful and has completed this task. It's our intention to work on the Ironic API and python-ironicclient to provide a similarly simple deployment user experience, even for users without nova, integrating lessons learned from the metalsmith project. To this end; we are letting the overall community know we no longer expect to add additional functionality to metalsmith. We will ensure it remains maintained and well-tested until it's functionality has been duplicated, but all new work in this area will be directed at the primary Ironic API and Ironic client projects. To this end, and as discussed at the PTG, I've proposed a change to the Metalsmith readme, notifying users of this state. https://review.opendev.org/c/openstack/metalsmith/+/899761 This change, or this mailing list post, is a great place to comment if you have any comments on this decision. Thanks, Jay Faulkner Ironic PTL

3 3

Openstack volume deletion
by Vijay Thilak S 01 Nov '23

01 Nov '23

Hi, OpenStack Kolla Ansible has been installed on Ubuntu 20 and integrated with Ceph storage. The volume associated with the instance is not being deleted by Cinder when I terminate the instance with this option (Delete Volume on Instance Delete), which is causing me problems. I need to log in to the server and run the cinder commands to manually delete the volume that is not associated with instances. I would appreciate it if you could provide a solution. Thanks, Vijay

4 10

OpenInfra Live: Meet the Computing Force Network Working Group Nov 2 at 9 AM (CST) / 14:00 UTC
by Kristin Barrientos 31 Oct '23

31 Oct '23

Hi everyone, This week’s OpenInfra Live episode is brought to you by the open source community. Episode: China Mobile and other 10+ organizations are driving the effort of Computing Force Network in open source. In July of 2022, organizations like, China Mobile launched the Computing Force Network (CFN) Working Group within the OpenInfra Foundation. You may be wondering, what is Computing Force Network? What open source software is CFN working group going to deliver and how are CFN and related technology going to help the infrastructure? Join us to find out. Date and time: Nov 2, 2023 at 9:00 AM CST (14:00 UTC) You can watch us live on: YouTube: https://www.youtube.com/watch?v=Slc-kOao3BY LinkedIn: https://www.linkedin.com/events/meetthecomputingforcenetworkwor711866369563… WeChat: The recording will be posted on OpenInfra Foundation WeChat after the live stream Speakers: Qihui Zhao, Jie Niu, Xiaoqiao Geng, Changjin Wang, Jintao Wang, Lei Huang and Ildiko Vancsa Thanks, Kristin Barrientos Marketing Coordinator OpenInfra Foundation

1 0

[ironic] IntelIPMIManagement driver: Anyone using it?
by Jay Faulkner 31 Oct '23

31 Oct '23

Hi all, Ironic currently has an Intel IPMI Management driver ( https://docs.openstack.org/ironic/latest/admin/drivers/intel-ipmi.html ), a mild enhancement over our generic IPMI driver, which adds the ability to select an Intel Speed Select Performance Profile via an Ironic automation step. Ironic cores have no knowledge of this driver being used by any deployers of Ironic. We are evaluating it for deprecation and eventual removal. If you are interested in this driver remaining in the Ironic codebase, please let us know. Please note: this is not the same as our generic IPMI driver, which supports power control, boot device management, and is not being considered anytime soon for deprecation :). If we decide to mark the driver for removal, it'll obviously remain in the codebase for a long deprecation period as is our norm. Thanks, Jay Faulkner Ironic PTL

1 0

[largescale-sig] Next meeting: Nov 1, 15utc
by Kristin Barrientos 31 Oct '23

31 Oct '23

Hi everyone, The Large Scale SIG will be meeting this Wednesday in #openstack-operators on OFTC IRC, at 15UTC, our EU+US-friendly time. You can doublecheck how that UTC time translates locally at: https://www.timeanddate.com/worldclock/fixedtime.html?iso=20231004T15 Feel free to add topics to the agenda: https://etherpad.opendev.org/p/large-scale-sig-meeting Thanks, Kristin Barrientos Marketing Coordinator OpenInfra Foundation

1 0

[blazar] Cancelling the next IRC meeting
by Pierre Riteau 31 Oct '23

31 Oct '23

Hello, I am cancelling the next Blazar IRC meeting due to being out of the office. The next meeting will be on November 16. Best wishes, Pierre Riteau (priteau)

1 0

[horizon] Cancelling Tomorrow Weekly meeting (1st Nov 2023)
by vishal manchanda 31 Oct '23

31 Oct '23

Hello Team, I won't be able to chair tomorrow's Horizon weekly meeting due to some urgent work at that time. If anything urgent, please reach out to me and the Horizon core team on IRC. Thanks & regards, Vishal Manchanda

1 0

[cinder] 2024.1 (Caracal) PTG Summary
by Rajat Dhasmana 31 Oct '23

31 Oct '23

Hello, Please find the summary of 2024.1 (Caracal) PTG in the wiki page[1]. It includes summary and action items for each topic. More details can be found in the PTG etherpad[2]. I would recommend everyone to please look for the action items assigned to you to plan it according to the cycle deadlines. [1] https://wiki.openstack.org/wiki/CinderCaracalPTGSummary [2] https://etherpad.opendev.org/p/caracal-ptg-cinder Thanks Rajat Dhasmana

1 0

Green threads
by Gk Gk 31 Oct '23

31 Oct '23

Hi All, This may be out of place for this question, but I was reading about this bug https://bugs.launchpad.net/oslo.messaging/+bug/2009138, which led to this query. Do green threads in python need GIL ( global interpreter lock) for execution ? Also, can green threads take advantage of multiple processors ? If yes/no, why ?

1 0

[qa] Office hour day and time change
by Martin Kopec 31 Oct '23

31 Oct '23

Hi everyone, as discussed during the PTG [1] we are changing the date and time of our office hours. The office hour will be taking place on *Wednesdays at 17:00 UTC* [2]. The change is effective *from November 8th*. The meeting agenda is available here [3]. I'm cancelling office hours on Tuesday October 31st and November 7th. [1] https://etherpad.opendev.org/p/oct2023-ptg-qa [2] https://review.opendev.org/c/opendev/irc-meetings/+/899457 [3] https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Thanks, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 1

capacity planning question.
by KK CHN 30 Oct '23

30 Oct '23

Hi list, I am in need of an infrastructure set up for data analytics / live video stream analytics application using big data and analytics technology.. The data is basically right now stored as structured data(no video streaming) in PostgresDatabase. ( Its an emergency call handling solution, In database which stores, caller info (address, mobile number, locations co-ordinates, emergency category metadata and dispatch information regarding rescue vehicles., Rescue vehicle location update (lat, long)every 30 seconds all are stored in the Postgres Database .. Input1 : I have to do an analytics on these data( say 600 GB for the last 2 years its the size grown from initial setup).To perform an analytical application development( using python and data analytics libraries, and displaying the results and analytical predication through a dashboard application.) Query 1. How much resource in terms of compute(GPU?(CPU) cores required for this analytical application? and memory ? And any specific type of storage(in memory like redis required ? ) etc, which I have to provision for these kind of application processing. ?? any hints most welcome.. Any more input required let me know I can provide if available. Input 2 In addition to the above I have to do video analytics from bodyworn cameras by police personnel, drone surveillance Videos from any emergency sites, patrol vehicle (from a mobile tablet device over 5G )live streaming of indent locations for few minutes ( say 3 to 5 minutes live streaming for each incident. ) There are 50 drones, 500 Emergency rescue service vehicles, 300 body worn camera personnels.. and roughly 5000 incidents / emergency incidents per day happening, which needs video streaming for at least 1000 incidents for a time duration of 4 to 5 minutes live streaming. Query2. What/(how many) kind of computing resources GPUs(CPUs)? RAM, Storage solutions I have to deploy in numbers( or cores of GPUs/how many/(CPUs)? RAM ? In Memory (Redis or similar ) or any other specific data storage mechanisms ? Any hints much appreciated.. Thank in advance, Krishane

1 0

[tacker][ptg] Caracal vPTG Summary
by Yasufumi Ogawa 30 Oct '23

30 Oct '23

Hi all, We had two days session for discussing new proposals for Caracal and reamined items couldn't fixed in the previous release [1]. Here is a summary of the discussion in Tacker team. # Revise FTs Gate tests, especially for functional tests, have been getting mess because of increasing amount of codes without active maintenance. For the issue, we have focused on reducing useless logs. And also re-design directory structure and testcase scenarios. We're going to continue the tasks and will fix it in this release. # Legacy API deprecating/obsoleting It was proposed to drop Tacker's legacy implementation in antelope, and will be completed in this release as planed. # Introducing Cilium CNI for high-performance network connectivity for VNF/CNF This proposal is for a new example app using Cilium CNI as a vRAN usecase in which VNF/CNF requires performance for network connectivity between VMs or containers. The first target is to confirm it's available to deploy with Tacker in a manner of VNF package. # Update requirements for monitoring We have introduced Prometheus for implementing FM/PM interfaces defined in ETSI-NFV standards from Yoga. This proposal is for more advanced feature than monitoring, called as observability, by using OpenTelemetry for considering future requirements. We agreed to implement an additional plugin dedicating to interwork between Otel collector and Tacker. # NFV Orchestration API v2.0 in Tacker Horizon Although many of v2 APIs in ETSI specifications have been implemented, tacker-horizon is not ready for the features. We'll update NFVO interfaces in Horizon to provide v2 features. Current legacy features are going to be dropped instead. # Improving Tacker Document As legacy implementations have been deprecated, documents are also removed. In addition, there are some other sections should be updated such as incorrect usecase guides don'twork, unmaintained operator's guides or so. # Tacker Tempest Plugin Tacker doesn't support tempest, but it's not good for considering if it leads to OpenStack vendors not including Tacker in their products, and also makes Tacker look unhealthy project. So, implement Tacker's tempest plugin and update our zuul jobs to use the plugin for. # Fix vimConnectionID in ResouceHandle In the current implementation, vimConnectionID differs from ETSI specs mistakenly. So, it's should be fixed soon and expected to be small changes. # Secure RBAC This item has been led by NEC team and development of UTs and FTs are under implemention as noted on the vPTG etherpad. However, we need to update ownership of the item for manpreetk moved to another organization. Confirm if NTT or NEC team own the item in caracal and share the result in IRC meeting. # Fix issues which were encounter in compliance test execution In next release of tacker (2024.1), NEC team proposing to make 'tacker-compliance-devstack-multinode-sol' job testing mandatory and voting after some issues occured in the test are fixed. # Maintain tosca-parser and heat-translator Assign team leaders for each projs temporarily for governance, Toshiaki Takahashi for tosca-parser and Ayumu Ueha heat-translator. Ask TC memebers anything requried to change the ownership. [1] https://etherpad.opendev.org/p/oct2023-ptg-tacker Thanks, Yasufumi

1 0

[tc] Weekly Meeting Tomorrow, Oct 31 1800 UTC in OFTC
by Jay Faulkner 30 Oct '23

30 Oct '23

Hi all, The OpenStackTechnical Committee will have our weekly meeting tomorrow, October 31, at 1800 UTC in #openstack-tc on OFTC. All are welcome to attend. The proposed agenda is below. Agenda Suggestions - Roll call - Follow up on tracked action items: https://meetings.opendev.org/meetings/tc/2023/tc.2023-10-17-18.00.html - JayF: Before next video meeting, write up a short document on pros/cons of moving TC video meetings to jitsi-meet. - Gate health check - Leaderless projects (gmann) - PTG follow-ups? - https://etherpad.opendev.org/p/tc-ptg-october-2023 - https://etherpad.opendev.org/p/tc-leaders-interaction-2024-1 - https://etherpad.opendev.org/p/oct2023-ptg-os-dbperf-crossproject - Open Discussion and Reviews Thank you, Jay Faulkner OpenStack TC Chair

1 0

[barbican] Weekly Meeting 31-10-2023 Canceled
by Grzegorz Grasza 30 Oct '23

30 Oct '23

Hi all, I'm canceling the meeting this Tuesday. I'm on a business trip this week. Thanks! / Greg

1 0

Re: Regarding Nova's inability to delete the Cinder volume for creating virtual machines (version Y)
by 无数的星球 30 Oct '23

30 Oct '23

Thank you for Gorka's reply! I have seen this document multiple times and it has not been taken seriously. Next, I will try.

1 0

Regarding Nova's inability to delete the Cinder volume for creating virtual machines (version Y)
by 无数的星球 30 Oct '23

30 Oct '23

When creating a virtual machine in the dashboard, create a volume and choose to delete the virtual machine while also deleting the volume. When deleting the virtual machine, there is no normal uninstallation of the volume and the volume is not deleted. The relevant error logs are shown in the image, but the openstack CLI can delete its volume. The specific commands are as follows. CLI： source /etc/keystone/admin-openrc.sh (Verify password file) openstack volume set --detached　191e555c-3947-4928-be46-9f09e2190877（volumeID） openstack volume delete 191e555c-3947-4928-be46-9f09e2190877（volumeID） It seems that Nova is unable to interact with the Cinder API to delete(or detached) commands, but I am not very professional. I don't know if it's a bug? Note that I have seen similar bug patch files, but they are invalid. URL:   BUG page   https://bugs.launchpad.net/nova/+bug/2016173   Page for patching files   https://review.opendev.org/c/openstack/nova/+/880399 Is not working!!!

2 1

[openstack-ansible][ptg] OpenStack-Ansible 2024.1 (Caracal) vPTG Summary
by Dmitriy Rabotyagov 30 Oct '23

30 Oct '23

Hey everyone, We had quite a productive discussion on the PTG and I will provide a summary of them below. It was divided into multiple sections, like things we need to finish for 2023.2 (Bobcat) - since OpenStack-Ansible has trailing release model we have some time to wrap things up; plans for 2024.1 (Caracal) and some things that moved out of scope until better times. You can also find original PTG with discussions here [1] Finalizing 2023.1 (Bobcat): * Documentation on using pretty endpoint names (ie identity.cloud.org instead of cloud.org:5000). For 2024.1 we will try to simplify the way of using such naming to make it more convenient for end-users. * Finalize support of RabbitMQ Quorum queues for all services [2]. With that they will be disabled by default at least for 2023.2 release and we will revise the defaults for 2024.1 * Implement openstack_resources common rolem that will handle different resources creation, like OpenStack users, projects, networks, images, aggregates, etc. It will be also used in roles like Tempest, Octavia, Trove and Magnum to create such resources, instead of each role implementing its own flow of doing that. We agreed to add a separate playbook that will allow deployers to leverage the role and run it after setup-openstack. * Debian 12 support. Most patches have landed, but we also need to reflect it's support in documentation * Review and release Skyline role. Tough discussion was about defaults and how Skyline should co-exist with Horizon. As a result we decided to deploy both options, when both are defined in the inventory (as usual), but the playbook that is run last will basically configure HAProxy backends to itself, fully neglecting other available services. So if you want to have both Skyline and Horizon running and functional - you will need to apply some overrides for that. There are also no guarantees that we will be able to merge and release the role in time, so we may end up with making it as part of OSA only for 2024.1. Scoped improvements for 2024.1 (Caracal): * Adoption of PKI role to be used with PIPE modules, that will allow the role to be pointed to any file (like encrypted with ansible-vault) or external storage that supports lookup, instead of plain text files on a deploy host. With that we will also take a look at using Hishcorp Vault as a PKI source and adding it as another driver in addition to the "standalone" one. * Natively supporting multi-AZ deployments for Neutron. At the moment for OVS/LXB you need to define some overrides to make that working, while for OVN you would need to manually execute a bunch of commands. POC patch has been already proposed [3], but it still needs some love. * Finalize support of OVN driver for Octavia. Initial patch has been proposed, but it also needs some love, despite it works in general [4] * Resume work on ProxySQL implementation as a replacement for HAProxy balancing for MariaDB deployment. That should improve load distribution between MariaDB backends. * Allow to select OS for LXC containers. At the moment we're forcing to use the host OS, while in some cases it might be preferred to use a different one, like during OS upgrades or switching the OpenStack deployment from one distro to another. * With that we are excited about incus (LXD fork) having the first release and it being packaged with traditional packages (only .deb at the moment). This opens the door for implementation of incus as a manager for LXC containers, that may simplify our flows and day2 operations significantly. * Invest some time into decreasing the runtime, both for CI/CD and for end-user deployments. * In case improvements for Quorum Queues will land to the oslo.messaging [5] - add a set of variables to roles for nicely controlling the behaviour. [1] https://etherpad.opendev.org/p/osa-caracal-ptg [2] https://review.opendev.org/q/topic:osa%252Fquorum_queues [3] https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/858271 [4] https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/868462 [5] https://review.opendev.org/q/topic:bug-2031497

1 0

[Glance] Caracal PTG Summary
by Pranali Deore 30 Oct '23

30 Oct '23

Hello Everyone, We had our eighth virtual PTG between 23rd OCt to 26th OCT 2023. Thanks to everyone who joined the virtual PTG sessions. Using Opendev Meetpad we had all the discussion around different topics for glance. I have created etherpad[1] with notes from the session and also included the recordings of each session. Here is the short summary of the discussions. *Tuesday , OCT 24th 2023* # Bobcat Retrospective On a positive note we have completed few important things like adding support for weighting mechanism to store, support for RBD driver to move images to the trash if they cannot be deleted immediately due to having snapshots, putting efforts in bridging the gap between glanceclient and openstackclient and started new location API work which will replace the image-update mechanism for consumers like cinder and nova for OSSN-0090 and OSSN-0065. On the other side we decided to speedup the work we've had in flight and we decided to have recent patches reviews meeting once in a month and backlogs review parties before each milestone to speedup the reviews. Recording: https://drive.google.com/drive/u/0/folders/1oYrgVLV5L89keDZLx7r1JWcG8pk65918 - Bobcat Retrospective # Common Tempest Failures for Glance Glance has faced so many random CI failure and timeouts since bobcat mid cycle, though progress has been made on general issues and few of the project-specific issue also been addressed like post failures causing by image conversion failure, tried reducing concurrency but still we see some random timeouts, one of the common tempest failure has been observed that some glance related jobs like glance-multistore-cinder-import, nova-ceph-multistore are failing with race condition while saving image data, there is a short term solution proposed by sending image_id while creating the image to ensure that each test can utilize own image and race condition could be avoided. https://docs.google.com/document/d/1S41X3FSjqGF87ehTvqLEC9QsDIkULjS1NiqCagZ… Decided to reproduce 1st failure from the gdoc and debug more on failures Recording: https://drive.google.com/drive/u/0/folders/1oYrgVLV5L89keDZLx7r1JWcG8pk65918 - Common tempest failures # Add missing CLI support for some Glance API in Openstack Client The migration work for adding missing CLI was initiated in the antelope cycle. We have added the SDK support for cache API , images API and metadef API(in progress). On the client side support for cache API and images API has been added and for metadef is in progress. This cycle we have decided to cover the remaining client and sdk support for APIs by distributing the work among the team. Recordings: https://drive.google.com/drive/u/0/folders/1oYrgVLV5L89keDZLx7r1JWcG8pk65918 - Adding missing OSC support # Default Glance to configure multiple stores Glance has deprecated single stores configuration since Stein cycle and will remove the support of same in upcoming cycle by dividing the wrok in small parts like remove/migrate single store unit/functional tests. Goal for this cycle: list down the work we have to do with regards to tests. and migrate unit/functional test from "single-store only" to "multistore-only" and send email to the mailing list to urge users to migrate to multi-store. Recording: https://drive.google.com/drive/u/0/folders/1oYrgVLV5L89keDZLx7r1JWcG8pk65918 - Default glane to multistore support.webm *Wednesday, OCT 25th 2023* # Remove Location Strategy In Bobcat we added support for the weighing mechanism to store. So we can safely remove the location strategy and by default use the latest weighing mechanism which will be very easy and efficient. Decided to deprecate location strategy in C and remove it in D, notify operators and users on ML and have a spec for the same in Caracal Cycle. Recording: https://drive.google.com/drive/u/0/folders/1oYrgVLV5L89keDZLx7r1JWcG8pk65918 - Remove Location Strategy # Improve process of restoring image to active state which is set as 'pending_delete' At the moment glance supports delaying of image-data from the store by setting delayed_delete to True and when image gets deleted it actually keeps image data in the backend and then Scrubber daemon deletes it permanently later. To restore the image, the deployer needs to set the UUID of that image in glance-api.conf to the 'restore' option under the Default section. If there are multiple images that need to be restored then this process will be kind of time consuming as each time a new Id will be needed to set in the configuration file and rerun the scrubber if it is not running as Daemon. Decided to introduce admin only API to restore the image and restrict Scrubber daemon to delete the images which are in pending_delete state. The restore part from the scrubber for a couple of cycles until the new API is stable.The restore option from the scrubber will be deprecated in this cycle and then will be removed in the following cycle. Recording: https://drive.google.com/drive/u/0/folders/1oYrgVLV5L89keDZLx7r1JWcG8pk65918 - Restore Image # Deprecate cachemanage middleware Cache operations (CR(u)D) is now part of glance API. Even though we need to depend on cache/cachemanage middleware to enable it. There are two middlewares related to caching, one is `cache` and other is `cachemanage` and we can now get rid of `cachemanage` middleware and glance-cache-manage command line utility. Agreed on deprecating cachmanage middleware in C & remove in D cycle. Decided to have a spec lite for the same after verifying how the cache APIs will respond if cache middleware is not enabled. Recording: https://drive.google.com/drive/u/0/folders/1oYrgVLV5L89keDZLx7r1JWcG8pk65918 - Deprecate Cachemanage Middleware # Use mysql database for caching instead of sqlite There are two different database(s) used for glance, MySql for regular operations and sqlite for caching related operations. We have decided to use single database i.e. MySql for all glance related operations so that it will reduce the efforts to maintain both databases. The Spec is up for review now: https://review.opendev.org/c/openstack/glance-specs/+/899367/4 Recording: https://drive.google.com/drive/u/0/folders/1oYrgVLV5L89keDZLx7r1JWcG8pk65918 - Centralize DB for Caching *Thursday, OCT 26th 2023* # Unified limits Topic was discussed and decided to postpone it to the next cycle since no one volunteered to work on it. # New Location APIs We have decided to have support of new location api which will replace the image-update mechanism for consumers like cinder and nova for OSSN-0090 and OSSN-0065. The implementation patches are up for review along with the unit/functional tests, documentation, api version bump, client side support and tempest coverage[2] The Poc patches of nova and cinder are also there to test this new location api functionality. In this cycle we are going to work on some missing test scenarios & verify PoC patches of consumers like nova and cinder are working with new location api functionality. Recording: https://drive.google.com/drive/u/0/folders/1oYrgVLV5L89keDZLx7r1JWcG8pk65918 - New Location API Apart from above topics you can find other miscellaneous topics discussed in PTG etherpad [1] along with the recordings of the sessions and milestone wise priorities at the end. If you have any questions/suggestions then please join us in our weekly meeting(each Thursday on #openstack-meeting irc channel at 1400 UTC) [1]: https://etherpad.opendev.org/p/oct2023-ptg-glance#L10 [2]: https://review.opendev.org/q/topic:New-Location-Apis Thanks & Regards, Pranali Deore

1 0

[qa][ptg] QA Caracal vPTG Summary
by Martin Kopec 30 Oct '23

30 Oct '23

Hi everyone, here is a summary of the discussions. The full agenda can be found here [1]. == Bobcat Retrospective == Considering the still shrinking number of active contributors, we still manage making releases and doing code reviews. However, less contributors make it hard to react to sudden failures or provide as stable jobs as possible - that's e.g. the case of the Centos 9 topic. == Stop supporting Centos 9 Stream? == No, Centos 9 stays. As concluded during the session, Centos 9 is required for FIPS as other distros (their latest versions) are far behind with supporting FIPS. Apart from that, CS9 is required for Manila, Cinder integration with Ceph as Ceph is tested on Centos 9 too. The biggest pain point is that Centos9 Stream is not versioned and thus it's hard to build a reliable gate. One of the goals for the cycle is improving the connection between Centos and OpenStack community - it's beneficial for both sides. One of the ways might be a 3rd party CI or simply a solution that would provide a quick feedback to the Centos group, e.g. by running Devstack with the default configuration, but at the same time wouldn't affect all the jobs on the OpenStack side that run the distro. This way we could at least avoid simple issues found right away (e.g. some deps issues). The QA team is happy to assist. == Triage previous and write new priority items == The list of the priority items planned for this cycle as well as the ones which were carried over from the previous cycles, can be found here [2]. [1] https://etherpad.opendev.org/p/oct2023-ptg-qa [2] https://etherpad.opendev.org/p/qa-caracal-priority Thank you all for attending the PTG and contributing to the discussion, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 0

[interop][ptg] Interop Caracal vPTG Summary
by Martin Kopec 30 Oct '23

30 Oct '23

Hi everyone, here is a summary of the discussions. The full agenda can be found here [1]. We had only one topic on the agenda - a new view on interoperability testing - don't confuse it with interop which is a certification program. The program just uses the interop tests for the certification purposes. The tests that are part of interop were long ago considered as enough for testing interoperability. However, as it's been pointed out as well as discussed many times already [2][3], the tests don't guarantee interoperability of tested clouds - they test only basic API thus they can be used for the certification program (related to OpenStack brand) at most. Interoperability is subjective, every user may be looking for a different level of interoperability. Read the ending of the article [2]. If there is someone who is interested in interoperability testing in OpenStack or even someone who requires that for various reasons, we could do something like this [4] - we could gather various tests from various testing platforms that test specific functionalities. It would provide anyone with metrics and thus allow them to compare clouds. It would be like when we're shopping, e.g. for a new computer - we have various metrics that help us decide which one suits us better like dimensions, weight, RAM, CPU, storage space etc. From the last PTG (and several one before) it, however, doesn't look like the community is interested in running actual interoperability testing. Let us know if you disagree, we don't want to develop something that is not needed by anyone. [1] https://etherpad.opendev.org/p/oct2023-ptg-interop [2] https://superuser.openinfra.dev/articles/new-view-on-interoperability-in-op… [3] https://etherpad.opendev.org/p/bobcat-ptg-interop [4] https://etherpad.opendev.org/p/openstack-interop-2.0 Thank you all for attending the PTG and contributing to the discussion, -- Martin Kopec Principal Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 0

[neutron] Bugs deputy report October 23rd - 29th
by Miguel Lavalle 29 Oct '23

29 Oct '23

Hi Neutrinos, Here's this week's bugs deputy report: Critical ====== High ==== https://bugs.launchpad.net/neutron/+bug/2040166 [OVN] The neutron_sync_mode conf is ignored for SB DB sync. Assignee: Lucas Gomes. https://bugs.launchpad.net/neutron/+bug/2040172 [OVN] OvnSbSynchronizer - clean/delete segmenthostmappings for unrelated hosts. Assignee: Harald Jensas. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/899077 https://bugs.launchpad.net/neutron/+bug/2041609 FIP update removes QoS policy. Assignee Lajos Katona. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/899469 Medium ====== https://bugs.launchpad.net/neutron/+bug/2040517 host not removed from table ml2_vxlan_endpoints with the agent delete. Needs and owner https://bugs.launchpad.net/neutron/+bug/2041612 [n-d-r] DRAgent removed from bgp speaker automatically. Needs verification and owner Low === RFEs ==== https://bugs.launchpad.net/neutron/+bug/2040001 [N-D-R] Change neighbor connect mode when the BGP peer does not support passive mode. Assignee: Tiago Pires. This RFE was initially discussed during this week's vPTG. Proposed spec: https://review.opendev.org/c/openstack/neutron-specs/+/899210 Undecided ======== Invalid ======== https://bugs.launchpad.net/neutron/+bug/2040242 [ipam] Why prevent first ip < (subnet.first + 1) if version of subnet is ipv6. Assignee Liu Xie. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/899374 https://bugs.launchpad.net/neutron/+bug/2040457 [allocation_pools] Need re-generation new pools from cidr and gateway-ip after update one subnet with empty 'allocation_pools'. Assignee: Liu Xie. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/899373

1 0

Regarding Nova's inability to delete the Cinder volume for creating virtual machines (version Y)
by 无数的星球 28 Oct '23

28 Oct '23

When creating a virtual machine in the dashboard, create a volume and choose to delete the virtual machine while also deleting the volume. When deleting the virtual machine, there is no normal uninstallation of the volume and the volume is not deleted. The relevant error logs are shown in the image, but the openstack CLI can delete its volume. The specific commands are as follows. CLI： source /etc/keystone/admin-openrc.sh (Verify password file) openstack volume set --detached　191e555c-3947-4928-be46-9f09e2190877（volumeID） openstack volume delete 191e555c-3947-4928-be46-9f09e2190877（volumeID） It seems that Nova is unable to interact with the Cinder API to delete(or detached) commands, but I am not very professional. I don't know if it's a bug?

1 0

horizon - dashboard behavior - shows elements from previous project
by Miguel Casique Meneses 27 Oct '23

27 Oct '23

There is a horizon chart behaving quite strangely version: horizon-0.3.2 To recreate the issue: login to openstack >> change project (using dropdown) >> show containers (odd behaviour, such as change to old project, or lists containers from previous project) >> show instances (odd behaviour, changes projects, or at least shows the wrong project in dropdown) Basically, it sometimes shows the elements from the previous browsed project and not the current one that has been selected. If we reload the page, it goes back to project that has been selected, but sometimes it does not The logs are not showing nothing relevant, but we can see when project has been swapped

3 2

[Glance] Caracal PTG Updates
by Pranali Deore 27 Oct '23

27 Oct '23

Hello Everyone, We have concluded Glance Caracal vPTG yesterday. I will share the summary early next week. You can find tentative milestone wise priorities for glance in PTG etherpad[1]. Join us in the weekly meeting on Thursday if you have any doubts/suggestions. Thanks & Regards, Pranali [1]: https://etherpad.opendev.org/p/oct2023-ptg-glance#L277

1 0