March 2023 - openstack-discuss - lists.openstack.org

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

How to config DPDK compute in OVN
by dangerzone ar 22 Apr '25

22 Apr '25

Hi....as far I understand ovs does support dpdk compute to be deployed. I'm referring to the guideline ' *https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html <https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html>* ' ... It's too brief.. May I know how to config dpdk compute in OVN networking openstack? Hope someone could help what is the config to be done.... Currently compute is run without dpdk over openstack train ovn based. Please need some info on how I can proceed. Thank you

2 1

ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version
by Swogat Pradhan 24 Dec '24

24 Dec '24

Hi, I am currently trying to deploy openstack wallaby using tripleo arch. I created the network jinja templates, ran the following commands also: #openstack overcloud network provision --stack overcloud --output networks-deployed-environment.yaml custom_network_data.yaml # openstack overcloud network vip provision --stack overcloud --output vip-deployed-environment.yaml custom_vip_data.yaml # openstack overcloud node provision --stack overcloud --overcloud-ssh-key /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml and used the environment files in the openstack overcloud deploy command: (undercloud) [stack@hkg2director ~]$ cat deploy.sh #!/bin/bash THT=/usr/share/openstack-tripleo-heat-templates/ CNF=/home/stack/ openstack overcloud deploy --templates $THT \ -r $CNF/templates/roles_data.yaml \ -n $CNF/workplace/custom_network_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e $CNF/templates/node-info.yaml \ -e $CNF/templates/scheduler-hints.yaml \ -e $CNF/workplace/networks-deployed-environment.yaml \ -e $CNF/workplace/vip-deployed-environment.yaml \ -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ -e $CNF/workplace/custom-net-bond-with-vlans.yaml Now when i run the ./deploy.sh script i encounter an error stating: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command.: tripleoclient.exceptions.InvalidConfiguration: Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command. 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END return value: 1 Can someone tell me where the mistake is? With regards, Swogat Pradhan

4 4

Kolla Ansible on Ubuntu 20.04 - cloud-init & other network issues
by Tobias McNulty 10 Oct '24

10 Oct '24

Hi, I'm attempting to use Kolla Ansible 14.6.0 to deploy OpenStack Yoga on a small 3-node Ubuntu 20.04 cluster. The nodes have 128 GB RAM each, dual Xeon processors, and dual 10G Intel NICs. The NICs are connected to access ports on a 10G switch with separate VLANs for the local and external networks. All the playbooks run cleanly, but cloud-init is failing in the Ubuntu 20.04 and 22.04 VMs I attempt to boot. The VM images are unmodified from https://cloud-images.ubuntu.com/, and cloud-init works fine if I mount a second volume with user-data. The error is a timeout attempting to reach 169.254.169.254. This occurs both when booting a VM in an internal routed network and directly in an external network. I tried various neutron plugin agents (ovn, linuxbridge, and openvswitch both with and without firewall_driver = openvswitch <https://docs.openstack.org/kolla-ansible/latest/reference/networking/neutro…>) first with a clean install of the entire OS each time, all with the same result. Running tcpdump looking for 169.254.169.254 shows nothing. As a possible clue, the virtual NICs are unable to pass any traffic (e.g., to reach an external DHCP server) unless I completely disable port security on the interface (even if the associated security group is wide open). But disabling port security does not fix cloud-init (not to mention I don't really want to disable port security). Are there any additional requirements related to deploying OpenStack with Kolla on Ubuntu 20.04? This is a fairly vanilla configuration using the multinode inventory as a starting point. I tried to follow the Quick Start <https://docs.openstack.org/kolla-ansible/yoga/user/quickstart.html> as closely as possible; the only material difference I see is that I'm using the same 3 nodes for control + compute. I am using MAAS so it's easy to get a clean OS install on all three nodes ahead of each attempt. I plan to try again with the standard (non-HWE) kernel just in case, but otherwise I am running out of ideas. In case of any additional clues, here are my globals.yml and inventory file, along with the playbook I'm using to configure the network, images, VMs, etc., after bootstrapping the cluster: https://gist.github.com/tobiasmcnulty/7dbbdbc67abc08cbb013bf5983852ed6 Thank you in advance for any advice! Cheers, Tobias

5 9

Live migration fails
by Szabo, Istvan (Agoda) 07 Aug '24

07 Aug '24

Hi, I have couple of compute nodes where the live migration fails with existing vms. When I quickly spawn a vm and try live migration it works so I assume shouldn't be a big problem with the compute node. However I have many existing vms where it fails with a servername not found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 ERROR nova.conductor.tasks.migrate [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Unable to find record for source node servername on servername: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Failed to compute_task_migrate_server: Compute host servername could not be found.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Setting instance to ACTIVE state.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.672 227612 ERROR oslo_messaging.rpc.server [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Exception during message handling: ComputeHostNotFound: Compute host am-osfecn-4025 Tried with this command: nova live-migration --block-migrate id. Any idea? Thank you. ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

5 16

[ptl][tc] OpenStack packages PyPi additional external maintainers audit & cleanup
by Ghanshyam Mann 16 May '24

16 May '24

Hi PTLs, As you might know or have seen for your project package on PyPi, OpenStack deliverables on PyPi have additional maintainers, For example, https://pypi.org/project/murano/, https://pypi.org/project/glance/ We should keep only 'openstackci' as a maintainer in PyPi so that releases of OpenStack deliverables can be managed in a single place. Otherwise, we might face the two sets of maintainers' places and packages might get released in PyPi by additional maintainers without the OpenStack project team knowing about it. One such case is in Horizon repo 'xstatic-font-awesome' where a new maintainer is added by an existing additional maintainer and this package was released without the Horizon team knowing about the changes and release. - https://github.com/openstack/xstatic-font-awesome/pull/2 To avoid the 'xstatic-font-awesome' case for other packages, TC discussed it in their weekly meetings[1] and agreed to audit all the OpenStack packages and then clean up the additional maintainers in PyPi (keep only 'openstackci' as maintainers). To help in this task, TC requests project PTL to perform the audit for their project's repo and add comments in the below etherpad. - https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup Thanks to knikolla to automate the listing of the OpenStack packages with additional maintainers in PyPi which you can find the result in output.txt at the bottom of this link. I have added the project list of who needs to check their repo in etherpad. - https://gist.github.com/knikolla/7303a65a5ddaa2be553fc6e54619a7a1 Please complete the audit for your project before March 15 so that TC can discuss the next step in vPTG. [1] https://meetings.opendev.org/meetings/tc/2023/tc.2023-01-11-16.00.log.html#… -gmann

18 27

[Watcher] How use Watcher to scheduler Openstack resource without cpu_usage
by Nguyễn Hữu Khôi 06 Mar '24

06 Mar '24

Hello guys. I want to use watcher to scheduler openstack resources but some openstack versions are missing cpu_usage metric. How do we use Watcher without it? Thank you. Nguyen Huu Khoi

3 5

[designate] Proposal to deprecate the agent framework and agent based backends
by Michael Johnson 09 Jan '24

09 Jan '24

TLDR: The Designate team would like to deprecate the backend agent framework and the agent based backends due to lack of development and design issues with the current implementation. The following backends would be deprecated: Bind9 (Agent), Denominator, Microsoft DNS (Agent), Djbdns (Agent), Gdnsd (Agent), and Knot2 (Agent). Designate includes many backend DNS server drivers[1], many of which are "native" (also known as xfr type backends) backend implementations. In addition to the "native" backends, Designate has an agent backend[2] that supports other backends via an agent process. To quote the agent backend documentation[2]: This backend uses an extension[3] of the DNS protocol itself to send management requests to the remote agent processes, where the requests will be actioned. The rpc traffic between designate and the agent is both unauthenticated and unencrypted. Do not run this traffic over unsecured networks. Here are the reasons we are proposing to deprecate the agent framework now: 1. The agent protocol used by Designate is using an "unassigned"[4][5] DNS opcode (14) that is causing problems with the dnspython library >= 2.3.0 which is now validating the opcode when building DNS messages. It is a bad practice to use "unassigned" values as they may be officially assigned at any time, likely with an incompatible message format. 2. The agent backends are not tested in the OpenStack jobs[1]. 3. Many of the agent backends have been marked as "Experimental" since 2016 with no additional contributions beyond general repository code maintenance. 4. The protocol between the Designate worker process and the agent process is unauthenticated and unencrypted. 5. We do not know of a development resource to rewrite the agent framework protocol to address issue #1 and #4 above. 6. The introduction of catalog zones[6] may eliminate the need for some of the agent based backend drivers. By marking the agent framework and agent based backends "deprecated" in the Antelope cycle, we would remove the code no earlier than the "C" release of OpenStack (per the OpenStack deprecation policy[7]). In the meantime, issue #1 has been worked around by overriding the dnspython opcode validation[7] as needed in the Designate code (similar to a monkey patch). This is not a sustainable long term solution. The following backend agent based drivers would be marked "deprecated" in addition to the agent framework itself: Bind9 (Agent) Denominator Microsoft DNS (Agent) Djbdns (Agent) Gdnsd (Agent) Knot2 (Agent) We plan to propose patches for the deprecation over the next week, but will not merge them until at least January 24th to allow time for comment from the community. If you have concerns about this deprecation plan or are interested in rewriting the agent framework protocol to address the above issues, please reply to this announcement. Michael [1] https://docs.openstack.org/designate/latest/admin/support-matrix.html [2] https://docs.openstack.org/designate/latest/admin/backends/agent.html [3] https://github.com/openstack/designate/blob/master/designate/backend/privat… [4] https://www.rfc-editor.org/rfc/rfc6895.html#section-2.2 [5] https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-pa… [6] https://www.ietf.org/archive/id/draft-ietf-dnsop-dns-catalog-zones-08.txt [7] https://docs.openstack.org/project-team-guide/deprecation.html [8] https://review.opendev.org/c/openstack/designate/+/870678

3 13

[openstack-ansible] Recommended branch for a production environment?
by Roger Rivera 05 Sep '23

05 Sep '23

Hello, We have deployed Openstack-Ansible in a test environment and we were wondering what the recommended repository branch is to deploy in a production environment that will be integrated with a separate Ceph cluster. 1. We are thinking about pinning to the stable/zed. Is that recommended over the master branch? 2. Are bugs ironed out on both master and stable/zed with the same cadence? 3. Additionally, is Debian 11 a better alternative than Ubuntu 22 for target hosts? We noticed Ubuntu 22.04 support was added recently, whereas Debian 11 has been supported for quite some time now which leads us to believe its stability/integration could be more mature at this point? Any suggestions will be appreciated. Thank you, -- *Roger *

3 7

[puppet] Proposing core team clean up
by Takashi Kajinami 23 Aug '23

23 Aug '23

Hello, I went through the current core reviewers and found out that the current list includes some people no longer working in the OpenStack community. I'd like to propose removing the following persons from the core reviewer team as we haven't seen these people making any activity in the community recently. * Pradeep Kilambi - puppet-aodh, puppet-ceilometer, puppet-gnocchi, puppet-panko * Richard Megginson - puppet-keystone * Denis Egorenko - puppet-murano * Serg Melikyan - puppet-murano * Sergey Kolekonov - puppet-neutron * David Moreau Simard - puppet-openstack-integration * Yu xingchao - all modules Also, I confirmed that Wes Hayutin is no longer working for OpenStack and he agreed to retire from the core reviewer position. Thus I'll remove him as well. I'll be open for any feedback for one week to hear any thoughts/feedback about this removal. I'd like to take this moment to thank all people for their past work. We always welcome them back if they resume their work in this project. Thank you, Takashi KAJINAMI

3 4

[cloudkitty] question to the scope_key in cloudkitty
by Noah Elias Feldt 28 Jul '23

28 Jul '23

Hello! I am currently trying to get CloudKitty to work with the Prometheus Collector and PyScripts. I noticed that the scope_key for Prometheus can only be changed once in the global configuration file. This was a problem for me because we need different scope_keys for different metrics. Many of the metrics still need the scope key "tenant_id" and some others "project_id". I have found in the documentation of CloudKitty no way to change this for each metric, so I have internally changed something in the source code and added the function to set the scope_key for each metric individually. I just wanted to ask if you would be interested if I submit the small change upstream? Thank you. Regards Noah Feldt

3 3

[cinder] cinder-backup volume stuck in creating
by Satish Patel 27 Jul '23

27 Jul '23

Folks, I have configured nova and cinder with ceph storage. VMs running on ceph storage but now when i am trying to create a backup of cinder volume its getting stuck on creating and doing nothing. Logs also do not give any indication of bad. My cinder.conf [DEFAULT] enabled_backends = rbd-1 backup_driver = cinder.backup.drivers.ceph.CephBackupDriver backup_ceph_conf = /etc/ceph/ceph.conf backup_ceph_user = cinder-backup backup_ceph_chunk_size = 134217728 backup_ceph_pool = backups backup_ceph_stripe_unit = 0 backup_ceph_stripe_count = 0 restore_discard_excess_bytes = true osapi_volume_listen = 10.73.0.181 osapi_volume_listen_port = 8776 Output of "openstack volume service list" showing cinder-backup service is up but when i create a backup it's getting stuck in this stage and no activity. I am not seeing anything getting transferred to the ceph backups pool also. Any clue? or method to debug? # openstack volume backup list --all +--------------------------------------+------+-------------+----------+------+ | ID | Name | Description | Status | Size | +--------------------------------------+------+-------------+----------+------+ | bc844d55-8c5a-4bd3-b0e9-7c4c780c95ad | foo1 | | creating | 20 | +--------------------------------------+------+-------------+----------+------+

5 7

Re: [all] [oslo.messaging] Interest in collaboration on a NATS driver
by Tobias Urdin 22 Jul '23

22 Jul '23

Hello Kai, Great to hear that you’re interested! I’m currently just testing this as a POC directly in devstack so there is real-world testing (not should it yet I think, needs more development first). Best regards Tobias > On 30 Aug 2022, at 18:32, Kai Bojens <kb(a)kbojens.de> wrote: > > Am 29.08.22 um 15:46 schrieb Tobias Urdin: > >> If anybody, or any company, out there would be interested in collaborating in a project to bring this support and maintain it feel free to >> reach out. I’m hoping somebody will bite but atleast I’ve put it out there for all of you. > > Hi, > I am very much interested to take a closer look at your work and maybe contribute to it. Although I'm working with OpenStack for my employer at the moment, I'd do this in my spare time as I'm not sure that I can convince him to add another staging system with a full OpenStack installation just for developing a RabbitMQ replacement. That's not on our agenda as we are mostly users and not developers of OpenStack. > > As I'm pretty new to the messaging topic and had just heard of NATS and your idea, I'll first would have to dive into NATS and then I would take a closer look at your code and maybe try to create some documentation or tests. So, I can't promise anything but as I said I'm very interested in your approach as I also see the massive load from RabbitMQ. > > This brings me to the most important question: Where do you run and test your code? > > Greetings, > Kai

8 30

[ptl] Need PTL volunteer for OpenStack Vitrage
by Ghanshyam Mann 11 Jul '23

11 Jul '23

Hi Eyal, I am reaching out to you as you were PTL for OpenStack Vitrage project in the last cycle. There is no PTL candidate for the next cycle (2023.2), and it is on the leaderless project list. Please check if you or anyone you know would like to lead this project. - https://etherpad.opendev.org/p/2023.2-leaderless Also, if anyone else would like to help leading this project, this is time to let TC knows. -gmann

3 3

Experience with VGPUs
by Oliver Weinmann 22 Jun '23

22 Jun '23

Dear All, we are planning to have a POC on VGPUs in our Openstack cluster. Therefore I have a few questions and generally wanted to ask how well VGPUs are supported in Openstack. The docs, in particular: https://docs.openstack.org/nova/zed/admin/virtual-gpu.html explain quite well the general implementation. But I am more interested in general experience with using VGPUs in Openstack. We currently have a small YOGA cluster, planning to upgrade to Zed soon, with a couple of compute nodes. Currently our users use consumer cards like RTX 3050/3060 on their laptops and the idea would be to provide VGPUs to these users. For this I would like to make a very small POC where we first equip one compute node with an Nvidia GPU. Gladly also a few tips on which card would be a good starting point are highly appreciated. I know this heavily depends on the server hardware but this is something I can figure out later. Also do we need additional software licenses to run this? I saw this very nice presentation from CERN on VGPUs: https://indico.cern.ch/event/776411/contributions/3345183/attachments/18516… In the table they are listing Quadro vDWS licenses. I assume we need these in order to use the cards? Also do we need something like Cyborg for this or is VGPU fully implemented in Nova? Best Regards, Oliver

17 36

[ironic] ARM Support in CI: Call for vendors / contributors / interested parties
by Jay Faulkner 14 Jun '23

14 Jun '23

Hi stackers, Ironic has published an experimental Ironic Python Agent image for ARM64 ( https://tarballs.opendev.org/openstack/ironic-python-agent-builder/dib/file…) and discussed promoting this image to supported via CI testing. However, we have a problem: there are no Ironic developers with easy access to ARM hardware at the moment, and no Ironic developers with free time to commit to improving our support of ARM hardware. So we're putting out a call for help: - If you're a hardware vendor and want your ARM hardware supported? Please come talk to the Ironic community about setting up third-party-CI. - Are you an operator or contributor from a company invested in ARM bare metal? Please come join the Ironic community to help us build this support. Thanks, Jay Faulkner Ironic PTL

6 11

[storlets] Proposal to make Train/Ussuri/Victoria EOL
by Takashi Kajinami 13 Jun '23

13 Jun '23

Hello, Currently we have multiple stable branches open but we haven't seen any backport proposed so far. To reduce number of branches we have to maintain, I'd like to propose retiring old stable branches(train, ussuri and victoria). In case you have any concerns, please let me know. Thank you, Takashi Kajinami

1 1

Kubernetes Conformance 1.24 + 1.25
by Kendall Nelson 05 Jun '23

05 Jun '23

Hello All! First of all, I want to say a huge thanks to Guilherme Steinmuller for all his help ensuring that OpenStack Magnum remains Kubernetes Certified [1]! We are certified for v1.24! However, in digging into the set of conformance tests this round, it appears that we will not be able to get 1.25 certified with the current Magnum. This means that in May when we have to re-cert (every 3 months they reach an EOL because that's k8s release cadence) we will not be able to maintain this which is the other part of why I wanted to write this email - to raise awareness and ask for help digging into what needs to happen to make sure we can retain this. Tangentially related, Guilherme has posted a patch with documentation for how to run the conformance tests for yourself and it could use some reviews [2]. And one last thing, I have begun conversation with the opendev folks about what it might take to setup jobs to run the conformance tests periodically so that we don't have to manually run them when it's time to recert. I am planning on submitting a forum topic for this for those interested :) Long live LOKI! - Kendall Nelson [1] https://github.com/cncf/k8s-conformance/pull/2446 [2] https://review.opendev.org/c/openstack/magnum/+/849156

5 10

[ironic][stable] Proposing EOL of ironic project branches older than Wallaby
by Jay Faulkner 01 Jun '23

01 Jun '23

Hi all, Ironic has a large amount of stable branches still in EM. We need to take action to ensure those branches are either retired or have CI repaired to the point of being usable. Specifically, I'm looking at these branches across all Ironic projects: - stable/queens - stable/rocky - stable/stein - stable/train - stable/ussuri - stable/victoria In lieu of any volunteers to maintain the CI, my recommendation for all the branches listed above is that they be marked EOL. If someone wants to volunteer to maintain CI for those branches, they can propose one of the below paths be taken instead: 1 - Someone volunteers to maintain these branches, and also report the status of CI of these older branches periodically on the Ironic whiteboard and in Ironic meetings. If you feel strongly that one of these branches needs to continue to be in service; volunteering in this way is how to save them. 2 - We seriously reduce CI. Basically removing all tempest tests to ensure that CI remains reliable and able to merge emergency or security fixes when needed. In some cases; this still requires CI fixes as some older inspector branches are failing *installing packages* in unit tests. I would still like, in this case, that someone volunteers to ensure the minimalist CI remains happy. My intention is to let this message serve as notice and a waiting period; and if I've not heard any response here or in Monday's Ironic meeting (in 6 days), I will begin taking action on retiring these branches. This is simply a start; other branches (including bugfix branches) are also in bad shape in CI, but getting these retired will significantly reduce the surface area of projects and branches to evaluate. I know it's painful to drop support for these branches; but we've provided good EM support for these branches for a long time and by pruning them away, we'll be able to save time to dedicate to other items. Thanks, Jay Faulkner

3 5

[neutron] Deprecate networking-odl project
by Rodolfo Alonso Hernandez 24 May '23

24 May '23

Hello all: During the last releases, the support for "networking-odl" project has decreased. Currently there is no active developer or maintainer in the community. This project depends on https://www.opendaylight.org/; the latest version released is Sulfur (16) while the version still used in the CI is Sodium (11) [1]. I would like first to make a call for developers to update this project. But if this is not possible, I will then start the procedure to deprecate it [2] (**not to retire it**). Regards. [1] https://github.com/openstack/networking-odl/blob/db5c79b3ee5054feb8a17df130… [2] https://docs.openstack.org/project-team-guide/repository.html#deprecating-a…

3 3

[heat][magnum][tacker] Future of SoftwareDeployment support
by Takashi Kajinami 10 May '23

10 May '23

Hello, We discussed this briefly in the past thread where we discussed maintenance of os-*-agent repos, and also talked about this topic during Heat PTG, but I'd like to formalize the discussion to get a clear agreement. Heat has been supporting SoftwareDeployment resources to configure software in instances using some agents such as os-collect-config[1]. [1] https://docs.openstack.org/heat/latest/template_guide/software_deployment.h… This feature was initially developed to be used by TripleO (IIUC), but TripleO is retired now and we are losing the first motivation to maintain the feature. # Even TripleO replaced most of its usage of softwaredeployment by config-download lately. Because the heat project team has drunk dramatically recently, we'd like to put more focus on core features. For that aim we are now wondering if we can deprecate and remove this feature, and would like to hear from anyone who has any concerns about this. Quickly looking through the repos, it seems currently Magnum and Tacker are using SoftwareDeployment, and it'd be nice especially if we can understand their current requirements. 1. Magnum It seems SoftwareDeployment is used by k8s_fedora_atomic_v1 driver but I'm not too sure whether this driver is still supported, because Fedora Atomic was EOLed a while ago, right ? 2. Tacker SoftwareDeployment can be found in only test code in the tacker repo. We have some references kept in heat-translator which look related to TOSCA templates. Thank you, Takashi Kajinami

4 8

[horizon][nova][masakari] Instances created with "Any AZ" problem
by Nguyễn Hữu Khôi 02 May '23

02 May '23

Hello guys. I playing with Nova AZ and Masakari https://docs.openstack.org/nova/latest/admin/availability-zones.html Masakari will move server by nova scheduler. Openstack Docs describe that: If the server was not created in a specific zone then it is free to be moved to other zones, i.e. the AvailabilityZoneFilter <https://docs.openstack.org/nova/latest/admin/scheduling.html#availabilityzo…> is a no-op. I see that everyone usually creates instances with "Any Availability Zone" on Horzion and also we don't specify AZ when creating instances by cli. By this way, when we use Masakari or we miragrated instances( or evacuate) so our instance will be moved to other zones. Can we attach AZ to server create requests API based on Any Availability Zone to limit instances moved to other zones? Thank you. Regards Nguyen Huu Khoi

5 20

[magnum] Propose deprecation of Docker Swarm, CoreOS, fedora-ironic
by Jake Yip 28 Apr '23

28 Apr '23

Hi, Due to limited resources, Magnum team has decided to put most effort into supporting the most popular use case - Kubernetes on Fedora CoreOS. In light of that, we are seeking maintainers for the less well maintained drivers / COE - Docker Swarm, CoreOS and fedora-ironic drivers. The reasons are as follows: - Docker Swarm - It uses Fedora Atomic which has been EOL [1] - CoreOS - It is EOL [2] - fedora-ironic - Not tested, unsure if it is still working If anyone wants to take up maintenance of any of the above, we will be glad. Please let us know either through the ML or at the Magnum meetings. If there are no suitable maintainers, we will start the deprecation of these before end of Antelope. Regards, Jake [1]: https://projectatomic.io/blog/2019/11/fedora-atomic-host-nearing-eol/ [2]: https://www.redhat.com/en/technologies/cloud-computing/openshift/what-was-c… -- Jake Yip Magnum Team

2 3

[Horizon]: allow user to access calendar on horizon
by James Leong 18 Apr '23

18 Apr '23

Hi all, I am using kolla-ansible for OpenStack deployment in the yoga version. Is it possible to allow the user (member role) to view the calendar graph under the reservation tab (lease)? Currently, only the admin will be able to view the calendar graph with all the reserved leases. However, a user with other roles cannot load the calendar information. On the dashboard, I saw it displayed " Unable to load reservations." When I look into the log file, I get the below error message. "blazarclient.exception.BlazarClientException: ERROR: Policy doesn't allow blazar:oshosts:get to be performed." Is there a way to allow the policy? Thanks for your help. James

2 2

Re: [all][stable][ptl] Propose to EOL Rocky series
by Elõd Illés 17 Apr '23

17 Apr '23

Hi, thanks for all the feedbacks from teams so far! @Zigo: Extended Maintenance process was created just for the same situation: to give space to interested parties to cooperate and keep things maintained even when stable releases are over their 'supported' lifetime. So it's good to see that there is interest in it! Unfortunately, with very old branches we've reached the state where gates can't be maintained and without a functional gate it's not safe to merge patches (yes, even security fixes) and they are just using resources (CI & maintainers' time). When gate is broken in such extent, then i think the community have to accept that it is not possible to merge patches confidently and needs to EOL that release. (Yes, it means that some vendors still have to do it for their own sake, testing it by their own, etc). Keeping things open in such state gives the false signal to vendors that 'it is maintained' however this is done without fully testing the changes. Another aspect is that code cannot be cleaned up until those old branches are still present (CI jobs, project configurations, etc) which gives pain for developers. So, however some vendors would appreciate probably to keep things open forever, for the community this is not beneficial and doable I think. Seeing that teams want to EOL their old branches is a proof that this is needed (unfortunately). Thanks, Előd Illés irc: elodilles @ #openstack-stable / #openstack-release ________________________________ From: Thomas Goirand <thomas(a)goirand.fr> Sent: Saturday, January 28, 2023 2:25 PM Cc: openstack-discuss(a)lists.openstack.org <openstack-discuss(a)lists.openstack.org> Subject: Re: [all][stable][ptl] Propose to EOL Rocky series Hi, I understand the gate is broken, however, Rocky is in Debian LTS, and I would like to keep the possibility to merge patches, even with the gate tests disabled. CVE-2022-47951 is an example why this is important... Thomas Goirand (zigo) On Jan 28, 2023 11:54, Radosław Piliszek <radoslaw.piliszek(a)gmail.com> wrote: > > Masakari is happy to EOL Rocky too. > > Radek > -yoctozepto > > On Fri, 27 Jan 2023 at 20:59, Jay Faulkner <jay(a)gr-oss.io> wrote: > > > > Thanks for doing all this cleanup work Elod. Ironic is OK with retirements of these shared resources up to Train. > > > > - > > Jay Faulkner > > > > On Fri, Jan 27, 2023 at 10:12 AM Elõd Illés <elod.illes(a)est.tech> wrote: > >> > >> Hi, > >> > >> Similarly like the Queens branch EOL proposal [1] I would like to propose > >> to transition every project's stable/rocky to End of Life: > >> > >> - gates are mostly broken > >> - minimal number of activity can be seen on this branch > >> - some core projects already transitioned their stable/rocky to EOL > >> recently (like ironic, neutron, nova) > >> - gate job definitions are still using the old, legacy zuul syntax > >> - gate jobs are based on Ubuntu Xenial, which is also beyond its public > >> maintenance window date and hard to maintain > >> > >> Based on the above, if there won't be any project who wants to keep open > >> their stable/rocky, then I'll start the process of EOL'ing Rocky stable > >> series as a whole. If anyone has any objection then please respond to > >> this mail. > >> > >> Thanks, > >> > >> Előd Illés > >> irc: elodilles @ #openstack-stable / #openstack-release > >> > >> [1] https://lists.openstack.org/pipermail/openstack-discuss/2022-October/031030… > >> >

3 4

[ptl] Need PTL volunteer for OpenStack Winstackers
by Ghanshyam Mann 13 Apr '23

13 Apr '23

Hi Lukas, I am reaching out to you as you were PTL for OpenStack Winstackers project in the last cycle. There is no PTL candidate for the next cycle (2023.2), and it is on the leaderless project list. Please check if you or anyone you know would like to lead this project. - https://etherpad.opendev.org/p/2023.2-leaderless Also, if anyone else would like to help leading this project, this is time to let TC knows. -gmann

7 14

[nova][cinder] future of rebuild without reimaging
by Bence Romsics 06 Apr '23

06 Apr '23

Hi All! We have users who use 'rebuild' on volume booted servers before nova microversion 2.93, relying on the behavior that it keeps the volume as is. And they would like to keep doing this even after the openstack distro moves to a(n at least) zed base (sometime in the future). As a naive user, it seems to me both behaviors make sense. I can easily imagine use cases for rebuild with and without reimaging. However since the implementation of https://specs.openstack.org/openstack/nova-specs/specs/zed/implemented/volu… rebuild without reimaging is only possible using an old microversion (<2.93). With that change merged, rebuild without reimaging seems to be a somewhat less than fully supported feature. A few examples of what I mean by that: First, there's this warning: https://opendev.org/openstack/python-openstackclient/src/commit/5eb89e4ca1c… In which it is unclear to me what exactly will become an error in a future release. Rebuild with a different image? Or any rebuild with microversion <2.93? Then old nova microversions may get dropped. Though what I heard from nova folks, this is unlikely to happen. Then there are a few hypothetical situations like: a) Rebuild gets a new api feature (in a new microversion) which can never be combined with the do-not-reimage behavior. b) Rebuild may have a bug, whose fix requires a microversion bump. This again can never be combined with the old behavior. What do you think, are these concerns purely theoretical or real? If we would like to keep having rebuild without reimaging, can we rely on the old microversion indefinitely? Alternatively shall we propose and implement a nova spec to explicitly expose the choice in the rebuild api (just to express the idea: osc server rebuild --reimage|--no-reimage)? If the topic is worth further discussion beyond the ML, I can also bring it to the nova ptg. Thanks in advance, Bence Romsics (rubasov) ps: I'll be afk for a few days, but I'll follow up next Tuesday.

7 16

[nova][cinder] Providing ephemeral storage to instances - Cinder or Nova
by Christian Rohmann 04 Apr '23

04 Apr '23

Hello OpenStack-discuss, I am currently looking into how one can provide fast ephemeral storage (backed by local NVME drives) to instances. There seem to be two approaches and I would love to double-check my thoughts and assumptions. 1) *Via Nova* instance storage and the configurable "ephemeral" volume for a flavor a) We currently use Ceph RBD als image_type (https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.im…) so instance images are stored in Ceph, not locally on disk. I believe this setting will also cause ephemeral volumes (destination_local) to be placed on a RBD and not /var/lib/nova/instances? Or is there a setting to set a different backend for local block devices providing "ephemeral" storage? So RBD for the root disk and a local LVM VG for ephemeral? b) Will an ephemeral volume also be migrated when the instance is shutoff as with live-migration? Or will there be an new volume created on the target host? I am asking because I want to avoid syncing 500G or 1T when it's only "ephemeral" and the instance will not expect any data on it on the next boot. c) Is the size of the ephemeral storage for flavors a fixed size or just the upper bound for users? So if I limit this to 1T, will such a flavor always provision a block device with his size? I suppose using LVM this will be thin provisioned anyways? 2) *Via Cinder*, running cinder-volume on each compute node to provide a volume type "ephemeral", using e.g. the LVM driver a) While not really "ephemeral" and bound to the instance lifecycle, this would allow users to provision ephemeral volume just as they need them. I suppose I could use backend specific quotas (https://docs.openstack.org/cinder/latest/cli/cli-cinder-quotas.html#view-bl…) to limit the number of size of such volumes? b) Do I need to use the instance locality filter (https://docs.openstack.org/cinder/latest/contributor/api/cinder.scheduler.f…) then? c) Since a volume will always be bound to a certain host, I suppose this will cause side-effects to instance scheduling? With the volume remaining after an instance has been destroyed (beating the purpose of it being "ephemeral") I suppose any other instance attaching this volume will be scheduling on this very machine? Is there any way around this? Maybe a driver setting to have such volumes "self-destroy" if they are not attached anymore? d) Same question as with Nova: What happens when an instance is live-migrated? Maybe others also have this use case and you can share your solution(s)? Thanks and with regards Christian

3 4

Opensearch service upgrade
by Daniel Pawlik 04 Apr '23

04 Apr '23

Hello, We would like to notify you that the Opensearch service [1] would be updated to a newer version on 03.04.2023 at 12:00 PM UTC. This procedure might take a while, depending of the cluster size. During that time, the Opensearch service would not be available. If anyone has any doubts, please reply to the email. Dan [1] - https://opensearch.logs.openstack.org/_dashboards/app/home

1 1

[nova][heat] The next steps to "fix" libvirt problems in Ubuntu Jammy
by Takashi Kajinami 04 Apr '23

04 Apr '23

Hello, Since we migrated our jobs from Ubuntu Focal to Ubuntu Jammy, heat gate jobs have become very flaky. Further investigation revealed that the issue is related to something in libvirt from Ubuntu Jammy and that prevents detaching devices from instances[1]. The same problem appears in different jobs[2] and we workaround the problem by disabling some affected jobs. In heat we also disabled some flaky tests but because of this we no longer run basic scenario tests which deploys instance/volume/network in a single stack, which means we lost the quite basic test coverage. My question is, is there anyone in the Nova team working on "fixing" this problem ? We might be able to implement some workaround (like checking status of the instances before attempting to delete it) but this should be fixed in libvirt side IMO, as this looks like a "regression" in Ubuntu Jammy. Probably we should report a bug against the libvirt package in Ubuntu but I'd like to hear some thoughts from the nova team because they are more directly affected by this problem. I'm now trying to set up a centos stream 9 job in Heat repo to see whether this can be reproduced if we use centos stream 9. I've been running that specific scenario test in centos stream 9 jobs in puppet repos but I've never seen this issue, so I suspect the issue is really specific to libvirt in Jammy. [1] https://bugs.launchpad.net/nova/+bug/1998274 [2] https://bugs.launchpad.net/nova/+bug/1998148 Thank you, Takashi

3 5

[ptls] PyPI maintainer cleanup - Action needed: Contact extra maintainers
by Jay Faulkner 03 Apr '23

03 Apr '23

Hi PTLs, The TC recently voted[1] to require humans be removed from PyPI access for OpenStack-managed projects. This helps ensure all releases are created via releases team tooling and makes it less likely for a user account compromise to impact OpenStack packages. Many projects have already updated https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup#L33 with a list of packages that contain extra maintainers. We'd like to request that PTLs, or their designate, reach out to any extra maintainers listed for projects you are responsible for and request they remove their access in accordance with policy. An example email, and detailed steps to follow have been provided at https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup-email-tem… . Thank you for your cooperation as we work to improve our security posture and harden against supply chain attacks. Thank you, Jay Faulkner TC Vice-Chair 1: https://opendev.org/openstack/governance/commit/979e339f899ef62d2a6871a99c9…

3 6

[barbican] database is growing and can not be purged
by Pavlo Shchelokovskyy 03 Apr '23

03 Apr '23

Hi all, we are observing the following behavior in Barbican: - OpenStack environment is using both encrypted Cinder volumes and encrypted local storage (lvm) for Nova instances - over the time, the secrets and orders tables are growing - many soft-deleted entries in secrets DB can not be purged by the db cleanup script As I understand what is happening - both Cinder and Nova create secrets in Barbican on behalf of the user when creating an encrypted volume or booting an instance with encrypted local storage. They both do it via castellan library, that under the hood creates orders in Barbican, waits for them to become active and returns to the caller only the ID of the generated secret. When time comes to delete the thing (volume or instance) Cinder/Nova again use castellan, but only delete the secret, not the order (they are not aware that there was any 'order' created anyway). As a result, the orders are left in place, and DB cleanup procedure does not delete soft-deleted secrets when there's an ACTIVE order referencing such secret. This is troublesomes on many levels - users who use Cinder or Nova may not even be aware that they are creating something in Barbican. Orders accumulating like that may eventually result in cryptic errors when e.g. when you run out of quota for orders. And what's more, default Barbican policies do allow 'normal' (creator) users to create an order, but not delete it (only project admin can do it), so even if the users are aware of Barbican involvement, they can not delete those orders manually anyway. Plus there's no good way in API to determine outright which orders are referencing deleted secrets. I see several ways of dealing with that and would like to ask for your opinion on what would be the best one: 1. Amend Barbican API to allow filtering orders by the secrets, when castellan deletes a secret - search for corresponding order and delete it as well, change default policy to actually allow order deletion by the same users who can create them. 2. Cascade-delete orders when deleting secrets - this is easy but probably violates that very policy that disallowed normal users to delete orders. 3. improve the database cleanup so it first marks any order that references a deleted secret also as deleted, so later when time comes both could be purged (or something like that). This also has a similar downside to the previous option by not being explicit enough. I've filed a bug for that https://storyboard.openstack.org/#!/story/2010625 and proposed a patch for option 2 (cascade delete), but would like to ask what would you see as the most appropriate way or may be there's something else that I've missed. Btw, the problem is probably even more pronounced with keypairs - when castellan is used to create those, under the hood both order and container are created besides the actual secrets, and again only the secret ids are returned to the caller. When time comes to delete things, the caller only knows about secret IDs, and can only delete them, leaving both container and order behind. Luckily, I did not find any place across OpenStack that actually creates keypairs using castellan... but the problem is definitely there. Best regards, -- Dr. Pavlo Shchelokovskyy Principal Software Engineer Mirantis Inc www.mirantis.com

2 2

[kolla] Image building question
by Satish Patel 03 Apr '23

03 Apr '23

Folks, I am playing with kolla image building to understand how it works. I am using the following command to build images and wanted to check with you folks if that is the correct way to do it. $ kolla-build -b ubuntu -t source keystone nova neutron glance Does the above command compile code from source or just download images from remote repositories and re-compile them? because in command output I've not noticed anything related to the compiling process going on. Here is the output of all images produced by kolla-build command. Do I need anything else or is this enough to deploy kolla? root@docker-reg:~# docker images REPOSITORY TAG IMAGE ID CREATED SIZE kolla/mariadb-server 15.1.0 2a497eee8269 26 minutes ago 595MB kolla/cron 15.1.0 342877f26a8a 30 minutes ago 250MB kolla/memcached 15.1.0 0d19a4902644 31 minutes ago 250MB kolla/mariadb-clustercheck 15.1.0 d84427d3c639 31 minutes ago 314MB kolla/mariadb-base 15.1.0 34447e3e59b6 31 minutes ago 314MB kolla/keepalived 15.1.0 82133b09fbf0 31 minutes ago 260MB kolla/prometheus-memcached-exporter 15.1.0 6c2d605f70ee 31 minutes ago 262MB <none> <none> e66b228c2a07 31 minutes ago 248MB kolla/rabbitmq 15.1.0 8de5c39379d3 32 minutes ago 309MB kolla/fluentd 15.1.0 adfd19027862 33 minutes ago 519MB kolla/haproxy-ssh 15.1.0 514357ac4d36 36 minutes ago 255MB kolla/haproxy 15.1.0 e5b9cfdf6dfc 37 minutes ago 257MB kolla/prometheus-haproxy-exporter 15.1.0 a679f65fd735 37 minutes ago 263MB kolla/prometheus-base 15.1.0 afeff3ed5dce 37 minutes ago 248MB kolla/glance-api 15.1.0 a2241f68f23a 38 minutes ago 1.04GB kolla/glance-base 15.1.0 7286772a03a4 About an hour ago 1.03GB kolla/neutron-infoblox-ipam-agent 15.1.0 f90ffc1a3326 About an hour ago 1.05GB kolla/neutron-server 15.1.0 69c844a2e3a9 About an hour ago 1.05GB kolla/neutron-l3-agent 15.1.0 4d87e6963c96 About an hour ago 1.05GB <none> <none> 486da9a6562e About an hour ago 1.05GB kolla/neutron-linuxbridge-agent 15.1.0 e5b3ca7e099c About an hour ago 1.04GB kolla/neutron-bgp-dragent 15.1.0 ac37377820c6 About an hour ago 1.04GB kolla/ironic-neutron-agent 15.1.0 90993adcd74b About an hour ago 1.04GB kolla/neutron-metadata-agent 15.1.0 8522f147f88d About an hour ago 1.04GB kolla/neutron-sriov-agent 15.1.0 8a92ce7d13c0 About an hour ago 1.04GB kolla/neutron-dhcp-agent 15.1.0 5c214b0171f5 About an hour ago 1.04GB kolla/neutron-metering-agent 15.1.0 7b3b91ecd77b About an hour ago 1.04GB kolla/neutron-openvswitch-agent 15.1.0 1f8807308814 About an hour ago 1.04GB kolla/neutron-base 15.1.0 f85b6a2e2725 About an hour ago 1.04GB kolla/nova-libvirt 15.1.0 0f3ecefe4752 About an hour ago 987MB kolla/nova-compute 15.1.0 241b7e7fafbe About an hour ago 1.47GB kolla/nova-spicehtml5proxy 15.1.0 b740820a7ad1 About an hour ago 1.15GB kolla/nova-novncproxy 15.1.0 1ba2f443d5c3 About an hour ago 1.22GB kolla/nova-compute-ironic 15.1.0 716612107532 About an hour ago 1.12GB kolla/nova-ssh 15.1.0 ae2397f4e1c1 About an hour ago 1.11GB kolla/nova-api 15.1.0 2aef02667ff8 About an hour ago 1.11GB kolla/nova-conductor 15.1.0 6f1da3400901 About an hour ago 1.11GB kolla/nova-scheduler 15.1.0 628326776b1d About an hour ago 1.11GB kolla/nova-serialproxy 15.1.0 28eb7a4a13f8 About an hour ago 1.11GB kolla/nova-base 15.1.0 e47420013283 About an hour ago 1.11GB kolla/keystone 15.1.0 e5530d829d5f 2 hours ago 947MB kolla/keystone-ssh 15.1.0 eaa7e3f3985a 2 hours ago 953MB kolla/keystone-fernet 15.1.0 8a4fa24853a8 2 hours ago 951MB kolla/keystone-base 15.1.0 b6f9562364a9 2 hours ago 945MB kolla/barbican-base 15.1.0 b2fdef1afb44 2 hours ago 915MB kolla/barbican-keystone-listener 15.1.0 58bd59de2c63 2 hours ago 915MB kolla/openstack-base 15.1.0 c805b4b3b1c1 2 hours ago 893MB kolla/base 15.1.0 f68e9ef3dd30 2 hours ago 248MB registry 2 8db46f9d7550 19 hours ago 24.2MB ubuntu 22.04 08d22c0ceb15 3 weeks ago 77.8MB

4 8

[tripleo] Removal of TripleO Master Integration and Component Lines
by Ronelle Landy 03 Apr '23

03 Apr '23

Hello All, Removal of TripleO Master Integration and Component Lines Per the decision to not maintain TripleO after the Zed release [1], the master/main integration and component lines are being removed in the following patches: https://review.rdoproject.org/r/c/config/+/48073 https://review.rdoproject.org/r/c/config/+/48074 https://review.rdoproject.org/r/c/rdo-jobs/+/48075 The last promoted release of master through TripleO is: https://trunk.rdoproject.org/centos9-master/current-tripleo/delorean.repo (hash: ddce25bad764dde7e0515094b4d40471), which was promoted on 03/28/2023. Check/gate testing for the master branch is in process of being removed as well. [1] https://review.opendev.org/c/openstack/governance/+/878799

3 2

[kolla] horizon image build failed
by Satish Patel 03 Apr '23

03 Apr '23

Folks, All other images build successfully but when i am trying to build horizon which failed with following error: $ kolla-build --registry docker-reg:4000 -b ubuntu -t source --tag zed horizon INFO:kolla.common.utils.horizon:Collecting XStatic-Dagre>=0.6.4.0 INFO:kolla.common.utils.horizon: Downloading XStatic_Dagre-0.6.4.1-py2.py3-none-any.whl (140 kB) INFO:kolla.common.utils.horizon: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.0/140.0 kB 14.4 MB/s eta 0:00:00 INFO:kolla.common.utils.horizon:Collecting XStatic-Dagre-D3>=0.4.17.0 INFO:kolla.common.utils.horizon: Downloading XStatic_Dagre_D3-0.4.17.0-py2.py3-none-any.whl (357 kB) INFO:kolla.common.utils.horizon: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 357.4/357.4 kB 13.5 MB/s eta 0:00:00 INFO:kolla.common.utils.horizon:Requirement already satisfied: XStatic-Font-Awesome>=4.7.0.0 in /var/lib/kolla/venv/lib/python3.10/site-packages (from vitrage-dashboard==3.6.1.dev2) (4.7.0.0) INFO:kolla.common.utils.horizon:Collecting XStatic-Graphlib>=2.1.7.0 INFO:kolla.common.utils.horizon: Downloading XStatic_Graphlib-2.1.7.0-py2.py3-none-any.whl (51 kB) INFO:kolla.common.utils.horizon: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.5/51.5 kB 114.3 MB/s eta 0:00:00 INFO:kolla.common.utils.horizon:Requirement already satisfied: XStatic-jQuery>=1.8.2.1 in /var/lib/kolla/venv/lib/python3.10/site-packages (from vitrage-dashboard==3.6.1.dev2) (1.12.4.1) INFO:kolla.common.utils.horizon:Collecting XStatic-lodash>=4.16.4.1 INFO:kolla.common.utils.horizon: Downloading XStatic_lodash-4.16.4.2-py3-none-any.whl (167 kB) INFO:kolla.common.utils.horizon: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 167.9/167.9 kB 12.4 MB/s eta 0:00:00 INFO:kolla.common.utils.horizon:Collecting XStatic-moment>=2.8.4.1 INFO:kolla.common.utils.horizon: Downloading XStatic_moment-2.8.4.3-py3-none-any.whl (58 kB) INFO:kolla.common.utils.horizon: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.0/58.0 kB 66.7 MB/s eta 0:00:00 INFO:kolla.common.utils.horizon:Collecting XStatic-Moment-Timezone>=0.5.22.0 INFO:kolla.common.utils.horizon: Downloading XStatic-Moment-Timezone-0.5.22.0.tar.gz (99 kB) INFO:kolla.common.utils.horizon: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 99.7/99.7 kB 45.1 MB/s eta 0:00:00 INFO:kolla.common.utils.horizon: Preparing metadata (setup.py): started INFO:kolla.common.utils.horizon: Preparing metadata (setup.py): finished with status 'error' INFO:kolla.common.utils.horizon: error: subprocess-exited-with-error INFO:kolla.common.utils.horizon: INFO:kolla.common.utils.horizon: × python setup.py egg_info did not run successfully. INFO:kolla.common.utils.horizon: │ exit code: 1 INFO:kolla.common.utils.horizon: ╰─> [6 lines of output] INFO:kolla.common.utils.horizon: Traceback (most recent call last): INFO:kolla.common.utils.horizon: File "<string>", line 2, in <module> INFO:kolla.common.utils.horizon: File "<pip-setuptools-caller>", line 34, in <module> INFO:kolla.common.utils.horizon: File "/tmp/pip-install-dqag1zef/xstatic-moment-timezone_60eeadc1dfb9492781fe3ca90e3b95c2/setup.py", line 2, in <module> INFO:kolla.common.utils.horizon: from xstatic.pkg import moment_timezone as xs INFO:kolla.common.utils.horizon: ImportError: cannot import name 'moment_timezone' from 'xstatic.pkg' (unknown location) INFO:kolla.common.utils.horizon: [end of output] INFO:kolla.common.utils.horizon: INFO:kolla.common.utils.horizon: note: This error originates from a subprocess, and is likely not a problem with pip. INFO:kolla.common.utils.horizon: INFO:kolla.common.utils.horizon:error: metadata-generation-failed INFO:kolla.common.utils.horizon:× Encountered error while generating package metadata. INFO:kolla.common.utils.horizon:╰─> See above for output. INFO:kolla.common.utils.horizon:note: This is an issue with the package mentioned above, not pip. INFO:kolla.common.utils.horizon:hint: See above for details. INFO:kolla.common.utils.horizon: INFO:kolla.common.utils.horizon:Removing intermediate container e6cd437ba529 ERROR:kolla.common.utils.horizon:Error'd with the following message ERROR:kolla.common.utils.horizon:The command '/bin/sh -c ln -s horizon-source/* horizon && sed -i /^horizon=/d /requirements/upper-constraints.txt && SETUPTOOLS_USE_DISTUTILS=stdlib python3 -m pip --no-cache-dir install --upgrade -c /requirements/upper-constraints.txt /horizon && mkdir -p /etc/openstack-dashboard && cp -r /horizon/openstack_dashboard/conf/* /etc/openstack-dashboard/ && cp /horizon/openstack_dashboard/local/local_settings.py.example /etc/openstack-dashboard/local_settings && cp /horizon/manage.py /var/lib/kolla/venv/bin/manage.py && if [ "$(ls /plugins)" ]; then SETUPTOOLS_USE_DISTUTILS=stdlib python3 -m pip --no-cache-dir install --upgrade -c /requirements/upper-constraints.txt /plugins/*; fi && for locale in /var/lib/kolla/venv/lib/python3.10/site-packages/*/locale; do (cd ${locale%/*} && /var/lib/kolla/venv/bin/django-admin compilemessages) done && chmod 644 /usr/local/bin/kolla_extend_start' returned a non-zero code: 1 INFO:kolla.common.utils:========================= INFO:kolla.common.utils:Successfully built images INFO:kolla.common.utils:========================= INFO:kolla.common.utils:base INFO:kolla.common.utils:openstack-base INFO:kolla.common.utils:=========================== INFO:kolla.common.utils:Images that failed to build INFO:kolla.common.utils:===========================

4 8

[tc] No TC weekly meeting next week
by Nikolla, Kristi 31 Mar '23

31 Mar '23

Hi all, There will be no TC meeting on Tuesday, April 4th. The next TC meeting will be held on Tuesday, April 11 at 18.00 UTC. More information and an ICS file can be found here https://meetings.opendev.org/#Technical_Committee_Meeting Thank you, Kristi Nikolla

1 0

[release] Release countdown for week R-26, Apr 03 - 07
by Elõd Illés 31 Mar '23

31 Mar '23

Hi, Welcome back to the release countdown emails! These will be sent at major points in the 2023.2 Bobcat development cycle, which should conclude with a final release on October 4th, 2023. Development Focus ----------------- At this stage in the release cycle, focus should be on planning the 2023.2 Bobcat development cycle and approving 2023.2 Bobcat specs. General Information ------------------- 2023.2 Bobcat is a 28 weeks long development cycle. In case you haven't seen it yet, please take a look over the schedule for this release: https://releases.openstack.org/bobcat/schedule.html By default, the team PTL is responsible for handling the release cycle and approving release requests. This task can (and probably should) be delegated to release liaisons. Now is a good time to review release liaison information for your team and make sure it is up to date: https://opendev.org/openstack/releases/src/branch/master/data/release_liais… By default, all your team deliverables from the 2023.1 Antelope release are continued in 2023.2 Bobcat with a similar release model. Upcoming Deadlines & Dates -------------------------- Bobcat-1 milestone: May 11th, 2023 OpenInfra Summit + PTG Vancouver 2023 - June 13-15, 2023 Előd Illés irc: elodilles @ #openstack-release

1 0

[heat][qa][requirements] Pinning tempest in stable/xena constraints
by Takashi Kajinami 31 Mar '23

31 Mar '23

Hello, I have had some local discussions with gmann, but I'd really like to move this discussion forward to fix the broken stable/xena gate in heat so I will start this thread, hoping the thread can provide more context behind my proposal. Historically stable branches of heat have been frequently affected by any change in requirements of tempest. This is mainly because in our CI we install our own in-tree integration tests[1] into tempest venv where tempest and heat-tempest-plugin are installed. Because in-tree integration tests are tied to that specific stable branch, this has been often causing conflicts in requirements (master constraint vs stable/X constraint). [1] https://github.com/openstack/heat/tree/master/heat_integrationtests In the past we changed our test installation[2] to use stable constraint to avoid this conflicts, but this approach does no longer work since stable/xena because 1. stable/xena u-c no longer includes tempest 2. latest tempest CAN'T be installed with stable/xena u-c because current tempest requires fasteners>=0.16.0 which conflicts with 0.14.1 in stable/xena u-c. [2] https://review.opendev.org/c/openstack/heat/+/803890 https://review.opendev.org/c/openstack/heat/+/848215 I've proposed the change to pin tempest[3] in stable/xena u-c so that people can install tempest with stable/xena u-c. [3] https://review.opendev.org/c/openstack/requirements/+/878228 I understand the reason tempest was removed from u-c was that we should use the latest tempest to test recent stable releases.I agree we can keep tempest excluded for stable/yoga and onwards because tempest is installable with their u-c, but stable/xena u-c is no longer compatible with master. Adding pin to xena u-c does not mainly affect the policy to test stable branches with latest tempest because for that we anyway need to use more recent u-c. I'm still trying to find out the workaround within heat but IMO adding tempest pin to stable/xena u-c is harmless but beneficial in case anyone is trying to use tempest with stable/xena u-c. Thank you, Takashi Kajinami

4 10

Reminder! In Person PTG 2023 Team Signup Deadline
by Kendall Nelson 31 Mar '23

31 Mar '23

Hello Everyone, This is the last call to sign your team up for the *in-person* Project Teams Gathering (PTG) happening in Vancouver at the OpenInfra Summit! If you haven't already done so and your team is interested in participating, please complete the survey[1] by April 2nd, 2023 at 7:00 UTC. Registration for the PTG is included as a part of registration for the OpenInfra Summit in Vancouver. Prices increase May 5th so register soon! Thanks! -Kendall (diablo_rojo) [1] Team Survey: https://openinfrafoundation.formstack.com/forms/june2023_ptg_survey [2] Summit Registration: https://vancouver2023.openinfra.dev/ <https://vancouver2023.openinfra.dev/>

1 0

[openstack-helm][ptg] agenda changes for (Fri) Mar/31/2023
by Vladimir Kozhukalov 31 Mar '23

31 Mar '23

Dear helmers, Since we don't have any new topics in the etherpad [1] to discuss apart from those we discussed yesterday I think we can unbook our meeting room. Thanks for attending yesterday. I'll send the summary of our discussions later. [1] https://etherpad.opendev.org/p/march2023-ptg-openstack-helm -- Best regards, Kozhukalov Vladimir

1 0

[nova][ptg] Today's agenda (Friday)
by Sylvain Bauza 31 Mar '23

31 Mar '23

Hey folks, Today is our last vPTG day. The agenda is so : 13:00 UTC - 14:45 UTC : * Users reported exhaustion of primary keys ('id') in some large tables like system_metadata. How could we achieve a data migration from sa.Integer to sa.BigInteger ? * Should cold-migrate have a specific new policy like os_compute_api:os-migrate-server:migrate-specify-host to accept non-admins to migrate if they don't provide a host value ? * Discuss the next steps with compute hostname robustification * Disable compute services after being discovered * Limited lower constraint job in nova and placement 14:45 UTC - 15:00 UTC : break 15:00 UTC - 17:00 UTC : * Openstack server show command output (cross-project discussion with openstackSDK contributors) * Should we support dynamically disabling post copy live migration based on vm_state is paused ? * Evacuation with multiple allocations * Bobcat proposed planning (cont.) * trim support for virtio-blk feature or bug? * Summit/PTG : what could we be doing for the physical PTG ? (if we have time) As a reminder, you can look at the topics here : https://etherpad.opendev.org/p/nova-bobcat-ptg#L426 and you can add your IRC nick in the courtesy ping list if you want to be around. Hope you had a good week and see you in an hour by now ! -Sylvain

1 0

[Glance] Bobcat PTG Updates
by Pranali Deore 31 Mar '23

31 Mar '23

Hello Everyone, We have concluded Glance vPTG yesterday and I will share the summary early next week. You can find tentative milestone wise priorities for glance in PTG etherpad[1]. Join us in the weekly meeting coming Thursday if you have any doubts/suggestions. Thanks & Regards, Pranali [1]: https://etherpad.opendev.org/p/glance-bobcat-ptg#L143

1 0

[neutron][ptg] Today's agenda
by Rodolfo Alonso Hernandez 31 Mar '23

31 Mar '23

Hello Neutrinos: Today is the last day of the PTG. You can check the agenda in [1]. This is a quick summary: * Open hour for core reviewers: join us! Neutron needs you. * FIPS jobs: status and distro support. If you have a last minute topic, today is the best moment to show up and present it (it doesn't matter if it is not on the agenda). Regards. [1]https://etherpad.opendev.org/p/neutron-bobcat-ptg

1 0

Migration from linuxbridge to ovs
by Luk 31 Mar '23

31 Mar '23

Hello, Can You share some thoughts/ideas or some clues regarding migration from linux bridge to ovs ? Does this migration is posible without interrupting traffic from VMs ? We have now linuxbridge with l3-ha, and we noticed that for example when doing live migration of VM from linuxbridge baked compute to openvswitch compute is created bridge... inside openvswitch, instead adding qvo device to br-int: Bridge brq91dc40ac-ea datapath_type: system Port qvo84e2bd98-e9 Interface qvo84e2bd98-e9 Port brq91dc40ac-ea Interface brq91dc40ac-ea type: internal After removing the brq91dc40ac-ea from ovs, and hard reboot, the qvo interface is added properly to br-int: Bridge br-int Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure datapath_type: system Port qvo84e2bd98-e9 tag: 1 Interface qvo84e2bd98-e9 Also, before hard reboot, there is no flow for br-int or any other openvswitch bridge regarding this VM/ip. Does anyone have same problems ? Have tried to migrate from lb to ovs ? Openstack version: ussuri OS: ubuntu 20 Regards Lukasz

3 3

[Octavia] moving back to Launchpad
by Gregory Thiemonge 31 Mar '23

31 Mar '23

Hi Folks, During the Antelope PTG, we discussed the move from Storyboard to Launchpad, and in the Bobcat PTG session, we decided to switch at the beginning of the B cycle (now). The Octavia Launchpad [0] is now re-enabled, we are marking all the old entries as Invalid. We don't plan to have an automated migration script, we will duplicate manually the most recent bugs reported in Storyboard (not started bugs). I'm also proposing patches to update the links to the bug tracker in the Octavia projects. Greg [0] https://launchpad.net/octavia

2 1

[kolla-ansible] deploy cinder and glance with multi-backend
by wodel youchi 30 Mar '23

30 Mar '23

Hi, I need to deploy cinder and glance with multi-backend. My experience for now is simple, my deployment is an HCI built on top of ceph storage, and I am using it to store both cinder volumes and glance images. I have some questions if you can help: - can multi-backend be deployed at first run with kolla-ansible, or do I need to do at least two runs? - reading the doc about multi-backend, I saw that the first backend is specified, do I need to remove the creation of the first backend from glabals.yml and put it in a configuration file cinder.conf and glance.conf? In other words, do I have to create all the backends from the config files, or can I still create the first one using globals.yml and the rest from the config files? I hope my question is clear. Regards.

1 0

[neutron][ptg] Today's agenda
by Rodolfo Alonso Hernandez 30 Mar '23

30 Mar '23

Hello Neutrinos: Same as yesterday, we have a packed agenda. Today we'll have the Nova-Neutron meetings, starting at 13UTC. Quick summary: * delete_on_termination for Neutron ports * Blueprint: "Add support for Napatech LinkVirt SmartNICs" * https://bugs.launchpad.net/neutron/+bug/1986003 * ovn-bgp-agent roadmap * neutron-dynamic-routing: Make static scheduler finally the default? See you in a few minutes. Regards.

1 0

[nova][ptg] Today's agenda (Thursday)
by Sylvain Bauza 30 Mar '23

30 Mar '23

Heya again, Yesterday was a very productive day. Thanks folks. Today, we'll have mostly cross-project discussions but we'll also try to discuss about 3 topics : - 13:00 - 14:30 UTC : Nova-Neutron cross-project sessions* in the neutron room (tbd)* - 14:30 - 14:45 UTC : Transition Xena to EM, any concerns ? - 14:45 - 15:00 UTC : break - 15:00 - 15:30 UTC : Glance/Cinder/Nova cross-project session about secure glance Direct URLs *in the glance room (newton)* - 15:30 - 16:30 UTC : Cinder/Nova cross-project session* in the nova room (diablo)* - 16:30 - 16:50 UTC : Discuss the next steps with compute hostname robustification - 16:50 - 16:00 UTC : (tentatively) Instance.name is not persisted and just sub'd by every service calling the object Thanks and enjoy this day. -Sylvain

1 1

Nova undefine secret | openstack | wallaby
by Swogat Pradhan 30 Mar '23

30 Mar '23

Hi, For some reason, i had to redeploy ceph for my hci nodes and then found that the deployment command is giving out the following error: 2023-03-28 01:49:46.709605 | | WARNING | ERROR: Can't run container nova_libvirt_init_secret stderr: error: Failed to set attributes from /etc/nova/secret.xml error: internal error: a secret with UUID bd136bb0-fd78-5429-ab80-80b8c571d821 already defined for use with client.openstack secret 2023-03-28 01:49:46.711176 | 48d539a1-1679-623b-0af7-000000004b45 | FATAL | Create containers managed by Podman for /var/lib/tripleo-config/container-startup-config/step_4 | dcn01-hci-0 | error={"changed": false, "msg": "Failed containers: nova_libvirt_init_secret"} Can you please tell me how I can undefine the existing secret? With regards, Swogat Pradhan

3 5

[oslo][heat][masakari][senlin][venus][all] oslo.db 13.0.0 will remove sqlalchemy-migrate support
by Stephen Finucane 30 Mar '23

30 Mar '23

tl;dr: Projects still relying on sqlalchemy-migrate for migrations need to start their switch to alembic immediately. Projects with "legacy" sqlalchemy-migrated based migrations need to drop them. A quick heads up that oslo.db 13.0.0 will be release in the next month or so and will remove sqlalchemy-migrate support and formally add support for sqlalchemy 2.x. The removal of sqlalchemy-migrate support should only affect projects using oslo.db's sqlalchemy-migrate wrappers, as opposed to using sqlalchemy-migrate directly. For any projects that rely on this functionality, a short-term fix is to vendor the removed code [1] in your project. However, I must emphasise that we're not removing sqlalchemy-migrate integration for the fun of it: it's not compatible with sqlalchemy 2.x and is no longer maintained. If your project uses sqlalchemy-migrate and you haven't migrated to alembic yet, you need to start doing so immediately. If you have migrated to alembic but still have sqlalchemy- migrate "legacy" migrations in-tree, you need to look at dropping these asap. Anything less will result in broken master when we bump upper-constraints to allow sqlalchemy 2.x in Bobcat. I've listed projects in $subject that appear to be using the removed modules. For more advice on migrating to sqlalchemy 2.x and alembic, please look at my previous post on the matter [2]. Cheers, Stephen [1] https://review.opendev.org/c/openstack/oslo.db/+/853025 [2] https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.…

3 3