April 2023 - openstack-discuss - lists.openstack.org

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

How to config DPDK compute in OVN
by dangerzone ar 22 Apr '25

22 Apr '25

Hi....as far I understand ovs does support dpdk compute to be deployed. I'm referring to the guideline ' *https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html <https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html>* ' ... It's too brief.. May I know how to config dpdk compute in OVN networking openstack? Hope someone could help what is the config to be done.... Currently compute is run without dpdk over openstack train ovn based. Please need some info on how I can proceed. Thank you

2 1

ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version
by Swogat Pradhan 23 Dec '24

23 Dec '24

Hi, I am currently trying to deploy openstack wallaby using tripleo arch. I created the network jinja templates, ran the following commands also: #openstack overcloud network provision --stack overcloud --output networks-deployed-environment.yaml custom_network_data.yaml # openstack overcloud network vip provision --stack overcloud --output vip-deployed-environment.yaml custom_vip_data.yaml # openstack overcloud node provision --stack overcloud --overcloud-ssh-key /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml and used the environment files in the openstack overcloud deploy command: (undercloud) [stack@hkg2director ~]$ cat deploy.sh #!/bin/bash THT=/usr/share/openstack-tripleo-heat-templates/ CNF=/home/stack/ openstack overcloud deploy --templates $THT \ -r $CNF/templates/roles_data.yaml \ -n $CNF/workplace/custom_network_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e $CNF/templates/node-info.yaml \ -e $CNF/templates/scheduler-hints.yaml \ -e $CNF/workplace/networks-deployed-environment.yaml \ -e $CNF/workplace/vip-deployed-environment.yaml \ -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ -e $CNF/workplace/custom-net-bond-with-vlans.yaml Now when i run the ./deploy.sh script i encounter an error stating: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command.: tripleoclient.exceptions.InvalidConfiguration: Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command. 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END return value: 1 Can someone tell me where the mistake is? With regards, Swogat Pradhan

4 4

[kolla] [openeuler] CI jobs status
by Michał Nasiadka 26 Nov '24

26 Nov '24

Hello Koalas, OpenEuler jobs in master (to be Antelope) are failing for a long time. If no one is interested in maintaining OpenEuler support in Kolla-Ansible - I will propose dropping it’s support on Wednesday Kolla meeting. Best regards, Michal

4 3

Kolla Ansible on Ubuntu 20.04 - cloud-init & other network issues
by Tobias McNulty 10 Oct '24

10 Oct '24

Hi, I'm attempting to use Kolla Ansible 14.6.0 to deploy OpenStack Yoga on a small 3-node Ubuntu 20.04 cluster. The nodes have 128 GB RAM each, dual Xeon processors, and dual 10G Intel NICs. The NICs are connected to access ports on a 10G switch with separate VLANs for the local and external networks. All the playbooks run cleanly, but cloud-init is failing in the Ubuntu 20.04 and 22.04 VMs I attempt to boot. The VM images are unmodified from https://cloud-images.ubuntu.com/, and cloud-init works fine if I mount a second volume with user-data. The error is a timeout attempting to reach 169.254.169.254. This occurs both when booting a VM in an internal routed network and directly in an external network. I tried various neutron plugin agents (ovn, linuxbridge, and openvswitch both with and without firewall_driver = openvswitch <https://docs.openstack.org/kolla-ansible/latest/reference/networking/neutro…>) first with a clean install of the entire OS each time, all with the same result. Running tcpdump looking for 169.254.169.254 shows nothing. As a possible clue, the virtual NICs are unable to pass any traffic (e.g., to reach an external DHCP server) unless I completely disable port security on the interface (even if the associated security group is wide open). But disabling port security does not fix cloud-init (not to mention I don't really want to disable port security). Are there any additional requirements related to deploying OpenStack with Kolla on Ubuntu 20.04? This is a fairly vanilla configuration using the multinode inventory as a starting point. I tried to follow the Quick Start <https://docs.openstack.org/kolla-ansible/yoga/user/quickstart.html> as closely as possible; the only material difference I see is that I'm using the same 3 nodes for control + compute. I am using MAAS so it's easy to get a clean OS install on all three nodes ahead of each attempt. I plan to try again with the standard (non-HWE) kernel just in case, but otherwise I am running out of ideas. In case of any additional clues, here are my globals.yml and inventory file, along with the playbook I'm using to configure the network, images, VMs, etc., after bootstrapping the cluster: https://gist.github.com/tobiasmcnulty/7dbbdbc67abc08cbb013bf5983852ed6 Thank you in advance for any advice! Cheers, Tobias

5 9

Live migration fails
by Szabo, Istvan (Agoda) 06 Aug '24

06 Aug '24

Hi, I have couple of compute nodes where the live migration fails with existing vms. When I quickly spawn a vm and try live migration it works so I assume shouldn't be a big problem with the compute node. However I have many existing vms where it fails with a servername not found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 ERROR nova.conductor.tasks.migrate [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Unable to find record for source node servername on servername: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Failed to compute_task_migrate_server: Compute host servername could not be found.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Setting instance to ACTIVE state.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.672 227612 ERROR oslo_messaging.rpc.server [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Exception during message handling: ComputeHostNotFound: Compute host am-osfecn-4025 Tried with this command: nova live-migration --block-migrate id. Any idea? Thank you. ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

5 16

[ptl][tc] OpenStack packages PyPi additional external maintainers audit & cleanup
by Ghanshyam Mann 16 May '24

16 May '24

Hi PTLs, As you might know or have seen for your project package on PyPi, OpenStack deliverables on PyPi have additional maintainers, For example, https://pypi.org/project/murano/, https://pypi.org/project/glance/ We should keep only 'openstackci' as a maintainer in PyPi so that releases of OpenStack deliverables can be managed in a single place. Otherwise, we might face the two sets of maintainers' places and packages might get released in PyPi by additional maintainers without the OpenStack project team knowing about it. One such case is in Horizon repo 'xstatic-font-awesome' where a new maintainer is added by an existing additional maintainer and this package was released without the Horizon team knowing about the changes and release. - https://github.com/openstack/xstatic-font-awesome/pull/2 To avoid the 'xstatic-font-awesome' case for other packages, TC discussed it in their weekly meetings[1] and agreed to audit all the OpenStack packages and then clean up the additional maintainers in PyPi (keep only 'openstackci' as maintainers). To help in this task, TC requests project PTL to perform the audit for their project's repo and add comments in the below etherpad. - https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup Thanks to knikolla to automate the listing of the OpenStack packages with additional maintainers in PyPi which you can find the result in output.txt at the bottom of this link. I have added the project list of who needs to check their repo in etherpad. - https://gist.github.com/knikolla/7303a65a5ddaa2be553fc6e54619a7a1 Please complete the audit for your project before March 15 so that TC can discuss the next step in vPTG. [1] https://meetings.opendev.org/meetings/tc/2023/tc.2023-01-11-16.00.log.html#… -gmann

18 27

[Watcher] How use Watcher to scheduler Openstack resource without cpu_usage
by Nguyễn Hữu Khôi 06 Mar '24

06 Mar '24

Hello guys. I want to use watcher to scheduler openstack resources but some openstack versions are missing cpu_usage metric. How do we use Watcher without it? Thank you. Nguyen Huu Khoi

3 5

[designate] Proposal to deprecate the agent framework and agent based backends
by Michael Johnson 09 Jan '24

09 Jan '24

TLDR: The Designate team would like to deprecate the backend agent framework and the agent based backends due to lack of development and design issues with the current implementation. The following backends would be deprecated: Bind9 (Agent), Denominator, Microsoft DNS (Agent), Djbdns (Agent), Gdnsd (Agent), and Knot2 (Agent). Designate includes many backend DNS server drivers[1], many of which are "native" (also known as xfr type backends) backend implementations. In addition to the "native" backends, Designate has an agent backend[2] that supports other backends via an agent process. To quote the agent backend documentation[2]: This backend uses an extension[3] of the DNS protocol itself to send management requests to the remote agent processes, where the requests will be actioned. The rpc traffic between designate and the agent is both unauthenticated and unencrypted. Do not run this traffic over unsecured networks. Here are the reasons we are proposing to deprecate the agent framework now: 1. The agent protocol used by Designate is using an "unassigned"[4][5] DNS opcode (14) that is causing problems with the dnspython library >= 2.3.0 which is now validating the opcode when building DNS messages. It is a bad practice to use "unassigned" values as they may be officially assigned at any time, likely with an incompatible message format. 2. The agent backends are not tested in the OpenStack jobs[1]. 3. Many of the agent backends have been marked as "Experimental" since 2016 with no additional contributions beyond general repository code maintenance. 4. The protocol between the Designate worker process and the agent process is unauthenticated and unencrypted. 5. We do not know of a development resource to rewrite the agent framework protocol to address issue #1 and #4 above. 6. The introduction of catalog zones[6] may eliminate the need for some of the agent based backend drivers. By marking the agent framework and agent based backends "deprecated" in the Antelope cycle, we would remove the code no earlier than the "C" release of OpenStack (per the OpenStack deprecation policy[7]). In the meantime, issue #1 has been worked around by overriding the dnspython opcode validation[7] as needed in the Designate code (similar to a monkey patch). This is not a sustainable long term solution. The following backend agent based drivers would be marked "deprecated" in addition to the agent framework itself: Bind9 (Agent) Denominator Microsoft DNS (Agent) Djbdns (Agent) Gdnsd (Agent) Knot2 (Agent) We plan to propose patches for the deprecation over the next week, but will not merge them until at least January 24th to allow time for comment from the community. If you have concerns about this deprecation plan or are interested in rewriting the agent framework protocol to address the above issues, please reply to this announcement. Michael [1] https://docs.openstack.org/designate/latest/admin/support-matrix.html [2] https://docs.openstack.org/designate/latest/admin/backends/agent.html [3] https://github.com/openstack/designate/blob/master/designate/backend/privat… [4] https://www.rfc-editor.org/rfc/rfc6895.html#section-2.2 [5] https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-pa… [6] https://www.ietf.org/archive/id/draft-ietf-dnsop-dns-catalog-zones-08.txt [7] https://docs.openstack.org/project-team-guide/deprecation.html [8] https://review.opendev.org/c/openstack/designate/+/870678

3 13

[ptl][tc][winstackers] Final call for Winstackers PTL and maintainers
by James Page 11 Sep '23

11 Sep '23

Hi All As announced by Lucian last November (see [0]) Cloudbase Solutions are no longer in a position to maintain support for running OpenStack on Windows and have also ceased operation of their 3rd party CI for the windows support across a number of OpenStack projects. This situation has resulted in the Winstackers project becoming PTL-less for the 2023.2 cycle with no volunteers responding to the TC's call to fill this role and take this feature in OpenStack forward (see [1]). This is the final call for any maintainers to step forward if this feature is important to them in OpenStack. The last user survey in 2022 indicated that 2% of respondents were running on Hyper-V so this might be important enough to warrant a commitment from someone operating OpenStack on Windows to maintain these features going forward. Here is a reminder from Lucian's original email on the full list of projects which are impacted in some way: * nova hyper-v driver - in-tree plus out-of-tree compute-hyperv driver * os-win - common Windows library for Openstack * neutron hyperv ml2 plugin and agent * ovs on Windows and neutron ovs agent support * cinder drivers - SMB and Windows iSCSI * os-brick Windows connectors - iSCSI, FC, SMB, RBD * ceilometer Windows poller * manila Windows driver * glance Windows support * freerdp gateway The lack of 3rd party CI for testing all of this really needs to be addressed as well. If no maintainers are forthcoming between now and the next PTG in June the TC will need to officially retire the project and start the process of removing support for Windows across the various projects that support this operating system in some way - either directly or through the use of os-win. For clarity this call refers to the use of the Hyper-V virtualisation driver and associated Windows server components to provide WIndows based OpenStack Hypervisors and does not relate to the ability to run Windows images as guests on OpenStack. Regards James [0] https://lists.openstack.org/pipermail/openstack-discuss/2022-November/03104… [1] https://lists.openstack.org/pipermail/openstack-discuss/2023-March/032888.h…

5 9

[openstack-ansible] Recommended branch for a production environment?
by Roger Rivera 05 Sep '23

05 Sep '23

Hello, We have deployed Openstack-Ansible in a test environment and we were wondering what the recommended repository branch is to deploy in a production environment that will be integrated with a separate Ceph cluster. 1. We are thinking about pinning to the stable/zed. Is that recommended over the master branch? 2. Are bugs ironed out on both master and stable/zed with the same cadence? 3. Additionally, is Debian 11 a better alternative than Ubuntu 22 for target hosts? We noticed Ubuntu 22.04 support was added recently, whereas Debian 11 has been supported for quite some time now which leads us to believe its stability/integration could be more mature at this point? Any suggestions will be appreciated. Thank you, -- *Roger *

3 7

[puppet] Proposing core team clean up
by Takashi Kajinami 23 Aug '23

23 Aug '23

Hello, I went through the current core reviewers and found out that the current list includes some people no longer working in the OpenStack community. I'd like to propose removing the following persons from the core reviewer team as we haven't seen these people making any activity in the community recently. * Pradeep Kilambi - puppet-aodh, puppet-ceilometer, puppet-gnocchi, puppet-panko * Richard Megginson - puppet-keystone * Denis Egorenko - puppet-murano * Serg Melikyan - puppet-murano * Sergey Kolekonov - puppet-neutron * David Moreau Simard - puppet-openstack-integration * Yu xingchao - all modules Also, I confirmed that Wes Hayutin is no longer working for OpenStack and he agreed to retire from the core reviewer position. Thus I'll remove him as well. I'll be open for any feedback for one week to hear any thoughts/feedback about this removal. I'd like to take this moment to thank all people for their past work. We always welcome them back if they resume their work in this project. Thank you, Takashi KAJINAMI

3 4

[cloudkitty] question to the scope_key in cloudkitty
by Noah Elias Feldt 28 Jul '23

28 Jul '23

Hello! I am currently trying to get CloudKitty to work with the Prometheus Collector and PyScripts. I noticed that the scope_key for Prometheus can only be changed once in the global configuration file. This was a problem for me because we need different scope_keys for different metrics. Many of the metrics still need the scope key "tenant_id" and some others "project_id". I have found in the documentation of CloudKitty no way to change this for each metric, so I have internally changed something in the source code and added the function to set the scope_key for each metric individually. I just wanted to ask if you would be interested if I submit the small change upstream? Thank you. Regards Noah Feldt

3 3

[cinder] cinder-backup volume stuck in creating
by Satish Patel 27 Jul '23

27 Jul '23

Folks, I have configured nova and cinder with ceph storage. VMs running on ceph storage but now when i am trying to create a backup of cinder volume its getting stuck on creating and doing nothing. Logs also do not give any indication of bad. My cinder.conf [DEFAULT] enabled_backends = rbd-1 backup_driver = cinder.backup.drivers.ceph.CephBackupDriver backup_ceph_conf = /etc/ceph/ceph.conf backup_ceph_user = cinder-backup backup_ceph_chunk_size = 134217728 backup_ceph_pool = backups backup_ceph_stripe_unit = 0 backup_ceph_stripe_count = 0 restore_discard_excess_bytes = true osapi_volume_listen = 10.73.0.181 osapi_volume_listen_port = 8776 Output of "openstack volume service list" showing cinder-backup service is up but when i create a backup it's getting stuck in this stage and no activity. I am not seeing anything getting transferred to the ceph backups pool also. Any clue? or method to debug? # openstack volume backup list --all +--------------------------------------+------+-------------+----------+------+ | ID | Name | Description | Status | Size | +--------------------------------------+------+-------------+----------+------+ | bc844d55-8c5a-4bd3-b0e9-7c4c780c95ad | foo1 | | creating | 20 | +--------------------------------------+------+-------------+----------+------+

5 7

Re: [all] [oslo.messaging] Interest in collaboration on a NATS driver
by Tobias Urdin 22 Jul '23

22 Jul '23

Hello Kai, Great to hear that you’re interested! I’m currently just testing this as a POC directly in devstack so there is real-world testing (not should it yet I think, needs more development first). Best regards Tobias > On 30 Aug 2022, at 18:32, Kai Bojens <kb(a)kbojens.de> wrote: > > Am 29.08.22 um 15:46 schrieb Tobias Urdin: > >> If anybody, or any company, out there would be interested in collaborating in a project to bring this support and maintain it feel free to >> reach out. I’m hoping somebody will bite but atleast I’ve put it out there for all of you. > > Hi, > I am very much interested to take a closer look at your work and maybe contribute to it. Although I'm working with OpenStack for my employer at the moment, I'd do this in my spare time as I'm not sure that I can convince him to add another staging system with a full OpenStack installation just for developing a RabbitMQ replacement. That's not on our agenda as we are mostly users and not developers of OpenStack. > > As I'm pretty new to the messaging topic and had just heard of NATS and your idea, I'll first would have to dive into NATS and then I would take a closer look at your code and maybe try to create some documentation or tests. So, I can't promise anything but as I said I'm very interested in your approach as I also see the massive load from RabbitMQ. > > This brings me to the most important question: Where do you run and test your code? > > Greetings, > Kai

8 30

[ptl] Need PTL volunteer for OpenStack Vitrage
by Ghanshyam Mann 11 Jul '23

11 Jul '23

Hi Eyal, I am reaching out to you as you were PTL for OpenStack Vitrage project in the last cycle. There is no PTL candidate for the next cycle (2023.2), and it is on the leaderless project list. Please check if you or anyone you know would like to lead this project. - https://etherpad.opendev.org/p/2023.2-leaderless Also, if anyone else would like to help leading this project, this is time to let TC knows. -gmann

3 3

Experience with VGPUs
by Oliver Weinmann 22 Jun '23

22 Jun '23

Dear All, we are planning to have a POC on VGPUs in our Openstack cluster. Therefore I have a few questions and generally wanted to ask how well VGPUs are supported in Openstack. The docs, in particular: https://docs.openstack.org/nova/zed/admin/virtual-gpu.html explain quite well the general implementation. But I am more interested in general experience with using VGPUs in Openstack. We currently have a small YOGA cluster, planning to upgrade to Zed soon, with a couple of compute nodes. Currently our users use consumer cards like RTX 3050/3060 on their laptops and the idea would be to provide VGPUs to these users. For this I would like to make a very small POC where we first equip one compute node with an Nvidia GPU. Gladly also a few tips on which card would be a good starting point are highly appreciated. I know this heavily depends on the server hardware but this is something I can figure out later. Also do we need additional software licenses to run this? I saw this very nice presentation from CERN on VGPUs: https://indico.cern.ch/event/776411/contributions/3345183/attachments/18516… In the table they are listing Quadro vDWS licenses. I assume we need these in order to use the cards? Also do we need something like Cyborg for this or is VGPU fully implemented in Nova? Best Regards, Oliver

17 36

[ironic] ARM Support in CI: Call for vendors / contributors / interested parties
by Jay Faulkner 14 Jun '23

14 Jun '23

Hi stackers, Ironic has published an experimental Ironic Python Agent image for ARM64 ( https://tarballs.opendev.org/openstack/ironic-python-agent-builder/dib/file…) and discussed promoting this image to supported via CI testing. However, we have a problem: there are no Ironic developers with easy access to ARM hardware at the moment, and no Ironic developers with free time to commit to improving our support of ARM hardware. So we're putting out a call for help: - If you're a hardware vendor and want your ARM hardware supported? Please come talk to the Ironic community about setting up third-party-CI. - Are you an operator or contributor from a company invested in ARM bare metal? Please come join the Ironic community to help us build this support. Thanks, Jay Faulkner Ironic PTL

6 11

[storlets] Proposal to make Train/Ussuri/Victoria EOL
by Takashi Kajinami 13 Jun '23

13 Jun '23

Hello, Currently we have multiple stable branches open but we haven't seen any backport proposed so far. To reduce number of branches we have to maintain, I'd like to propose retiring old stable branches(train, ussuri and victoria). In case you have any concerns, please let me know. Thank you, Takashi Kajinami

1 1

Kubernetes Conformance 1.24 + 1.25
by Kendall Nelson 05 Jun '23

05 Jun '23

Hello All! First of all, I want to say a huge thanks to Guilherme Steinmuller for all his help ensuring that OpenStack Magnum remains Kubernetes Certified [1]! We are certified for v1.24! However, in digging into the set of conformance tests this round, it appears that we will not be able to get 1.25 certified with the current Magnum. This means that in May when we have to re-cert (every 3 months they reach an EOL because that's k8s release cadence) we will not be able to maintain this which is the other part of why I wanted to write this email - to raise awareness and ask for help digging into what needs to happen to make sure we can retain this. Tangentially related, Guilherme has posted a patch with documentation for how to run the conformance tests for yourself and it could use some reviews [2]. And one last thing, I have begun conversation with the opendev folks about what it might take to setup jobs to run the conformance tests periodically so that we don't have to manually run them when it's time to recert. I am planning on submitting a forum topic for this for those interested :) Long live LOKI! - Kendall Nelson [1] https://github.com/cncf/k8s-conformance/pull/2446 [2] https://review.opendev.org/c/openstack/magnum/+/849156

5 10

[ironic][stable] Proposing EOL of ironic project branches older than Wallaby
by Jay Faulkner 01 Jun '23

01 Jun '23

Hi all, Ironic has a large amount of stable branches still in EM. We need to take action to ensure those branches are either retired or have CI repaired to the point of being usable. Specifically, I'm looking at these branches across all Ironic projects: - stable/queens - stable/rocky - stable/stein - stable/train - stable/ussuri - stable/victoria In lieu of any volunteers to maintain the CI, my recommendation for all the branches listed above is that they be marked EOL. If someone wants to volunteer to maintain CI for those branches, they can propose one of the below paths be taken instead: 1 - Someone volunteers to maintain these branches, and also report the status of CI of these older branches periodically on the Ironic whiteboard and in Ironic meetings. If you feel strongly that one of these branches needs to continue to be in service; volunteering in this way is how to save them. 2 - We seriously reduce CI. Basically removing all tempest tests to ensure that CI remains reliable and able to merge emergency or security fixes when needed. In some cases; this still requires CI fixes as some older inspector branches are failing *installing packages* in unit tests. I would still like, in this case, that someone volunteers to ensure the minimalist CI remains happy. My intention is to let this message serve as notice and a waiting period; and if I've not heard any response here or in Monday's Ironic meeting (in 6 days), I will begin taking action on retiring these branches. This is simply a start; other branches (including bugfix branches) are also in bad shape in CI, but getting these retired will significantly reduce the surface area of projects and branches to evaluate. I know it's painful to drop support for these branches; but we've provided good EM support for these branches for a long time and by pruning them away, we'll be able to save time to dedicate to other items. Thanks, Jay Faulkner

3 5

[Cinder][LVM backend] LVM vg backed by a shared LUN
by wodel youchi 31 May '23

31 May '23

Hi, The examples I could find on the internet using LVM as backend for Cinder, they expose a local disk using lvm via Cinder. I did this configuration and I am wondering if it's correct, especially from a "simultaneous access" point of view. I have an iSCSI target backed by targetcli that exposes a LUN to my compute nodes. I did configure the iscsi connexion manually on each one of them and they all see the LUN, then on one of them I created the cinder-volumes VG (the other nodes can see the modifications), then I configured Cinder with lvm backend using this VG and it worked. I created some volumes on it without issues using my account. But what about when there are multiple tenants that try to create multiple volumes on it, is this configuration safe? Regards.

3 2

[cinder] - Test cases or procedure for active/active support validation
by Roquesalane, Jean-Pierre 30 May '23

30 May '23

Hi community! Here at Dell, we're in the process of certifying our Dell PowerMax storage platform against cinder volume Active/active configuration. I was wondering how vendors who already support this functionality have managed to handle this and if they can share their testing scenario. Thank you. JP Jean-Pierre Roquesalane SW System Sr Principal Engineer - OpenStack Champion Dell Technologies | Security and Ecosystem Software Mobile +33 6 21 86 79 04<tel:1-613-314-8106> Email : jeanpierre.roquesalane(a)dell.com<mailto:jeanpierre.roquesalane@dell.com> [cid:image001.png@01D976C9.D5B91180] Internal Use - Confidential Dell S.A, Si?ge Social 1 rond point Benjamin Franklin 34000 Montpellier. Capital 1,782,769 Euros, 351 528 229 RCS Montpellier -APE 4651Z -TVA Intracommunautaire FR 20 351 528 229, SIRET 351 528 229 00088 Vat Number : FR 20351528229 (France) / IT00001709997 (Italy) / ESN0012622G (Spain)

2 1

[kolla-ansible][yoga] Does glance support cinder-nfs as backend?
by wodel youchi 30 May '23

30 May '23

Hi, The openstak documentation says that glance supports cinder as backend, but it does not exclude any backend used by cinder itself. I'm having trouble configuring glance to use a cinder backend which is backed by an nfs share. Is this configuration supported? First, the rootwrap was missing, after adding it, I faced the lack of privileges, which was corrected by starting the glance-api container in privileged mode and finally I am facing a non existing filter error. Glance is trying to mount the nfs share to use it!!! Which I don't understand , why mount a share that is already mounted by cinder which glance is supposed to use as an intermediary!!!? When I push an image I get this error: Stderr: '/var/lib/kolla/venv/bin/glance-rootwrap: Unauthorized command: mount -t nfs 20.1.0.32:/kolla_nfs /var/lib/glance/mnt/nfs/f6f6b4ee42b4f3522a75f422887010ad2c47f8624 f97bf3623b13014f22186b7 (no filter matched)\n' Regards.

3 3

[kolla-ansible][yoga] Glance backend cinder Privsep daemon failed to start operation not permitted
by wodel youchi 30 May '23

30 May '23

Hi, I am trying to configure glance to use cinder as a backend. This is my glance-api.conf [cinder] cinder_store_auth_address = https://dashint.example.com:5000/v3 cinder_store_user_name = cinder cinder_store_password = cinderpass cinder_store_project_name = service cinder_volume_type = nfstype rootwrap_config = /etc/glance/rootwrap.conf ==> /var/log/kolla/glance/glance-api.log <== > 2023-04-12 18:02:20.842 64 INFO oslo.privsep.daemon > [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 > b0f76b5c6dcb457fa716762bbf954837 - default default] Spawned new privsep > daemon via rootwrap > 2023-04-12 18:02:20.733 360 INFO oslo.privsep.daemon [-] privsep daemon > starting > 2023-04-12 18:02:20.735 360 INFO oslo.privsep.daemon [-] privsep process > running with uid/gid: 0/0 > > *2023-04-12 18:02:20.737 360 ERROR oslo.privsep.daemon [-] [Errno 1] > Operation not permitted Traceback (most recent call last): * > File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", > line 584, in helper_main > Daemon(channel, context).run() > File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", > line 394, in run > self._drop_privs() > File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", > line 430, in _drop_privs > capabilities.drop_all_caps_except(self.caps, self.caps, []) > File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/capabilities.py", > line 156, in drop_all_caps_except > raise OSError(errno, os.strerror(errno)) > PermissionError: [Errno 1] Operation not permitted > 2023-04-12 18:02:20.844 64 WARNING oslo_privsep.comm > [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 > b0f76b5c6dcb457fa716762bbf954837 - default > default] Unexpected error: <class 'BrokenPipeError'>: BrokenPipeError: > [Errno 32] Broken pipe > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon > [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 > b0f76b5c6dcb457fa716762bbf954837 - default > default] Error while sending initial PING to privsep: [Errno 32] Broken > pipe: BrokenPipeError: [Errno 32] Broken pipe > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon Traceback (most > recent call last): > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", > line 192, in exchange_ping > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon reply = > self.send_recv((comm.Message.PING.value,)) > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/comm.py", > line 186, in send_recv > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon > self.writer.send((myid, msg)) > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/comm.py", > line 60, in send > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon > self.writesock.sendall(buf) > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon File > "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/greenio/base.py", > line 407, in sendall > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon tail = > self.send(data, flags) > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon File > "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/greenio/base.py", > line 401, in send > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon return > self._send_loop(self.fd.send, data, flags) > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon File > "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/greenio/base.py", > line 388, in _send_loop > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon return > send_method(data, *args) > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon BrokenPipeError: > [Errno 32] Broken pipe > 2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon > 2023-04-12 18:02:20.846 64 CRITICAL oslo.privsep.daemon > [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 > b0f76b5c6dcb457fa716762bbf954837 - defau > lt default] Privsep daemon failed to start > 2023-04-12 18:02:20.847 64 ERROR glance_store._drivers.cinder > [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 > b0f76b5c6dcb457fa716762bbf954837 - > default default] Failed to write to volume > 46316c12-6c24-40af-afde-1c16edd616b6.: > oslo_privsep.daemon.FailedToDropPrivileges: Privsep daemon failed to start > 2023-04-12 18:02:20.890 64 ERROR glance.api.v2.image_data > [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 > b0f76b5c6dcb457fa716762bbf954837 - def > ault default] *Failed to upload image data due to internal error: > oslo_privsep.daemon.FailedToDropPrivileges: Privsep daemon failed to start * > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi > [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 > b0f76b5c6dcb457fa716762bbf954837 - default d > efault] Caught error: Privsep daemon failed to start: > oslo_privsep.daemon.FailedToDropPrivileges: Privsep daemon failed to start > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi Traceback (most recent > call last): > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/common/wsgi.py", > line 1332, in __call__ > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi request, > **action_args) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/common/wsgi.py", > line 1370, in dispatch > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi return > method(*args, **kwargs) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/common/utils.py", > line 414, in wrapped > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi return func(self, > req, *args, **kwargs) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/api/v2/image_data.py", > line 303, in upload > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi > self._restore(image_repo, image) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 227, in __exit__ > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi > self.force_reraise() > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 200, in force_reraise > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi raise self.value > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/api/v2/image_data.py", > line 163, in upload > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi > image.set_data(data, size, backend=backend) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/notifier.py", line > 497, in set_data > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi > _send_notification(notify_error, 'image.upload', msg) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 227, in __exit__ > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi > self.force_reraise() > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > line 200, in force_reraise > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi raise self.value > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/notifier.py", line > 444, in set_data > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi > set_active=set_active) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/quota/__init__.py", > line 323, in set_data > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi > set_active=set_active) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/location.py", line > 585, in set_data > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi > self._upload_to_store(data, verifier, backend, size) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/location.py", line > 485, in _upload_to_store > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi verifier=verifier) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance_store/multi_backend.py", > line 399, in add_with_multihash > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi image_id, data, > size, hashing_algo, store, context, verifier) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance_store/multi_backend.py", > line 481, in store_add_to_backe > nd_with_multihash > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi image_id, data, > size, hashing_algo, context=context, verifier=verifier) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance_store/driver.py", > line 279, in add_adapter > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi metadata_dict) = > store_add_fun(*args, **kwargs) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance_store/capabilities.py", > line 176, in op_checker > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi return > store_op_fun(store, *args, **kwargs) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance_store/_drivers/cinder.py", > line 985, in add > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi with > self._open_cinder_volume(client, volume, 'wb') as f: > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/usr/lib64/python3.6/contextlib.py", line 81, in __enter__ > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi return > next(self.gen) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/glance_store/_drivers/cinder.py", > line 739, in _open_cinder_vol > ume > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi root_helper, > my_ip, use_multipath, enforce_multipath, host=host) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/os_brick/utils.py", line > 169, in trace_logging_wrapper > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi return f(*args, > **kwargs) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/os_brick/initiator/connector.py", > line 240, in get_connector_pr > operties > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi execute=execute)) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", > line 70, in get_connec > tor_properties > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi initiator = > iscsi.get_initiator() > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", > line 963, in get_initi > ator > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi > root_helper=self._root_helper) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/os_brick/executor.py", > line 53, in _execute > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi result = > self.__execute(*args, **kwargs) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/os_brick/privileged/rootwrap.py", > line 172, in execute > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi return > execute_root(*cmd, **kwargs) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/priv_context.py", > line 269, in _wrap > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi self.start() > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/priv_context.py", > line 283, in start > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi channel = > daemon.RootwrapClientChannel(context=self) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", > line 374, in __init__ > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi > super(RootwrapClientChannel, self).__init__(sock, context) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", > line 187, in __init__ > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi > self.exchange_ping() > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", > line 201, in exchange_ping > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi raise > FailedToDropPrivileges(msg) > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi *oslo_privsep.daemon.FailedToDropPrivileges: > Privsep daemon failed to start * > 2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi > 2023-04-12 18:02:20.927 64 INFO eventlet.wsgi.server > [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 > b0f76b5c6dcb457fa716762bbf954837 - default > default] 20.3.0.34,127.0.0.1 - - [12/Apr/2023 18:02:20] "PUT > /v2/images/52ed7ed7-330e-4249-abb9-5ec99712846f/file HTTP/1.1" 500 430 > 2.727683 > It seems like a lack of privileges, any ideas? Regards.

4 11

[neutron] Deprecate networking-odl project
by Rodolfo Alonso Hernandez 23 May '23

23 May '23

Hello all: During the last releases, the support for "networking-odl" project has decreased. Currently there is no active developer or maintainer in the community. This project depends on https://www.opendaylight.org/; the latest version released is Sulfur (16) while the version still used in the CI is Sodium (11) [1]. I would like first to make a call for developers to update this project. But if this is not possible, I will then start the procedure to deprecate it [2] (**not to retire it**). Regards. [1] https://github.com/openstack/networking-odl/blob/db5c79b3ee5054feb8a17df130… [2] https://docs.openstack.org/project-team-guide/repository.html#deprecating-a…

3 3

Re: 答复: [ptl] Need PTL volunteer for OpenStack Sahara
by Ghanshyam Mann 17 May '23

17 May '23

Thanks Jerry, Brin for the updates and showing interest in Sahara project. In PTG discussion, there were some concern on Sahara maintenance even we had PTL volunteer to lead this project but not much things got merged and gate was broken many times. But let me discuss it in TC meeting tomorrow and update here. -gmann ---- On Mon, 10 Apr 2023 00:22:08 -0700 Brin Zhang(张百林) wrote --- > Hi Gmann, > Jerry Zhou works the same company as qiujunting (Qiujunting has out of the office, so he cannot response us, I am so sorry), and he will be involved in Sahara. > Hope TC can add him in Sahara team, let him gradually complete the maintenance of the Sahara project. > Thanks. > > brinzhang > > -----邮件原件----- > 发件人: Jerry Zhou (周鑫勇) zhouxinyong(a)inspur.com> > 发送时间: 2023年4月7日 11:10 > 收件人: gmann(a)ghanshyammann.com; Juntingqiu Qiujunting (邱军婷) qiujunting(a)inspur.com> > 抄送: openstack-discuss(a)lists.openstack.org; Tom Guo(郭涛2810) guotao.bj(a)inspur.com>; Brin Zhang(张百林) zhangbailin(a)inspur.com>; Weiting Kong (孔维亭) kongweiting(a)inspur.com> > 主题: 答复: [ptl] Need PTL volunteer for OpenStack Sahara > > Hi Gmann, > I can lead this project. > > -----邮件原件----- > 发件人: Ghanshyam Mann gmann(a)ghanshyammann.com> > 发送时间: 2023年3月23日 2:35 > 收件人: Qiujunting qiujunting(a)inspur.com>; openstack-discuss openstack-discuss(a)lists.openstack.org> > 主题: [ptl] Need PTL volunteer for OpenStack Sahara > > Hi Qiu, > > I am reaching out to you as you were PTL for OpenStack Sahara project in the last cycle. > > There is no PTL candidate for the next cycle (2023.2), and it is on the leaderless project list. Please check if you or anyone you know would like to lead this project. > > - https://etherpad.opendev.org/p/2023.2-leaderless > > Also, if anyone else would like to help leading this project, this is time to let TC knows. > > -gmann > >

4 7

[heat][magnum][tacker] Future of SoftwareDeployment support
by Takashi Kajinami 09 May '23

09 May '23

Hello, We discussed this briefly in the past thread where we discussed maintenance of os-*-agent repos, and also talked about this topic during Heat PTG, but I'd like to formalize the discussion to get a clear agreement. Heat has been supporting SoftwareDeployment resources to configure software in instances using some agents such as os-collect-config[1]. [1] https://docs.openstack.org/heat/latest/template_guide/software_deployment.h… This feature was initially developed to be used by TripleO (IIUC), but TripleO is retired now and we are losing the first motivation to maintain the feature. # Even TripleO replaced most of its usage of softwaredeployment by config-download lately. Because the heat project team has drunk dramatically recently, we'd like to put more focus on core features. For that aim we are now wondering if we can deprecate and remove this feature, and would like to hear from anyone who has any concerns about this. Quickly looking through the repos, it seems currently Magnum and Tacker are using SoftwareDeployment, and it'd be nice especially if we can understand their current requirements. 1. Magnum It seems SoftwareDeployment is used by k8s_fedora_atomic_v1 driver but I'm not too sure whether this driver is still supported, because Fedora Atomic was EOLed a while ago, right ? 2. Tacker SoftwareDeployment can be found in only test code in the tacker repo. We have some references kept in heat-translator which look related to TOSCA templates. Thank you, Takashi Kajinami

4 8

[all][tc][release] Hold the Python3.8 dropping
by Ghanshyam Mann 08 May '23

08 May '23

Hello Everyone, As you know the gate is breaking due to Python3.8 drop[1][2], which we discussed in the TC channel and meeting[3]. TC is preparing the guidelines on not dropping Python3.8 in 2023.2 pti. Meanwhile, governance guidelines are prepared, this email is a heads-up to hold the dropping of Python3.8 from your project/repo. Do not bump the min py version in setup.cfg (do not bump version in python_requires ). @Release team, It will be helpful if you can check/hold the release of any project/lib dropping the Python3.8. [1] https://lists.openstack.org/pipermail/openstack-discuss/2023-April/033450.h… [2] https://lists.openstack.org/pipermail/openstack-discuss/2023-April/033454.h… [3] https://meetings.opendev.org/meetings/tc/2023/tc.2023-04-25-18.00.log.html#… -gmann

1 1

[openstack-ansible] Proposing Neil Hanlon for core reviewer
by Dmitriy Rabotyagov 04 May '23

04 May '23

Hi everyone, I'm pleased to propose Neil Hanlon as OpenStack-Ansible Core Reviewer. Neil has helped us a lot lately with maintenance of RHEL-like distros - both CentOS Stream and Rocky Linux, and basically has brought in support for the last one. Neil is present at meetings and always is responsive in IRC. At the same time they were providing useful reviews lately [1] If there are no objections, I will add Neil to the team on 25th of April 2023. Until then, please feel free to provide your feedback/opinions on the matter. [1] https://www.stackalytics.io/report/contribution?module=openstackansible-gro…

6 7

[all][tc][vitrage] Final call for Vitrage project maintainers
by Ghanshyam Mann 03 May '23

03 May '23

Hello Everyone, You might have noticed that the Vitrage project is leaderless in this cycle[1]. Eyal, who was the previous PTL and the only maintainer of this project for more than two years[2], will not be able to continue in this project[3]. We thank and appreciate all previous maintainers' work on this project. This is the final call for maintainers. If anyone is using or interested in this project, this is the right time to step up and let us know in this email or IRC(OFTC) #openstack-tc channel. If there is no response by the end of April 2023, we will discuss the retirement of this project. [1] https://etherpad.opendev.org/p/2023.2-leaderless [2] https://www.stackalytics.io/?module=vitrage-group&release=wallaby [3] https://lists.openstack.org/pipermail/openstack-discuss/2023-April/033333.h… -gmann

2 2

[OpenvSwitch][Neutron] native flow based firewall Vs LinuxBridge Iptables firewall
by Satish Patel 02 May '23

02 May '23

Folks, As we know, openvswitch uses a linuxbridge based firewall to implement security-groups on openstack. It works great but it has so many packet hops. It also makes troubleshooting a little complicated. OpenvSwitch does support native firewall features in flows, Does it mature enough to implement in production and replace it with LinuxBridge based IPtables firewall? ~S

3 3

[horizon][nova][masakari] Instances created with "Any AZ" problem
by Nguyễn Hữu Khôi 02 May '23

02 May '23

Hello guys. I playing with Nova AZ and Masakari https://docs.openstack.org/nova/latest/admin/availability-zones.html Masakari will move server by nova scheduler. Openstack Docs describe that: If the server was not created in a specific zone then it is free to be moved to other zones, i.e. the AvailabilityZoneFilter <https://docs.openstack.org/nova/latest/admin/scheduling.html#availabilityzo…> is a no-op. I see that everyone usually creates instances with "Any Availability Zone" on Horzion and also we don't specify AZ when creating instances by cli. By this way, when we use Masakari or we miragrated instances( or evacuate) so our instance will be moved to other zones. Can we attach AZ to server create requests API based on Any Availability Zone to limit instances moved to other zones? Thank you. Regards Nguyen Huu Khoi

5 20

[kolla-ansible][all][nova][zed][latest] USB passthrough / Hot-plug / Cold-plug
by Modyngs 02 May '23

02 May '23

090c090cFolks, A USB device (not necessarily a mass/Flash drive) needs to be connected to one of the VMs (an openstack instance). Openstack installed with kolla-ansible{Latest version/ Also tested on Zed} ALL IN ONE Deployments on ubuntu server 22.04 ( Core i9 12900K). I found the [nova-libvirt] container which contains *virsh *and is able to edit it or use custom config for VMs. I've gone through lots of the docs, to name a few: https://libvirt.org/formatdomain.html https://wiki.openstack.org/wiki/Nova/USB_device_hot_cold_plug https://wiki.openstack.org/wiki/Nova/proposal_about_usb_passthrough https://documentation.suse.com/sles/15-SP1/html/SLES-all/cha-libvirt-config… https://wiki.openstack.org/wiki/Nova/USB_device_hot_cold_plug https://docs.nxp.com/bundle/GUID-487B2E69-BB19-42CB-AC38-7EF18C0FE3AE/page/… but none of them worked for me!! To reproduce: *Cold-Plug :* *$ lsusb (on host)* Bus 001 Device 014: ID 090c:1000 Silicon Motion >>>> Note Device number changed every time I disconnect the device. So it might be different in the changed attempt shown below) *$ docker exec -it nova_libvirt /bin/bash* *%Turn the Desired VM off* *# virsh list --all* Id Name State --------------------------------------------- 2 instance-00000002 running 19 instance-00000008 running - instance-0000000a shut off *# virsh edit instance-0000000a* Add the changes [1][2][3][4],... ( many efforts have been done but few samples of them are) [1]: under <devices> added <hostdev mode='subsystem' type='usb'> <source> <vendor id='0x090c'/> <product id='0x1000'/> </source> </hostdev> [2]:under <devices> added <controller type='usb' index='0'/> <hostdev mode='subsystem' type='usb'> <source> <vendor id='0x090c'/> <product id='0x1000'/> </source> </hostdev> [3]: under <devices> added <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x090c'/> <product id='0x1000'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </hostdev> [4]: under <devices> added <controller type='usb' model='nec-xhci' index='0'/> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x090c'/> <product id='0x1000'/> </source> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </hostdev> [5]: <controller type='usb' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x090c'/> <product id='0x1000'/> </source> <address type='usb' bus='1' port='2'/> <controller type='usb' index='0'/> </hostdev> *%Start the VM* *expected behavior:* when login to the VM, lsusb or df -h shows the USB *what happened:* it wont show the USB from the VM *OR *virsh dumpxml instance-0000000a > instance-0000000a.xml and then change the configs as above and then virsh attach-device instance-0000000a --file /path/to/updated-instance-0000000a.xml --config *Hot-Plug :* *$ lsusb (on host)* Bus 001 Device 014: ID 090c:1000 Silicon Motion >>>> Note Device number changed every time I disconnect the device. So it might be different in the changed attempt shown below) *$ docker exec -it nova_libvirt /bin/bash* *# virsh list --all* Id Name State --------------------------------------------- 2 instance-00000002 running 19 instance-00000008 running 20 instance-0000000a running *#nano USB.xml* *%add changes explained in *[1][2][3][4],... *$ virsh attach-device instance-0000000a /path/to/USB.xml/file* *expected behavior:* lsusb or df -h shows the USB *what happened:* it wont show the USB from the VM *Can you please guide me through this? Any recommendation would be much appreciated!Any custom changes comes to your mind ( Reply it) would be solution for this problem /;* Thanks Best regards

2 1

Offline Openstack Release:Train Deployment
by Vineet Thakur 01 May '23

01 May '23

Greetings openstack community members, It's regarding the deployment of openstack release: train on CentoOS 7.9 environment by using openstack-ansible tool. (Offline environment) We are facing challenges to download the packages, some package links are not accessible or somewhere PIP packages are not available as per the constraint requirement list. See: https://artifacts.ci.centos.org/sig-cloudinstance/centos-7-191001/x86-64/ce… For pip packages: https://opendev.org/openstack/requirements/raw/0cfc2ef6318b639bb4287fa051ea… During installation we are facing a lot of errors, and not sure what would be the best approach in such scenarios where packages are not available or need to be downloaded from random sites. We are in the mid of a mega deployment and appreciate it if you can guide us through. Awaiting your response, thank you. Kind Regards, Vineet Thakur

3 3

[kolla-ansible][zed][15.1.0] Horizon failed to load trunk and QoS
by Stefan Bryzga 01 May '23

01 May '23

Hi all, Some time ago I deployed a small cluster with kolla-ansible v15.1.0. Before deployment I enabled neutron_qos and neutron_trunk flags in /etc/kolla/globals.yml. Both functions work fine via cli and I can create new entries via dashboard however horizon won't display created trunks/qos on the page (It shows "No items to display"). In web browser console I get this error while loading page: TypeError: Cannot read properties of undefined (reading 'data') at addTrackBy (output.0a9483334bbd.js:1115:59) at processQueue (output.43a09e34a303.js:1581:786) at output.43a09e34a303.js:1585:110 at Scope.$digest (output.43a09e34a303.js:1636:200) at Scope.$apply (output.43a09e34a303.js:1643:269) at done (output.43a09e34a303.js:1356:123) at completeRequest (output.43a09e34a303.js:1375:20) at XMLHttpRequest.requestLoaded (output.43a09e34a303.js:1368:1) 'Possibly unhandled rejection: {}' Does anyone knows how to fix this? Should I modify any configs to display new entries in horizon? Best Regards, Stefan

2 1

[neutron] Neutron CI not working
by Rodolfo Alonso Hernandez 28 Apr '23

28 Apr '23

Hello Neutrinos: There are some problems in a set of nodes that we are using for tempest tests. This is also affecting other projects like Octavia. Please follow [1] for more information. Yatin has collected information of the error and the affected nodes. Actions: * Temporarily disable these nodes from the pool [2]. That will allow us (Neutron) to continue the CI testing. * Send a support ticket to the node provider. Once we have more information, we'll provide it. Before [2] is merged, please hold your rechecks for the tempest jobs in Neutron. Regards. [1]https://bugs.launchpad.net/neutron/+bug/2017992 [2]https://review.opendev.org/c/openstack/project-config/+/881810

1 1

[neutron] CI meeting on May 2nd cancelled
by Slawek Kaplonski 28 Apr '23

28 Apr '23

Hi, I will be on PTO next Tuesday so lets cancel CI meeting this time. See You on the meeting on May 9th. -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

[tripleo] Nominate dbengt for tripleo-core !
by Lukas Bezdicka 28 Apr '23

28 Apr '23

Hello, I'd like to nominate Daniel Bengtsson as tripleo-core. He works in upgrades and he'll have to maintain train, wallaby and zed for quite a while. He is a long term contributor to tripleo and oslo. I trust his judgement and I hope you'll support his nomination! Thanks, Lukas Bezdicka

6 6

[CloudKitty] Cancelling weekly meeting 1st May
by Rafael Weingärtner 28 Apr '23

28 Apr '23

Hello guys, May 1st is a public holiday for most of the core reviewers. Therefore, we will be canceling that meeting. If you need any assistance until the next meeting, you can reach me via e-mail. Best regards, -- Rafael Weingärtner

1 0

[kolla] Cancelling weekly meeting 3rd May
by Michał Nasiadka 28 Apr '23

28 Apr '23

Hello Koalas, Since 3rd of May is public holiday in Poland and other cores are on vacation - cancelling weekly meeting. Let’s meet again on 10th May. Best regards, Michal

1 0

[magnum] Propose deprecation of Docker Swarm, CoreOS, fedora-ironic
by Jake Yip 28 Apr '23

28 Apr '23

Hi, Due to limited resources, Magnum team has decided to put most effort into supporting the most popular use case - Kubernetes on Fedora CoreOS. In light of that, we are seeking maintainers for the less well maintained drivers / COE - Docker Swarm, CoreOS and fedora-ironic drivers. The reasons are as follows: - Docker Swarm - It uses Fedora Atomic which has been EOL [1] - CoreOS - It is EOL [2] - fedora-ironic - Not tested, unsure if it is still working If anyone wants to take up maintenance of any of the above, we will be glad. Please let us know either through the ML or at the Magnum meetings. If there are no suitable maintainers, we will start the deprecation of these before end of Antelope. Regards, Jake [1]: https://projectatomic.io/blog/2019/11/fedora-atomic-host-nearing-eol/ [2]: https://www.redhat.com/en/technologies/cloud-computing/openshift/what-was-c… -- Jake Yip Magnum Team

2 3

[neutron] Neutron drivers meeting cancelled
by Rodolfo Alonso Hernandez 28 Apr '23

28 Apr '23

Hello Neutrinos: Due to the lack of agenda, today's drivers meeting is cancelled. Have a nice weekend!

1 0

Re: openstack-discuss Digest, Vol 54, Issue 88
by BEDDA Fadhel 28 Apr '23

28 Apr '23

Good morning, I am still looking for a complete and clear procedure to set up an openstack solution on a vmware worksation virtual machine: ubuntu or centos. THANKS Le lun. 17 avr. 2023 à 17:30, <openstack-discuss-request(a)lists.openstack.org> a écrit : > Send openstack-discuss mailing list submissions to > openstack-discuss(a)lists.openstack.org > > To subscribe or unsubscribe via the World Wide Web, visit > > https://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss > > or, via email, send a message with subject or body 'help' to > openstack-discuss-request(a)lists.openstack.org > > You can reach the person managing the list at > openstack-discuss-owner(a)lists.openstack.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of openstack-discuss digest..." > > > Today's Topics: > > 1. Re: [Kolla-ansible][Yoga] Cannot live migrate VMs (wodel youchi) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 11 Apr 2023 12:57:47 +0100 > From: wodel youchi <wodel.youchi(a)gmail.com> > To: Pierre Riteau <pierre(a)stackhpc.com> > Cc: OpenStack Discuss <openstack-discuss(a)lists.openstack.org> > Subject: Re: [Kolla-ansible][Yoga] Cannot live migrate VMs > Message-ID: > <CAJV_UBDH4tJ11pkOagYpWNVCrZHruTmA6A-k9P= > DLrqchPvNPg(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi, > > Indeed, there was a mistake in the cinder.conf file. > Thanks. > > Regards. > > < > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai… > > > Virus-free.www.avast.com > < > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campai… > > > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> > > Le mar. 11 avr. 2023 ? 10:45, Pierre Riteau <pierre(a)stackhpc.com> a ?crit > : > > > You should check the status of your Cinder services. > > > > On Sun, 9 Apr 2023 at 19:43, wodel youchi <wodel.youchi(a)gmail.com> > wrote: > > > >> Hi, > >> > >> I cannot live-migrate any VM I have. > >> All my compute nodes are enabled. I have a few VMs, I am in the testing > >> phase. > >> I tried to live-migrate toward an empty node, then toward a node which > >> already had a VM in it. It is the same problem. > >> > >> In the log files I have : > >> > >> Source node : > >> Source > >> > >>> ==> /var/log/kolla/nova/nova-compute.log <== > >>> 2023-04-09 11:32:49.419 7 INFO nova.compute.resource_tracker > >>> [req-4e91d562-d7ad-4585-a827-3db6277fd81e - - - - -] [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Updating resource usage from > >>> migration f60eb077-8e1c-404a-aca5-64e5609ca02c > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [-] [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Pre live migration failed at > >>> computehci02: oslo_messaging.rpc.client.RemoteError: Remote error: > >>> ClientException Gateway Time-out (HTTP 504) > >>> ['Traceback (most recent call last):\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", > >>> line 165, in _process_incoming\n res = > >>> self.dispatcher.dispatch(message)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", > >>> line 309, in dispatch\n return self._do_dispatch(endpoint, method, > ctxt, > >>> args)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", > >>> line 229, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' > File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/exception_wrapper.py", > >>> line 72, in wrapped\n context, exc, binary)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 227, in __exit__\n self.force_reraise()\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 200, in force_reraise\n raise self.value\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/exception_wrapper.py", > >>> line 63, in wrapped\n return f(self, context, *args, **kw)\n', ' > File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/utils.py", > >>> line 1439, in decorated_function\n return function(self, context, > *args, > >>> **kwargs)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > >>> line 213, in decorated_function\n kwargs[\'instance\'], e, > >>> sys.exc_info())\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 227, in __exit__\n self.force_reraise()\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 200, in force_reraise\n raise self.value\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > >>> line 201, in decorated_function\n return function(self, context, > *args, > >>> **kwargs)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > >>> line 8367, in pre_live_migration\n bdm.save()\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 227, in __exit__\n self.force_reraise()\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 200, in force_reraise\n raise self.value\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > >>> line 8304, in pre_live_migration\n connector=connector, > >>> mountpoint=bdm.device_name)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > >>> line 395, in wrapper\n res = method(self, ctx, *args, **kwargs)\n', > ' > >>> File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > >>> line 433, in wrapper\n res = method(self, ctx, volume_id, *args, > >>> **kwargs)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > >>> line 813, in attachment_create\n instance_uuid=instance_id)\n', ' > File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 227, in __exit__\n self.force_reraise()\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 200, in force_reraise\n raise self.value\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > >>> line 801, in attachment_create\n volume_id, _connector, > instance_id)\n', > >>> ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/api_versions.py", > >>> line 421, in substitution\n return method.func(obj, *args, > **kwargs)\n', > >>> ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/v3/attachments.py", > >>> line 39, in create\n retval = self._create(\'/attachments\', body, > >>> \'attachment\')\n', ' File > >>> "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/base.py", > >>> line 306, in _create\n resp, body = self.api.client.post(url, > >>> body=body)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > >>> line 227, in post\n return self._cs_request(url, \'POST\', > **kwargs)\n', > >>> ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > >>> line 215, in _cs_request\n return self.request(url, method, > >>> **kwargs)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > >>> line 201, in request\n raise exceptions.from_response(resp, > body)\n', > >>> 'cinderclient.exceptions.ClientException: Gateway Time-out (HTTP > 504)\n']. > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Traceback (most recent call > last): > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > >>> line 8501, in _do_pre_live_migration_from_source > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] block_migration, disk, dest, > >>> migrate_data) > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/rpcapi.py", > >>> line 948, in pre_live_migration > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] **msg_args) > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/client.py", > >>> line 192, in call > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] retry=self.retry, > >>> transport_options=self.transport_options) > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/transport.py", > >>> line 128, in _send > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] > >>> transport_options=transport_options) > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", > >>> line 691, in send > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] > >>> transport_options=transport_options) > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/_drivers/amqpdriver.py", > >>> line 681, in _send > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] raise result > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] > >>> oslo_messaging.rpc.client.RemoteError: Remote error: ClientException > >>> Gateway Time-out (HTTP 504) > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] ['Traceback (most recent call > >>> last):\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", > >>> line 165, in _process_incoming\n res = > >>> self.dispatcher.dispatch(message)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", > >>> line 309, in dispatch\n return self._do_dispatch(endpoint, method, > ctxt, > >>> args)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", > >>> line 229, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' > File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/exception_wrapper.py", > >>> line 72, in wrapped\n context, exc, binary)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 227, in __exit__\n self.force_reraise()\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 200, in force_reraise\n raise self.value\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/exception_wrapper.py", > >>> line 63, in wrapped\n return f(self, context, *args, **kw)\n', ' > File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/utils.py", > >>> line 1439, in decorated_function\n return function(self, context, > *args, > >>> **kwargs)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > >>> line 213, in decorated_function\n kwargs[\'instance\'], e, > >>> sys.exc_info())\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 227, in __exit__\n self.force_reraise()\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 200, in force_reraise\n raise self.value\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > >>> line 201, in decorated_function\n return function(self, context, > *args, > >>> **kwargs)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > >>> line 8367, in pre_live_migration\n bdm.save()\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 227, in __exit__\n self.force_reraise()\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 200, in force_reraise\n raise self.value\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > >>> line 8304, in pre_live_migration\n connector=connector, > >>> mountpoint=bdm.device_name)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > >>> line 395, in wrapper\n res = method(self, ctx, *args, **kwargs)\n', > ' > >>> File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > >>> line 433, in wrapper\n res = method(self, ctx, volume_id, *args, > >>> **kwargs)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > >>> line 813, in attachment_create\n instance_uuid=instance_id)\n', ' > File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 227, in __exit__\n self.force_reraise()\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 200, in force_reraise\n raise self.value\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > >>> line 801, in attachment_create\n volume_id, _connector, > instance_id)\n', > >>> ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/api_versions.py", > >>> line 421, in substitution\n return method.func(obj, *args, > **kwargs)\n', > >>> ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/v3/attachments.py", > >>> line 39, in create\n retval = self._create(\'/attachments\', body, > >>> \'attachment\')\n', ' File > >>> "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/base.py", > >>> line 306, in _create\n resp, body = self.api.client.post(url, > >>> body=body)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > >>> line 227, in post\n return self._cs_request(url, \'POST\', > **kwargs)\n', > >>> ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > >>> line 215, in _cs_request\n return self.request(url, method, > >>> **kwargs)\n', ' File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > >>> line 201, in request\n raise exceptions.from_response(resp, > body)\n', > >>> 'cinderclient.exceptions.ClientException: Gateway Time-out (HTTP > 504)\n']. > >>> 2023-04-09 11:33:30.118 7 ERROR nova.compute.manager [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] > >>> 2023-04-09 11:33:30.157 7 INFO nova.compute.manager [-] [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Swapping old allocation on > >>> dict_keys(['2857ae62-3519-4b13-b700-858c827dd78d']) held by migration > >>> f60eb077-8e1c-404a-aca5-64e5609ca02c for instance > >> > >> > >> > >> In the destination node : > >> > >>> nova-compute.log > >>> > >>> > >>> > >>> > >>> *2023-04-09 11:32:30.052 7 WARNING os_brick.initiator.connectors.nvmeof > >>> [req-69822759-d982-4d01-b448-232796077002 > 0439953e7cfe4a13a1b4bb118b5dc3c4 > >>> b0f76b5c6dcb457fa716762bbf954837 - default default] Process execution > error > >>> in _get_host_uuid: Unexpected error while running command.Command: > blkid > >>> overlay -s UUID -o valueExit code: 2Stdout: ''Stderr: '': > >>> oslo_concurrency.processutils.ProcessExecutionError: Unexpected error > while > >>> running command.* > >>> > >>> 2023-04-09 11:33:05.354 7 INFO nova.compute.resource_tracker > >>> [req-9d480442-87f7-4d4e-a866-c46d645ac7f2 - - - - -] [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Updating resource usage from > >>> migration f60eb077-8e1c-404a-aca5-64e5609ca02c > >>> 2023-04-09 11:33:05.392 7 WARNING nova.compute.resource_tracker > >>> [req-9d480442-87f7-4d4e-a866-c46d645ac7f2 - - - - -] Instance > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7 has been moved to another host > >>> computehci01(computehci01). There are allocations remaining against the > >>> source host that might need to be removed: {'resources': {'VCPU': 1, > >>> 'MEMORY_MB': 2048}}. > >>> 2023-04-09 11:33:30.086 7 ERROR nova.volume.cinder > >>> [req-69822759-d982-4d01-b448-232796077002 > 0439953e7cfe4a13a1b4bb118b5dc3c4 > >>> b0f76b5c6dcb457fa716762bbf954837 - default default] [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Create attachment failed for > volume > >>> 3d7bd0af-5e4c-48fc-a027-f6c9dd6280bb. Error: Gateway Time-out (HTTP > 504) > >>> Code: 504: cinderclient.exceptions.ClientException: Gateway Time-out > (HTTP > >>> 504) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > >>> [req-69822759-d982-4d01-b448-232796077002 > 0439953e7cfe4a13a1b4bb118b5dc3c4 > >>> b0f76b5c6dcb457fa716762bbf954837 - default default] Exception during > >>> message handling: cinderclient.exceptions.ClientException: Gateway > Time-out > >>> (HTTP 504) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server Traceback > >>> (most recent call last): > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", > >>> line 165, in _process_incoming > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server res = > >>> self.dispatcher.dispatch(message) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", > >>> line 309, in dispatch > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server return > >>> self._do_dispatch(endpoint, method, ctxt, args) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", > >>> line 229, in _do_dispatch > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server result = > >>> func(ctxt, **new_args) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/exception_wrapper.py", > >>> line 72, in wrapped > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server context, > >>> exc, binary) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 227, in __exit__ > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > >>> self.force_reraise() > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 200, in force_reraise > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server raise > >>> self.value > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/exception_wrapper.py", > >>> line 63, in wrapped > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server return > >>> f(self, context, *args, **kw) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/utils.py", > >>> line 1439, in decorated_function > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server return > >>> function(self, context, *args, **kwargs) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > >>> line 213, in decorated_function > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > >>> kwargs['instance'], e, sys.exc_info()) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 227, in __exit__ > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > >>> self.force_reraise() > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 200, in force_reraise > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server raise > >>> self.value > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > >>> line 201, in decorated_function > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server return > >>> function(self, context, *args, **kwargs) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > >>> line 8367, in pre_live_migration > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > bdm.save() > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 227, in __exit__ > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > >>> self.force_reraise() > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 200, in force_reraise > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server raise > >>> self.value > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/compute/manager.py", > >>> line 8304, in pre_live_migration > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > >>> connector=connector, mountpoint=bdm.device_name) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > >>> line 395, in wrapper > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server res = > >>> method(self, ctx, *args, **kwargs) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > >>> line 433, in wrapper > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server res = > >>> method(self, ctx, volume_id, *args, **kwargs) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > >>> line 813, in attachment_create > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > >>> instance_uuid=instance_id) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 227, in __exit__ > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > >>> self.force_reraise() > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", > >>> line 200, in force_reraise > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server raise > >>> self.value > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/nova/volume/cinder.py", > >>> line 801, in attachment_create > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > volume_id, > >>> _connector, instance_id) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/api_versions.py", > >>> line 421, in substitution > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server return > >>> method.func(obj, *args, **kwargs) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/v3/attachments.py", > >>> line 39, in create > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server retval = > >>> self._create('/attachments', body, 'attachment') > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/base.py", > >>> line 306, in _create > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server resp, > body > >>> = self.api.client.post(url, body=body) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > >>> line 227, in post > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server return > >>> self._cs_request(url, 'POST', **kwargs) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > >>> line 215, in _cs_request > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server return > >>> self.request(url, method, **kwargs) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server File > >>> > "/var/lib/kolla/venv/lib/python3.6/site-packages/cinderclient/client.py", > >>> line 201, in request > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server raise > >>> exceptions.from_response(resp, body) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > >>> cinderclient.exceptions.ClientException: Gateway Time-out (HTTP 504) > >>> 2023-04-09 11:33:30.115 7 ERROR oslo_messaging.rpc.server > >>> 2023-04-09 11:33:30.238 7 INFO nova.virt.block_device > >>> [req-69822759-d982-4d01-b448-232796077002 > 0439953e7cfe4a13a1b4bb118b5dc3c4 > >>> b0f76b5c6dcb457fa716762bbf954837 - default default] [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Attempting to driver detach > volume > >>> 3d7bd0af-5e4c-48fc-a027-f6c9dd6280bb from mountpoint /dev/vda > >>> 2023-04-09 11:33:30.239 7 WARNING nova.virt.block_device > >>> [req-69822759-d982-4d01-b448-232796077002 > 0439953e7cfe4a13a1b4bb118b5dc3c4 > >>> b0f76b5c6dcb457fa716762bbf954837 - default default] [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Detaching volume from unknown > instance > >>> 2023-04-09 11:33:30.240 7 WARNING nova.virt.libvirt.driver > >>> [req-69822759-d982-4d01-b448-232796077002 > 0439953e7cfe4a13a1b4bb118b5dc3c4 > >>> b0f76b5c6dcb457fa716762bbf954837 - default default] [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] During detach_volume, instance > >>> disappeared.: nova.exception.InstanceNotFound: Instance > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7 could not be found. > >>> 2023-04-09 11:33:30.678 7 INFO nova.virt.libvirt.driver [-] [instance: > >>> f4c3ee82-d5cf-4b79-8608-76be7faeb2c7] Instance destroyed successfully. > >>> 2023-04-09 11:33:30.682 7 INFO os_vif > >>> [req-69822759-d982-4d01-b448-232796077002 > 0439953e7cfe4a13a1b4bb118b5dc3c4 > >>> b0f76b5c6dcb457fa716762bbf954837 - default default] Successfully > unplugged > >>> vif > >>> > VIFBridge(active=True,address=fa:16:3e:9b:92:46,bridge_name='qbr46651303-65',has_traffic_filtering=True,id=46651303-65e1-4809-95cc-7c2d97acf16b,network=Network(532470c1-dd2f-4505-8557-ba24e9d4062b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap46651303-65') > >>> > >> > >> All I can see is > >> > >> > >> > >> > >> *b0f76b5c6dcb457fa716762bbf954837 - default default] Process execution > >> error in _get_host_uuid: Unexpected error while running command.Command: > >> blkid overlay -s UUID -o valueExit code: 2Stdout: ''Stderr: '': > >> oslo_concurrency.processutils.ProcessExecutionError: Unexpected error > while > >> running command.* > >> > >> Searching the web I didn't find anything useful about this error, and I > >> don't know if it's the origin of the problem. > >> > >> My default storage is ceph rbd, I have an HCI deployment, and all my VMs > >> are for now created on Ceph only. But I have configured two other cinder > >> backends, NFS and LVM, but they are node used for now. > >> > >> Thanks in advance. > >> > >> Regards. > >> > > >

2 1

[release][cinder] please delete cinderlib stable/ussuri branch
by Brian Rosmaita 27 Apr '23

27 Apr '23

Hello Release Team, Some changes were bot-proposed recently to the cinderlib stable/ussuri branch, which was weird because that branch was tagged EOL by change I75092342fe684892f8a73eab9ad5da4515cb1da5. The change merged on 2022-08-31, and the ussuri-eol tag exists in the code repository, but apparently the branch was never deleted. Could you please delete the branch? thanks!

2 3

OpenStack Swift quota
by Xu, Wei [E CPE] 27 Apr '23

27 Apr '23

Hello OpenStack community, Right now, because we tend to limit each user's individual storage capacity, we're trying to implement account and container-level quotas on our OpenStack Swift cluster (Yoga version). Every time we tried to execute a command like "swift post -m quota-bytes: 500" using the Swift CLI client to implement account quotas, it returned a 403 error saying "403 Forbidden [first 60 chars of response] <html><h1>Forbidden</h1><p>Access was denied to this resourc", even if we are using an account already with the reseller/ResellerAdmin role, and both quotas added to the [pipeline:main] on our proxy server. Could you help us identify what could be behind this issue and how to resolve it? Also, it seems like default OpenStack Swift only supports account-level and container-level quota implementation. Is it possible to implement not only user-level but also project-level and domain-level quotas? I'm looking forward to hearing back from you. Thank you very much! Yours sincerely, Wei Xu, M.S. Research Assistant Department of ECpE, Iowa State University

3 9

[kolla-ansible][watcher]Time zone issue in container
by Stefan Bryzga 27 Apr '23

27 Apr '23

Hi all, Recently I created fresh deployment based on kolla-ansible yoga and debian 11. Everything works correctly except Watcher services. Containers are in constant restart loop. On Debian host OS time-related files are as follows: lrwxrwxrwx 1 root root 33 Mar 21 07:53 localtime -> /usr/share/zoneinfo/Europe/Warsaw -rw-r--r-- 1 root root 14 Mar 21 07:53 timezone x@control01:/etc$ cat timezone Europe/Warsaw However every container have: lrwxrwxrwx 1 root root 27 Apr 11 02:00 localtime -> /usr/share/zoneinfo/Etc/UTC -rw-r--r-- 1 root root 14 Mar 21 07:53 timezone (nova-api)[root@control01 etc]# cat timezone Europe/Warsaw In watcher logs I found error: 2023-04-27 10:08:12.607 7 ERROR python-watcher tzlocal.utils.ZoneInfoNotFoundError: 'Multiple conflicting time zone configurations found:\n/etc/timezone: Europe/Warsaw\n/etc/localtime is a symlink to: UTC\nFix the configuration, or set the time zone in a TZ environment variable.\n' 2023-04-27 10:08:12.607 7 ERROR python-watcher Which I believe is caused by wrong symlink for /etc/localtime in containers. Did anyone experience similar issue and know how to resolve it. Best regards, Stefan

2 2

[kolla-ansible] all-in-one deployment on single NIC
by Satish Patel 27 Apr '23

27 Apr '23

Folks, I am trying to create a lab with a single server which has a single NIC card and as you know kolla-ansible needs two nic for all-in-one (api and neutron nic). I want to test VLAN base provider for VMs so can I configure single nic on trunk port and create two VLAN interfaces foo.10 and foo.20 bridge and assign foo.10 to api and foo.20 for neutron. Does that make sense and will help the test lab? I was also reading somewhere that you need to create a veth interface two split single nic for kolla. Did anyone test kolla with single nic with two bridges to create dual nic simulation?

1 0

[neutron] [kolla-ansible] alembic mismatch, schema differences
by Gregory Orange 26 Apr '23

26 Apr '23

Hi all, We've caused ourselves some bother with our neutron database, and I found the following thread instructive, so I hope we can get some help to resolve the bother. https://lists.openstack.org/pipermail/openstack-discuss/2022-November/03131… We're running kolla-ansible train (yes, upgrades planned). We wanted to test something with a wallaby container, and while mainly just pulling train versions of containers on a single compute node, we also changed openstack_release variable to wallaby in globals.yml - this was the mistake. Later that day, we ran kolla-ansible deploy --limit $HOST for a new compute node. We stopped at the following error, realising the above mistake. --- 2023-04-05 12:25:50.487 7 ERROR nova nova.exception.TooOldComputeService: Current Nova version does not support computes older than Victoria but the minimum compute service level in your cell is 40 and the oldest supported service level is 52. 2023-04-05 12:25:50.487 7 ERROR nova --- Now when we deploy a node it gets to task "Running Neutron bootstrap container" and gives error: ---- "alembic.script.revision.ResolutionError: No such revision or branch '6135a7bd4425'" ---- Using insight from the above list thread, I fetched DB entries: MariaDB [neutron]> select * from alembic_version; +--------------+ | version_num | +--------------+ | 5c85685d616d | | 6135a7bd4425 | +--------------+ Running the same from the test cluster which was deployed in the same way: MariaDB [neutron]> select * from alembic_version; +--------------+ | version_num | +--------------+ | 5c85685d616d | | c613d0b82681 | +--------------+ Also, comparing `neutron-db-manage history --verbose` across the two clusters, output is the same in both test and prod, except for the order of 2 lines: ---- -Branches into: 56893333aa52, 30018084ed99 -Branches into: c40fbb377ad, 67c8e8d61d5 +Branches into: 30018084ed99, 56893333aa52 +Branches into: 67c8e8d61d5, c40fbb377ad ---- Finally, I looked at the schema of the neutron database across the two. There are significant differences. for i in `mysql neutron -bNe "show tables;"`; do echo "TABLE: $i"; mysql neutron -e "desc $i;"; done > prod-neutron-descs Repeat for test. $ cat prod-neutron-descs | nc termbin.com 9999 https://termbin.com/8ppi $ cat test-neutron-descs | nc termbin.com 9999 https://termbin.com/cfqz Our cluster and Neutron in particular is operating fine, but we can't deploy new nodes to it. May I have some help on how to resolve this? Thank you, Greg.

4 8

[kolla-ansible][swift][ceph]
by Nguyễn Hữu Khôi 26 Apr '23

26 Apr '23

Hello guys. I am working to make openstack swift save data on ceph. I am using kolla ansible and see it need enable swift first but I don't want to make swift ring on my controller. I just want use external ceph. Pls you give me some advices Thank you Nguyen Huu Khoi

5 8