February 2023 - openstack-discuss

ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version
by Swogat Pradhan 23 Dec '24

23 Dec '24

Hi, I am currently trying to deploy openstack wallaby using tripleo arch. I created the network jinja templates, ran the following commands also: #openstack overcloud network provision --stack overcloud --output networks-deployed-environment.yaml custom_network_data.yaml # openstack overcloud network vip provision --stack overcloud --output vip-deployed-environment.yaml custom_vip_data.yaml # openstack overcloud node provision --stack overcloud --overcloud-ssh-key /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml and used the environment files in the openstack overcloud deploy command: (undercloud) [stack@hkg2director ~]$ cat deploy.sh #!/bin/bash THT=/usr/share/openstack-tripleo-heat-templates/ CNF=/home/stack/ openstack overcloud deploy --templates $THT \ -r $CNF/templates/roles_data.yaml \ -n $CNF/workplace/custom_network_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e $CNF/templates/node-info.yaml \ -e $CNF/templates/scheduler-hints.yaml \ -e $CNF/workplace/networks-deployed-environment.yaml \ -e $CNF/workplace/vip-deployed-environment.yaml \ -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ -e $CNF/workplace/custom-net-bond-with-vlans.yaml Now when i run the ./deploy.sh script i encounter an error stating: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command.: tripleoclient.exceptions.InvalidConfiguration: Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command. 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END return value: 1 Can someone tell me where the mistake is? With regards, Swogat Pradhan

4 4

Kolla Ansible on Ubuntu 20.04 - cloud-init & other network issues
by Tobias McNulty 10 Oct '24

10 Oct '24

Hi, I'm attempting to use Kolla Ansible 14.6.0 to deploy OpenStack Yoga on a small 3-node Ubuntu 20.04 cluster. The nodes have 128 GB RAM each, dual Xeon processors, and dual 10G Intel NICs. The NICs are connected to access ports on a 10G switch with separate VLANs for the local and external networks. All the playbooks run cleanly, but cloud-init is failing in the Ubuntu 20.04 and 22.04 VMs I attempt to boot. The VM images are unmodified from https://cloud-images.ubuntu.com/, and cloud-init works fine if I mount a second volume with user-data. The error is a timeout attempting to reach 169.254.169.254. This occurs both when booting a VM in an internal routed network and directly in an external network. I tried various neutron plugin agents (ovn, linuxbridge, and openvswitch both with and without firewall_driver = openvswitch <https://docs.openstack.org/kolla-ansible/latest/reference/networking/neutro…>) first with a clean install of the entire OS each time, all with the same result. Running tcpdump looking for 169.254.169.254 shows nothing. As a possible clue, the virtual NICs are unable to pass any traffic (e.g., to reach an external DHCP server) unless I completely disable port security on the interface (even if the associated security group is wide open). But disabling port security does not fix cloud-init (not to mention I don't really want to disable port security). Are there any additional requirements related to deploying OpenStack with Kolla on Ubuntu 20.04? This is a fairly vanilla configuration using the multinode inventory as a starting point. I tried to follow the Quick Start <https://docs.openstack.org/kolla-ansible/yoga/user/quickstart.html> as closely as possible; the only material difference I see is that I'm using the same 3 nodes for control + compute. I am using MAAS so it's easy to get a clean OS install on all three nodes ahead of each attempt. I plan to try again with the standard (non-HWE) kernel just in case, but otherwise I am running out of ideas. In case of any additional clues, here are my globals.yml and inventory file, along with the playbook I'm using to configure the network, images, VMs, etc., after bootstrapping the cluster: https://gist.github.com/tobiasmcnulty/7dbbdbc67abc08cbb013bf5983852ed6 Thank you in advance for any advice! Cheers, Tobias

5 9

Live migration fails
by Szabo, Istvan (Agoda) 06 Aug '24

06 Aug '24

Hi, I have couple of compute nodes where the live migration fails with existing vms. When I quickly spawn a vm and try live migration it works so I assume shouldn't be a big problem with the compute node. However I have many existing vms where it fails with a servername not found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 ERROR nova.conductor.tasks.migrate [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Unable to find record for source node servername on servername: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Failed to compute_task_migrate_server: Compute host servername could not be found.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Setting instance to ACTIVE state.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.672 227612 ERROR oslo_messaging.rpc.server [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Exception during message handling: ComputeHostNotFound: Compute host am-osfecn-4025 Tried with this command: nova live-migration --block-migrate id. Any idea? Thank you. ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

5 16

[ptl][tc] OpenStack packages PyPi additional external maintainers audit & cleanup
by Ghanshyam Mann 16 May '24

16 May '24

Hi PTLs, As you might know or have seen for your project package on PyPi, OpenStack deliverables on PyPi have additional maintainers, For example, https://pypi.org/project/murano/, https://pypi.org/project/glance/ We should keep only 'openstackci' as a maintainer in PyPi so that releases of OpenStack deliverables can be managed in a single place. Otherwise, we might face the two sets of maintainers' places and packages might get released in PyPi by additional maintainers without the OpenStack project team knowing about it. One such case is in Horizon repo 'xstatic-font-awesome' where a new maintainer is added by an existing additional maintainer and this package was released without the Horizon team knowing about the changes and release. - https://github.com/openstack/xstatic-font-awesome/pull/2 To avoid the 'xstatic-font-awesome' case for other packages, TC discussed it in their weekly meetings[1] and agreed to audit all the OpenStack packages and then clean up the additional maintainers in PyPi (keep only 'openstackci' as maintainers). To help in this task, TC requests project PTL to perform the audit for their project's repo and add comments in the below etherpad. - https://etherpad.opendev.org/p/openstack-pypi-maintainers-cleanup Thanks to knikolla to automate the listing of the OpenStack packages with additional maintainers in PyPi which you can find the result in output.txt at the bottom of this link. I have added the project list of who needs to check their repo in etherpad. - https://gist.github.com/knikolla/7303a65a5ddaa2be553fc6e54619a7a1 Please complete the audit for your project before March 15 so that TC can discuss the next step in vPTG. [1] https://meetings.opendev.org/meetings/tc/2023/tc.2023-01-11-16.00.log.html#… -gmann

18 27

[Watcher] How use Watcher to scheduler Openstack resource without cpu_usage
by Nguyễn Hữu Khôi 06 Mar '24

06 Mar '24

Hello guys. I want to use watcher to scheduler openstack resources but some openstack versions are missing cpu_usage metric. How do we use Watcher without it? Thank you. Nguyen Huu Khoi

3 5

[designate] Proposal to deprecate the agent framework and agent based backends
by Michael Johnson 09 Jan '24

09 Jan '24

TLDR: The Designate team would like to deprecate the backend agent framework and the agent based backends due to lack of development and design issues with the current implementation. The following backends would be deprecated: Bind9 (Agent), Denominator, Microsoft DNS (Agent), Djbdns (Agent), Gdnsd (Agent), and Knot2 (Agent). Designate includes many backend DNS server drivers[1], many of which are "native" (also known as xfr type backends) backend implementations. In addition to the "native" backends, Designate has an agent backend[2] that supports other backends via an agent process. To quote the agent backend documentation[2]: This backend uses an extension[3] of the DNS protocol itself to send management requests to the remote agent processes, where the requests will be actioned. The rpc traffic between designate and the agent is both unauthenticated and unencrypted. Do not run this traffic over unsecured networks. Here are the reasons we are proposing to deprecate the agent framework now: 1. The agent protocol used by Designate is using an "unassigned"[4][5] DNS opcode (14) that is causing problems with the dnspython library >= 2.3.0 which is now validating the opcode when building DNS messages. It is a bad practice to use "unassigned" values as they may be officially assigned at any time, likely with an incompatible message format. 2. The agent backends are not tested in the OpenStack jobs[1]. 3. Many of the agent backends have been marked as "Experimental" since 2016 with no additional contributions beyond general repository code maintenance. 4. The protocol between the Designate worker process and the agent process is unauthenticated and unencrypted. 5. We do not know of a development resource to rewrite the agent framework protocol to address issue #1 and #4 above. 6. The introduction of catalog zones[6] may eliminate the need for some of the agent based backend drivers. By marking the agent framework and agent based backends "deprecated" in the Antelope cycle, we would remove the code no earlier than the "C" release of OpenStack (per the OpenStack deprecation policy[7]). In the meantime, issue #1 has been worked around by overriding the dnspython opcode validation[7] as needed in the Designate code (similar to a monkey patch). This is not a sustainable long term solution. The following backend agent based drivers would be marked "deprecated" in addition to the agent framework itself: Bind9 (Agent) Denominator Microsoft DNS (Agent) Djbdns (Agent) Gdnsd (Agent) Knot2 (Agent) We plan to propose patches for the deprecation over the next week, but will not merge them until at least January 24th to allow time for comment from the community. If you have concerns about this deprecation plan or are interested in rewriting the agent framework protocol to address the above issues, please reply to this announcement. Michael [1] https://docs.openstack.org/designate/latest/admin/support-matrix.html [2] https://docs.openstack.org/designate/latest/admin/backends/agent.html [3] https://github.com/openstack/designate/blob/master/designate/backend/privat… [4] https://www.rfc-editor.org/rfc/rfc6895.html#section-2.2 [5] https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-pa… [6] https://www.ietf.org/archive/id/draft-ietf-dnsop-dns-catalog-zones-08.txt [7] https://docs.openstack.org/project-team-guide/deprecation.html [8] https://review.opendev.org/c/openstack/designate/+/870678

3 13

[openstack-ansible] Recommended branch for a production environment?
by Roger Rivera 05 Sep '23

05 Sep '23

Hello, We have deployed Openstack-Ansible in a test environment and we were wondering what the recommended repository branch is to deploy in a production environment that will be integrated with a separate Ceph cluster. 1. We are thinking about pinning to the stable/zed. Is that recommended over the master branch? 2. Are bugs ironed out on both master and stable/zed with the same cadence? 3. Additionally, is Debian 11 a better alternative than Ubuntu 22 for target hosts? We noticed Ubuntu 22.04 support was added recently, whereas Debian 11 has been supported for quite some time now which leads us to believe its stability/integration could be more mature at this point? Any suggestions will be appreciated. Thank you, -- *Roger *

3 7

[puppet] Proposing core team clean up
by Takashi Kajinami 23 Aug '23

23 Aug '23

Hello, I went through the current core reviewers and found out that the current list includes some people no longer working in the OpenStack community. I'd like to propose removing the following persons from the core reviewer team as we haven't seen these people making any activity in the community recently. * Pradeep Kilambi - puppet-aodh, puppet-ceilometer, puppet-gnocchi, puppet-panko * Richard Megginson - puppet-keystone * Denis Egorenko - puppet-murano * Serg Melikyan - puppet-murano * Sergey Kolekonov - puppet-neutron * David Moreau Simard - puppet-openstack-integration * Yu xingchao - all modules Also, I confirmed that Wes Hayutin is no longer working for OpenStack and he agreed to retire from the core reviewer position. Thus I'll remove him as well. I'll be open for any feedback for one week to hear any thoughts/feedback about this removal. I'd like to take this moment to thank all people for their past work. We always welcome them back if they resume their work in this project. Thank you, Takashi KAJINAMI

3 4

[cloudkitty] question to the scope_key in cloudkitty
by Noah Elias Feldt 28 Jul '23

28 Jul '23

Hello! I am currently trying to get CloudKitty to work with the Prometheus Collector and PyScripts. I noticed that the scope_key for Prometheus can only be changed once in the global configuration file. This was a problem for me because we need different scope_keys for different metrics. Many of the metrics still need the scope key "tenant_id" and some others "project_id". I have found in the documentation of CloudKitty no way to change this for each metric, so I have internally changed something in the source code and added the function to set the scope_key for each metric individually. I just wanted to ask if you would be interested if I submit the small change upstream? Thank you. Regards Noah Feldt

3 3

[cinder] cinder-backup volume stuck in creating
by Satish Patel 27 Jul '23

27 Jul '23

Folks, I have configured nova and cinder with ceph storage. VMs running on ceph storage but now when i am trying to create a backup of cinder volume its getting stuck on creating and doing nothing. Logs also do not give any indication of bad. My cinder.conf [DEFAULT] enabled_backends = rbd-1 backup_driver = cinder.backup.drivers.ceph.CephBackupDriver backup_ceph_conf = /etc/ceph/ceph.conf backup_ceph_user = cinder-backup backup_ceph_chunk_size = 134217728 backup_ceph_pool = backups backup_ceph_stripe_unit = 0 backup_ceph_stripe_count = 0 restore_discard_excess_bytes = true osapi_volume_listen = 10.73.0.181 osapi_volume_listen_port = 8776 Output of "openstack volume service list" showing cinder-backup service is up but when i create a backup it's getting stuck in this stage and no activity. I am not seeing anything getting transferred to the ceph backups pool also. Any clue? or method to debug? # openstack volume backup list --all +--------------------------------------+------+-------------+----------+------+ | ID | Name | Description | Status | Size | +--------------------------------------+------+-------------+----------+------+ | bc844d55-8c5a-4bd3-b0e9-7c4c780c95ad | foo1 | | creating | 20 | +--------------------------------------+------+-------------+----------+------+

5 7

Re: [all] [oslo.messaging] Interest in collaboration on a NATS driver
by Tobias Urdin 22 Jul '23

22 Jul '23

Hello Kai, Great to hear that you’re interested! I’m currently just testing this as a POC directly in devstack so there is real-world testing (not should it yet I think, needs more development first). Best regards Tobias > On 30 Aug 2022, at 18:32, Kai Bojens <kb(a)kbojens.de> wrote: > > Am 29.08.22 um 15:46 schrieb Tobias Urdin: > >> If anybody, or any company, out there would be interested in collaborating in a project to bring this support and maintain it feel free to >> reach out. I’m hoping somebody will bite but atleast I’ve put it out there for all of you. > > Hi, > I am very much interested to take a closer look at your work and maybe contribute to it. Although I'm working with OpenStack for my employer at the moment, I'd do this in my spare time as I'm not sure that I can convince him to add another staging system with a full OpenStack installation just for developing a RabbitMQ replacement. That's not on our agenda as we are mostly users and not developers of OpenStack. > > As I'm pretty new to the messaging topic and had just heard of NATS and your idea, I'll first would have to dive into NATS and then I would take a closer look at your code and maybe try to create some documentation or tests. So, I can't promise anything but as I said I'm very interested in your approach as I also see the massive load from RabbitMQ. > > This brings me to the most important question: Where do you run and test your code? > > Greetings, > Kai

8 30

Experience with VGPUs
by Oliver Weinmann 22 Jun '23

22 Jun '23

Dear All, we are planning to have a POC on VGPUs in our Openstack cluster. Therefore I have a few questions and generally wanted to ask how well VGPUs are supported in Openstack. The docs, in particular: https://docs.openstack.org/nova/zed/admin/virtual-gpu.html explain quite well the general implementation. But I am more interested in general experience with using VGPUs in Openstack. We currently have a small YOGA cluster, planning to upgrade to Zed soon, with a couple of compute nodes. Currently our users use consumer cards like RTX 3050/3060 on their laptops and the idea would be to provide VGPUs to these users. For this I would like to make a very small POC where we first equip one compute node with an Nvidia GPU. Gladly also a few tips on which card would be a good starting point are highly appreciated. I know this heavily depends on the server hardware but this is something I can figure out later. Also do we need additional software licenses to run this? I saw this very nice presentation from CERN on VGPUs: https://indico.cern.ch/event/776411/contributions/3345183/attachments/18516… In the table they are listing Quadro vDWS licenses. I assume we need these in order to use the cards? Also do we need something like Cyborg for this or is VGPU fully implemented in Nova? Best Regards, Oliver

17 36

Kubernetes Conformance 1.24 + 1.25
by Kendall Nelson 05 Jun '23

05 Jun '23

Hello All! First of all, I want to say a huge thanks to Guilherme Steinmuller for all his help ensuring that OpenStack Magnum remains Kubernetes Certified [1]! We are certified for v1.24! However, in digging into the set of conformance tests this round, it appears that we will not be able to get 1.25 certified with the current Magnum. This means that in May when we have to re-cert (every 3 months they reach an EOL because that's k8s release cadence) we will not be able to maintain this which is the other part of why I wanted to write this email - to raise awareness and ask for help digging into what needs to happen to make sure we can retain this. Tangentially related, Guilherme has posted a patch with documentation for how to run the conformance tests for yourself and it could use some reviews [2]. And one last thing, I have begun conversation with the opendev folks about what it might take to setup jobs to run the conformance tests periodically so that we don't have to manually run them when it's time to recert. I am planning on submitting a forum topic for this for those interested :) Long live LOKI! - Kendall Nelson [1] https://github.com/cncf/k8s-conformance/pull/2446 [2] https://review.opendev.org/c/openstack/magnum/+/849156

5 10

[ironic][stable] Proposing EOL of ironic project branches older than Wallaby
by Jay Faulkner 01 Jun '23

01 Jun '23

Hi all, Ironic has a large amount of stable branches still in EM. We need to take action to ensure those branches are either retired or have CI repaired to the point of being usable. Specifically, I'm looking at these branches across all Ironic projects: - stable/queens - stable/rocky - stable/stein - stable/train - stable/ussuri - stable/victoria In lieu of any volunteers to maintain the CI, my recommendation for all the branches listed above is that they be marked EOL. If someone wants to volunteer to maintain CI for those branches, they can propose one of the below paths be taken instead: 1 - Someone volunteers to maintain these branches, and also report the status of CI of these older branches periodically on the Ironic whiteboard and in Ironic meetings. If you feel strongly that one of these branches needs to continue to be in service; volunteering in this way is how to save them. 2 - We seriously reduce CI. Basically removing all tempest tests to ensure that CI remains reliable and able to merge emergency or security fixes when needed. In some cases; this still requires CI fixes as some older inspector branches are failing *installing packages* in unit tests. I would still like, in this case, that someone volunteers to ensure the minimalist CI remains happy. My intention is to let this message serve as notice and a waiting period; and if I've not heard any response here or in Monday's Ironic meeting (in 6 days), I will begin taking action on retiring these branches. This is simply a start; other branches (including bugfix branches) are also in bad shape in CI, but getting these retired will significantly reduce the surface area of projects and branches to evaluate. I know it's painful to drop support for these branches; but we've provided good EM support for these branches for a long time and by pruning them away, we'll be able to save time to dedicate to other items. Thanks, Jay Faulkner

3 5

[magnum] Propose deprecation of Docker Swarm, CoreOS, fedora-ironic
by Jake Yip 28 Apr '23

28 Apr '23

Hi, Due to limited resources, Magnum team has decided to put most effort into supporting the most popular use case - Kubernetes on Fedora CoreOS. In light of that, we are seeking maintainers for the less well maintained drivers / COE - Docker Swarm, CoreOS and fedora-ironic drivers. The reasons are as follows: - Docker Swarm - It uses Fedora Atomic which has been EOL [1] - CoreOS - It is EOL [2] - fedora-ironic - Not tested, unsure if it is still working If anyone wants to take up maintenance of any of the above, we will be glad. Please let us know either through the ML or at the Magnum meetings. If there are no suitable maintainers, we will start the deprecation of these before end of Antelope. Regards, Jake [1]: https://projectatomic.io/blog/2019/11/fedora-atomic-host-nearing-eol/ [2]: https://www.redhat.com/en/technologies/cloud-computing/openshift/what-was-c… -- Jake Yip Magnum Team

2 3

Re: [all][stable][ptl] Propose to EOL Rocky series
by Elõd Illés 17 Apr '23

17 Apr '23

Hi, thanks for all the feedbacks from teams so far! @Zigo: Extended Maintenance process was created just for the same situation: to give space to interested parties to cooperate and keep things maintained even when stable releases are over their 'supported' lifetime. So it's good to see that there is interest in it! Unfortunately, with very old branches we've reached the state where gates can't be maintained and without a functional gate it's not safe to merge patches (yes, even security fixes) and they are just using resources (CI & maintainers' time). When gate is broken in such extent, then i think the community have to accept that it is not possible to merge patches confidently and needs to EOL that release. (Yes, it means that some vendors still have to do it for their own sake, testing it by their own, etc). Keeping things open in such state gives the false signal to vendors that 'it is maintained' however this is done without fully testing the changes. Another aspect is that code cannot be cleaned up until those old branches are still present (CI jobs, project configurations, etc) which gives pain for developers. So, however some vendors would appreciate probably to keep things open forever, for the community this is not beneficial and doable I think. Seeing that teams want to EOL their old branches is a proof that this is needed (unfortunately). Thanks, Előd Illés irc: elodilles @ #openstack-stable / #openstack-release ________________________________ From: Thomas Goirand <thomas(a)goirand.fr> Sent: Saturday, January 28, 2023 2:25 PM Cc: openstack-discuss(a)lists.openstack.org <openstack-discuss(a)lists.openstack.org> Subject: Re: [all][stable][ptl] Propose to EOL Rocky series Hi, I understand the gate is broken, however, Rocky is in Debian LTS, and I would like to keep the possibility to merge patches, even with the gate tests disabled. CVE-2022-47951 is an example why this is important... Thomas Goirand (zigo) On Jan 28, 2023 11:54, Radosław Piliszek <radoslaw.piliszek(a)gmail.com> wrote: > > Masakari is happy to EOL Rocky too. > > Radek > -yoctozepto > > On Fri, 27 Jan 2023 at 20:59, Jay Faulkner <jay(a)gr-oss.io> wrote: > > > > Thanks for doing all this cleanup work Elod. Ironic is OK with retirements of these shared resources up to Train. > > > > - > > Jay Faulkner > > > > On Fri, Jan 27, 2023 at 10:12 AM Elõd Illés <elod.illes(a)est.tech> wrote: > >> > >> Hi, > >> > >> Similarly like the Queens branch EOL proposal [1] I would like to propose > >> to transition every project's stable/rocky to End of Life: > >> > >> - gates are mostly broken > >> - minimal number of activity can be seen on this branch > >> - some core projects already transitioned their stable/rocky to EOL > >> recently (like ironic, neutron, nova) > >> - gate job definitions are still using the old, legacy zuul syntax > >> - gate jobs are based on Ubuntu Xenial, which is also beyond its public > >> maintenance window date and hard to maintain > >> > >> Based on the above, if there won't be any project who wants to keep open > >> their stable/rocky, then I'll start the process of EOL'ing Rocky stable > >> series as a whole. If anyone has any objection then please respond to > >> this mail. > >> > >> Thanks, > >> > >> Előd Illés > >> irc: elodilles @ #openstack-stable / #openstack-release > >> > >> [1] https://lists.openstack.org/pipermail/openstack-discuss/2022-October/031030… > >> >

3 4

DCN compute service goes down when a instance is scheduled to launch | wallaby | tripleo
by Swogat Pradhan 27 Mar '23

27 Mar '23

Hi, I currently have 3 compute nodes on edge site1 where i am trying to launch vm's. When the VM is in spawning state the node goes down (openstack compute service list), the node comes backup when i restart the nova compute service but then the launch of the vm fails. nova-compute.log 2023-02-26 08:15:51.808 7 INFO nova.compute.manager [req-bc0f5f2e-53fc-4dae-b1da-82f1f972d617 - - - - -] Running instance usage audit for host dcn01-hci-0.bdxworld.com from 2023-02-26 07:00:00 to 2023-02-26 08:00:00. 0 instances. 2023-02-26 08:49:52.813 7 INFO nova.compute.claims [req-3a1547ea-326f-4dd0-9127-7f4a4bdf1e45 b240e3e89d99489284cd731e75f2a5db 4160ce999a31485fa643aed0936dfef0 - default default] [instance: 0c62c1ef-9010-417d-a05f-4db77e901600] Claim successful on node dcn01-hci-0.bdxworld.com 2023-02-26 08:49:54.225 7 INFO nova.virt.libvirt.driver [req-3a1547ea-326f-4dd0-9127-7f4a4bdf1e45 b240e3e89d99489284cd731e75f2a5db 4160ce999a31485fa643aed0936dfef0 - default default] [instance: 0c62c1ef-9010-417d-a05f-4db77e901600] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names 2023-02-26 08:49:54.398 7 INFO nova.virt.block_device [req-3a1547ea-326f-4dd0-9127-7f4a4bdf1e45 b240e3e89d99489284cd731e75f2a5db 4160ce999a31485fa643aed0936dfef0 - default default] [instance: 0c62c1ef-9010-417d-a05f-4db77e901600] Booting with volume c4bd7885-5973-4860-bbe6-7a2f726baeee at /dev/vda 2023-02-26 08:49:55.216 7 WARNING nova.cache_utils [req-3a1547ea-326f-4dd0-9127-7f4a4bdf1e45 b240e3e89d99489284cd731e75f2a5db 4160ce999a31485fa643aed0936dfef0 - default default] Cache enabled with backend dogpile.cache.null. 2023-02-26 08:49:55.283 7 INFO oslo.privsep.daemon [req-3a1547ea-326f-4dd0-9127-7f4a4bdf1e45 b240e3e89d99489284cd731e75f2a5db 4160ce999a31485fa643aed0936dfef0 - default default] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova.conf', '--config-file', '/etc/nova/nova-compute.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpin40tah6/privsep.sock'] 2023-02-26 08:49:55.791 7 INFO oslo.privsep.daemon [req-3a1547ea-326f-4dd0-9127-7f4a4bdf1e45 b240e3e89d99489284cd731e75f2a5db 4160ce999a31485fa643aed0936dfef0 - default default] Spawned new privsep daemon via rootwrap 2023-02-26 08:49:55.717 2647 INFO oslo.privsep.daemon [-] privsep daemon starting 2023-02-26 08:49:55.722 2647 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 2023-02-26 08:49:55.726 2647 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none 2023-02-26 08:49:55.726 2647 INFO oslo.privsep.daemon [-] privsep daemon running as pid 2647 2023-02-26 08:49:55.956 7 WARNING os_brick.initiator.connectors.nvmeof [req-3a1547ea-326f-4dd0-9127-7f4a4bdf1e45 b240e3e89d99489284cd731e75f2a5db 4160ce999a31485fa643aed0936dfef0 - default default] Process execution error in _get_host_uuid: Unexpected error while running command. Command: blkid overlay -s UUID -o value Exit code: 2 Stdout: '' Stderr: '': oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. 2023-02-26 08:49:58.247 7 INFO nova.virt.libvirt.driver [req-3a1547ea-326f-4dd0-9127-7f4a4bdf1e45 b240e3e89d99489284cd731e75f2a5db 4160ce999a31485fa643aed0936dfef0 - default default] [instance: 0c62c1ef-9010-417d-a05f-4db77e901600] Creating image Is there a way to solve this issue? With regards, Swogat Pradhan

5 40

[all][ptg] Pre-PTG discussion: New Keystone Middleware Feature Supporting OAuth2.0 with External Authorization Service
by Hiromu Asahina 27 Mar '23

27 Mar '23

Hello Everyone, Recently, Tacker and Keystone have been working together on a new Keystone Middleware that can work with external authentication services, such as Keycloak. The code has already been submitted [1], but we want to make this middleware a generic plugin that works with as many OpenStack services as possible. To that end, we would like to hear from other projects with similar use cases (especially Ironic and Barbican, which run as standalone services). We will make a time slot to discuss this topic at the next vPTG. Please contact me if you are interested and available to participate. [1] https://review.opendev.org/c/openstack/keystonemiddleware/+/868734 -- Hiromu Asahina

4 11

[TripleO] Last maintained release of TripleO is Wallaby
by James Slagle 24 Mar '23

24 Mar '23

The team of TripleO developers, reviewers, and community have no plans to do another release of TripleO, or to continue maintaining the Zed release. This means the last maintained release of TripleO is Wallaby. The plan is to continue to maintain the train and wallaby branches of TripleO repositories under the openstack namespace. We plan to continue to run existing CI on those branches, although we anticipate reducing to a minimal set of jobs in the future. Going forward, the TripleO team expects to continue involvement in other OpenStack projects, and focus on different approaches to OpenStack deployment. The team is not aware of enough other community interest to continue to maintain the zed, master, and future branches of TripleO. Eventually, we will empty commit these branches with a pointer to Wallaby being the last release. If other interests do exist to maintain TripleO going forward, then those plans can adjust. TripleO does not plan on nominating a PTL in the current Bobcat election cycle. If a PTL-like contact is needed for train and wallaby maintenance, then someone will be nominated. I am also not sure how to reflect this project structure change within openstack/governance, and am looking for guidance on how to do so. This type of change did not seem to exactly fit within the concepts of inactive or retired projects. I can work on the patches for governance with some direction, if anyone can provide it. Thanks, and please let us know any questions or feedback about this plan. -- -- James Slagle --

6 21

[Openstack] Lack of Balance solution such as Watcher.
by Nguyễn Hữu Khôi 16 Mar '23

16 Mar '23

I am building a private cloud.Everything is ok. But I cannot find a way to balance vm when It have heavy load which cause impact other vm on the compute node. I found solutions such as Watcher and Leveller. But they need to be done manually. Watcher is not good because It need cpu metric such as cpu load in Ceilometer which is removed so we cannot use it. Leveller is good, but It is obsolete. Nguyen Huu Khoi

4 7

live resize
by Parsa Aminian 15 Mar '23

15 Mar '23

hello on openstack with ceph backend is it possible to live resize instances ? I want to change flavor without any down time .

5 6

[security][ironic] Ironic + the VMT
by Jay Faulkner 10 Mar '23

10 Mar '23

Hey all, A recent topic at Ironic meetings the last couple of weeks have been around vulnerability management. Ironic has not been using the OpenStack VMT traditionally; for reasons that AFAICT are lost to time. Is there any reason Ironic should not be vulnerability-managed? Is the security team willing to have us? The only potential complication is that Ironic may receive reports for vendor libraries used by Ironic but not maintained by Ironic -- I was hoping there might already be some historical precedent for how we handle those; it can't be that unique to Ironic. What do folks think? Thanks, Jay Faulkner Ironic PTL TC Member

2 2

[openstack-dev][PCI passthrough] How to use PCI passthrough feature correctly? And is this BUG in update_devices_from_hypervisor_resources?
by Simon Jones 10 Mar '23

10 Mar '23

Hi all, I'm working on openstack Yoga's PCI passthrough feature, follow this link: https://docs.openstack.org/nova/latest/admin/pci-passthrough.html I configure exactly as the link said, but when I create server use this command, I found ERROR: ``` openstack server create --flavor cirros-os-dpu-test-1 --image cirros \ --nic net-id=066c8dc2-c98b-4fb8-a541-8b367e8f6e69 \ --security-group default --key-name mykey provider-instance | fault | {'code': 500, 'created': '2023-02-23T06:13:43Z', 'message': 'No valid host was found. There are not enough hosts available.', 'details': 'Traceback (most recent call last):\n File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 1548, in schedule_and_build_instances\n host_lists = self._schedule_instances(context, request_specs[0],\n File "/usr/lib/python3/dist-packages/nova/conductor/manager.py", line 908, in _schedule_instances\n host_lists = self.query_client.select_destinations(\n File "/usr/lib/python3/dist-packages/nova/scheduler/client/query.py", line 41, in select_destinations\n return self.scheduler_rpcapi.select_destinations(context, spec_obj,\n File "/usr/lib/python3/dist-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations\n return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/client.py", line 189, in call\n result = self.transport._send(\n File "/usr/lib/python3/dist-packages/oslo_messaging/transport.py", line 123, in _send\n return self._driver.send(target, ctxt, message,\n File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send\n return self._send(target, ctxt, message, wait_for_reply, timeout,\n File "/usr/lib/python3/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send\n raise result\nnova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available.\nTraceback (most recent call last):\n\n File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 241, in inner\n return func(*args, **kwargs)\n\n File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 223, in select_destinations\n selections = self._select_destinations(\n\n File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 250, in _select_destinations\n selections = self._schedule(\n\n File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 416, in _schedule\n self._ensure_sufficient_hosts(\n\n File "/usr/lib/python3/dist-packages/nova/scheduler/manager.py", line 455, in _ensure_sufficient_hosts\n raise exception.NoValidHost(reason=reason)\n\nnova.exception.NoValidHost: No valid host was found. There are not enough hosts available.\n\n'} | // this is what I configured: gyw@c1:~$ openstack flavor show cirros-os-dpu-test-1 +----------------------------+------------------------------+ | Field | Value | +----------------------------+------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | access_project_ids | None | | description | None | | disk | 1 | | id | 0 | | name | cirros-os-dpu-test-1 | | os-flavor-access:is_public | True | | properties | pci_passthrough:alias='a1:1' | | ram | 64 | | rxtx_factor | 1.0 | | swap | | | vcpus | 1 | +----------------------------+------------------------------+ // in controller node /etc/nova/nova.conf [filter_scheduler] enabled_filters = PciPassthroughFilter available_filters = nova.scheduler.filters.all_filters [pci] passthrough_whitelist = {"vendor_id": "15b3", "product_id": "101e", "physical_network": null, "remote_managed": "true"} alias = { "vendor_id":"15b3", "product_id":"101e", "device_type":"type-VF", "name":"a1" } // in compute node /etc/nova/nova.conf [pci] passthrough_whitelist = {"vendor_id": "15b3", "product_id": "101e", "physical_network": null, "remote_managed": "true"} alias = { "vendor_id":"15b3", "product_id":"101e", "device_type":"type-VF", "name":"a1" } ``` The detail ERROR I found is: - The reason why "There are not enough hosts available" is, nova-scheduler's log shows "There are 0 hosts available but 1 instances requested to build", which means no hosts support PCI passthough feature. This is nova-schduler's log ``` 2023-02-28 06:11:58.329 1942637 DEBUG nova.scheduler.manager [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Starting to schedule for instances: ['8ddfbe2c-f929-4b62-8b73-67902df8fb60'] select_destinations /usr/lib/python3/dist-packages/nova/scheduler/manager.py:141 2023-02-28 06:11:58.330 1942637 DEBUG nova.scheduler.request_filter [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] compute_status_filter request filter added forbidden trait COMPUTE_STATUS_DISABLED compute_status_filter /usr/lib/python3/dist-packages/nova/scheduler/request_filter.py:254 2023-02-28 06:11:58.330 1942637 DEBUG nova.scheduler.request_filter [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Request filter 'compute_status_filter' took 0.0 seconds wrapper /usr/lib/python3/dist-packages/nova/scheduler/request_filter.py:46 2023-02-28 06:11:58.331 1942637 DEBUG nova.scheduler.request_filter [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Request filter 'accelerators_filter' took 0.0 seconds wrapper /usr/lib/python3/dist-packages/nova/scheduler/request_filter.py:46 2023-02-28 06:11:58.332 1942637 DEBUG nova.scheduler.request_filter [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Request filter 'remote_managed_ports_filter' took 0.0 seconds wrapper /usr/lib/python3/dist-packages/nova/scheduler/request_filter.py:46 2023-02-28 06:11:58.485 1942637 DEBUG oslo_concurrency.lockutils [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Lock "567eb2f1-7173-4eee-b9e7-66932ed70fea" acquired by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py:386 2023-02-28 06:11:58.488 1942637 DEBUG oslo_concurrency.lockutils [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Lock "567eb2f1-7173-4eee-b9e7-66932ed70fea" "released" by "nova.context.set_target_cell.<locals>.get_or_set_cached_cell_and_set_connections" :: held 0.003s inner /usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py:400 2023-02-28 06:11:58.494 1942637 DEBUG oslo_db.sqlalchemy.engines [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3/dist-packages/oslo_db/sqlalchemy/engines.py:314 2023-02-28 06:11:58.520 1942637 INFO nova.scheduler.host_manager [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Host mapping not found for host c1c2. Not tracking instance info for this host. 2023-02-28 06:11:58.520 1942637 DEBUG oslo_concurrency.lockutils [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Lock "('c1c2', 'c1c2')" acquired by "nova.scheduler.host_manager.HostState.update.<locals>._locked_update" :: waited 0.000s inner /usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py:386 2023-02-28 06:11:58.521 1942637 DEBUG nova.scheduler.host_manager [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Update host state from compute node: ComputeNode(cpu_allocation_ratio=16.0,cpu_info='{"arch": "x86_64", "model": "Broadwell-noTSX-IBRS", "vendor": "Intel", "topology": {"cells": 1, "sockets": 1, "cores": 6, "threads": 2}, "features": ["sse4.2", "mds-no", "stibp", "pdpe1gb", "xsaveopt", "ht", "intel-pt", "mtrr", "abm", "tm", "lm", "umip", "mca", "pku", "ds_cpl", "rdrand", "adx", "rdseed", "lahf_lm", "xgetbv1", "nx", "invpcid", "rdtscp", "tsc", "xsavec", "pcid", "arch-capabilities", "pclmuldq", "spec-ctrl", "fsgsbase", "avx2", "md-clear", "vmx", "syscall", "mmx", "ds", "ssse3", "avx", "dtes64", "fxsr", "msr", "acpi", "vpclmulqdq", "smap", "erms", "pge", "cmov", "sha-ni", "fsrm", "x2apic", "xsaves", "cx8", "pse", "pse36", "clflushopt", "vaes", "pni", "ssbd", "movdiri", "movbe", "clwb", "xtpr", "de", "invtsc", "fpu", "tsc-deadline", "pae", "clflush", "ibrs-all", "waitpkg", "sse", "sse2", "bmi1", "3dnowprefetch", "cx16", "popcnt", "rdctl-no", "fma", "tsc_adjust", "xsave", "ss", "skip-l1dfl-vmentry", "sse4.1", "rdpid", "monitor", "vme", "tm2", "pat", "pschange-mc-no", "movdir64b", "gfni", "mce", "smep", "sep", "apic", "arat", "f16c", "bmi2", "aes", "pbe", "est", "pdcm"]}',created_at=2023-02-14T03:19:40Z,current_workload=0,deleted=False,deleted_at=None,disk_allocation_ratio=1.0,disk_available_least=415,free_disk_gb=456,free_ram_mb=31378,host='c1c2',host_ip=192.168.28.21,hypervisor_hostname='c1c2',hypervisor_type='QEMU',hypervisor_version=4002001,id=8,local_gb=456,local_gb_used=0,mapped=0,memory_mb=31890,memory_mb_used=512,metrics='[]',numa_topology='{" nova_object.name": "NUMATopology", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"cells": [{" nova_object.name": "NUMACell", "nova_object.namespace": "nova", "nova_object.version": "1.5", "nova_object.data": {"id": 0, "cpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "pcpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "memory": 31890, "cpu_usage": 0, "memory_usage": 0, "pinned_cpus": [], "siblings": [[0, 1], [10, 11], [2, 3], [6, 7], [4, 5], [8, 9]], "mempages": [{"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 4, "total": 8163962, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "reserved", "total"]}, {" nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 2048, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "reserved", "total"]}, {"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 1048576, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "reserved", "total"]}], "network_metadata": {"nova_object.name": "NetworkMetadata", "nova_object.namespace": "nova", "nova_object.version": "1.0", "nova_object.data": {"physnets": [], "tunneled": false}, "nova_object.changes": ["physnets", "tunneled"]}, "socket": 0}, "nova_object.changes": ["cpuset", "memory_usage", "cpu_usage", "id", "pinned_cpus", "pcpuset", "socket", "network_metadata", "siblings", "mempages", "memory"]}]}, "nova_object.changes": ["cells"]}',pci_device_pools=PciDevicePoolList,ram_allocation_ratio=1.5,running_vms=0,service_id=None,stats={failed_builds='0'},supported_hv_specs=[HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec],updated_at=2023-02-28T06:01:33Z,uuid=c360cc82-f0fd-4662-bccd-e1f02b27af51,vcpus=12,vcpus_used=0) _locked_update /usr/lib/python3/dist-packages/nova/scheduler/host_manager.py:167 2023-02-28 06:11:58.524 1942637 DEBUG nova.scheduler.host_manager [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Update host state with aggregates: [] _locked_update /usr/lib/python3/dist-packages/nova/scheduler/host_manager.py:170 2023-02-28 06:11:58.524 1942637 DEBUG nova.scheduler.host_manager [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Update host state with service dict: {'id': 17, 'uuid': '6d0921a6-427d-4a82-a7d2-41dfa003125a', 'host': 'c1c2', 'binary': 'nova-compute', 'topic': 'compute', 'report_count': 121959, 'disabled': False, 'disabled_reason': None, 'last_seen_up': datetime.datetime(2023, 2, 28, 6, 11, 49, tzinfo=datetime.timezone.utc), 'forced_down': False, 'version': 61, 'created_at': datetime.datetime(2023, 2, 14, 3, 19, 40, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2023, 2, 28, 6, 11, 49, tzinfo=datetime.timezone.utc), 'deleted_at': None, 'deleted': False} _locked_update /usr/lib/python3/dist-packages/nova/scheduler/host_manager.py:173 2023-02-28 06:11:58.524 1942637 DEBUG nova.scheduler.host_manager [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Update host state with instances: [] _locked_update /usr/lib/python3/dist-packages/nova/scheduler/host_manager.py:176 2023-02-28 06:11:58.525 1942637 DEBUG oslo_concurrency.lockutils [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Lock "('c1c2', 'c1c2')" "released" by "nova.scheduler.host_manager.HostState.update.<locals>._locked_update" :: held 0.004s inner /usr/lib/python3/dist-packages/oslo_concurrency/lockutils.py:400 2023-02-28 06:11:58.525 1942637 DEBUG nova.filters [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Starting with 1 host(s) get_filtered_objects /usr/lib/python3/dist-packages/nova/filters.py:70 2023-02-28 06:11:58.526 1942637 DEBUG nova.pci.stats [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] ---- before ---- _filter_pools /usr/lib/python3/dist-packages/nova/pci/stats.py:542 2023-02-28 06:11:58.526 1942637 DEBUG nova.pci.stats [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] [] _filter_pools /usr/lib/python3/dist-packages/nova/pci/stats.py:543 2023-02-28 06:11:58.526 1942637 DEBUG nova.pci.stats [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] ---- after ---- _filter_pools /usr/lib/python3/dist-packages/nova/pci/stats.py:545 2023-02-28 06:11:58.527 1942637 DEBUG nova.pci.stats [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] [] _filter_pools /usr/lib/python3/dist-packages/nova/pci/stats.py:546 2023-02-28 06:11:58.527 1942637 DEBUG nova.pci.stats [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Not enough PCI devices left to satisfy request _filter_pools /usr/lib/python3/dist-packages/nova/pci/stats.py:556 2023-02-28 06:11:58.527 1942637 DEBUG nova.scheduler.filters.pci_passthrough_filter [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] (c1c2, c1c2) ram: 31378MB disk: 424960MB io_ops: 0 instances: 0 doesn't have the required PCI devices (InstancePCIRequests(instance_uuid=<?>,requests=[InstancePCIRequest])) host_passes /usr/lib/python3/dist-packages/nova/scheduler/filters/pci_passthrough_filter.py:52 2023-02-28 06:11:58.528 1942637 INFO nova.filters [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Filter PciPassthroughFilter returned 0 hosts 2023-02-28 06:11:58.528 1942637 DEBUG nova.filters [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Filtering removed all hosts for the request with instance ID '8ddfbe2c-f929-4b62-8b73-67902df8fb60'. Filter results: [('PciPassthroughFilter', None)] get_filtered_objects /usr/lib/python3/dist-packages/nova/filters.py:114 2023-02-28 06:11:58.528 1942637 INFO nova.filters [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Filtering removed all hosts for the request with instance ID '8ddfbe2c-f929-4b62-8b73-67902df8fb60'. Filter results: ['PciPassthroughFilter: (start: 1, end: 0)'] 2023-02-28 06:11:58.529 1942637 DEBUG nova.scheduler.manager [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] Filtered [] _get_sorted_hosts /usr/lib/python3/dist-packages/nova/scheduler/manager.py:610 2023-02-28 06:11:58.529 1942637 DEBUG nova.scheduler.manager [req-13b1baee-e02d-40fc-926d-d497e70ca0dc ff627ad39ed94479b9c5033bc462cf78 512866f9994f4ad8916d8539a7cdeec9 - default default] There are 0 hosts available but 1 instances requested to build. _ensure_sufficient_hosts /usr/lib/python3/dist-packages/nova/scheduler/manager.py:450 ``` Then I search database, I found PCI configure of compute node is not upload: ``` gyw@c1:~$ openstack resource provider inventory show c360cc82-f0fd-4662-bccd-e1f02b27af51 PCI_DEVICE No inventory of class PCI_DEVICE for c360cc82-f0fd-4662-bccd-e1f02b27af51 (HTTP 404) gyw@c1:~$ openstack resource provider inventory show c360cc82-f0fd-4662-bccd-e1f02b27af51 PCI_DEVICE No inventory of class PCI_DEVICE for c360cc82-f0fd-4662-bccd-e1f02b27af51 (HTTP 404) gyw@c1:~$ openstack resource class show PCI_DEVICE +-------+------------+ | Field | Value | +-------+------------+ | name | PCI_DEVICE | +-------+------------+ gyw@c1:~$ openstack resource provider inventory show c360cc82-f0fd-4662-bccd-e1f02b27af51 MEMORY_MB +------------------+-------+ | Field | Value | +------------------+-------+ | allocation_ratio | 1.5 | | min_unit | 1 | | max_unit | 31890 | | reserved | 512 | | step_size | 1 | | total | 31890 | | used | 0 | +------------------+-------+ 这个 31890 能跟上面compute node resource tracker上报的日志对上。 gyw@c1:~$ openstack resource provider inventory show c360cc82-f0fd-4662-bccd-e1f02b27af51 VCPU 、^Cgyw@c1:~$ openstack resource provider inventory show c360cc82-f0fd-4662-bccd-e1f02b27af51 VCPU +------------------+-------+ | Field | Value | +------------------+-------+ | allocation_ratio | 16.0 | | min_unit | 1 | | max_unit | 12 | | reserved | 0 | | step_size | 1 | | total | 12 | | used | 0 | +------------------+-------+ gyw@c1:~$ openstack resource provider inventory show c360cc82-f0fd-4662-bccd-e1f02b27af51 SRIOV_NET_VF No inventory of class SRIOV_NET_VF for c360cc82-f0fd-4662-bccd-e1f02b27af51 (HTTP 404) gyw@c1:~$ openstack resource provider inventory show c360cc82-f0fd-4662-bccd-e1f02b27af51 DISK_GB +------------------+-------+ | Field | Value | +------------------+-------+ | allocation_ratio | 1.0 | | min_unit | 1 | | max_unit | 456 | | reserved | 0 | | step_size | 1 | | total | 456 | | used | 0 | +------------------+-------+ gyw@c1:~$ openstack resource provider inventory show c360cc82-f0fd-4662-bccd-e1f02b27af51 IPV4_ADDRESS No inventory of class IPV4_ADDRESS for c360cc82-f0fd-4662-bccd-e1f02b27af51 (HTTP 404) MariaDB [nova]> select * from compute_nodes; +---------------------+---------------------+---------------------+----+------------+-------+-----------+----------+------------+----------------+---------------+-----------------+--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------+-------------+--------------+------------------+-------------+---------------------+---------+---------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+-----------------+------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+----------------------+----------------------+--------------------------------------+-----------------------+--------+ | created_at | updated_at | deleted_at | id | service_id | vcpus | memory_mb | local_gb | vcpus_used | memory_mb_used | local_gb_used | hypervisor_type | hypervisor_version | cpu_info | disk_available_least | free_ram_mb | free_disk_gb | current_workload | running_vms | hypervisor_hostname | deleted | host_ip | supported_instances | pci_stats | metrics | extra_resources | stats | numa_topology | host | ram_allocation_ratio | cpu_allocation_ratio | uuid | disk_allocation_ratio | mapped | +---------------------+---------------------+---------------------+----+------------+-------+-----------+----------+------------+----------------+---------------+-----------------+--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------+-------------+--------------+------------------+-------------+---------------------+---------+---------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+-----------------+------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+----------------------+----------------------+--------------------------------------+-----------------------+--------+ | 2023-01-04 01:55:44 | 2023-01-04 03:02:28 | 2023-02-13 08:34:08 | 1 | NULL | 4 | 3931 | 60 | 0 | 512 | 0 | QEMU | 4002001 | {"arch": "x86_64", "model": "Broadwell-noTSX-IBRS", "vendor": "Intel", "topology": {"cells": 1, "sockets": 4, "cores": 1, "threads": 1}, "features": ["pat", "cmov", "ibrs-all", "pge", "sse4.2", "sse", "mmx", "ibrs", "avx2", "syscall", "fpu", "mtrr", "xsaves", "mce", "invpcid", "tsc_adjust", "ssbd", "pku", "ibpb", "xsave", "xsaveopt", "pae", "lm", "pdcm", "bmi1", "avx512vnni", "stibp", "x2apic", "avx512dq", "pcid", "nx", "bmi2", "erms", "3dnowprefetch", "de", "avx512bw", "arch-capabilities", "pni", "fma", "rdctl-no", "sse4.1", "rdseed", "arat", "avx512vl", "avx512f", "pclmuldq", "msr", "fxsr", "sse2", "amd-stibp", "hypervisor", "tsx-ctrl", "clflushopt", "cx16", "clwb", "xgetbv1", "xsavec", "adx", "rdtscp", "mds-no", "cx8", "aes", "tsc-deadline", "pse36", "fsgsbase", "umip", "spec-ctrl", "lahf_lm", "md-clear", "avx512cd", "amd-ssbd", "vmx", "apic", "f16c", "pse", "tsc", "movbe", "smep", "ss", "pschange-mc-no", "ssse3", "popcnt", "avx", "vme", "smap", "pdpe1gb", "mca", "skip-l1dfl-vmentry", "abm", "sep", "clflush", "rdrand"]} | 49 | 3419 | 60 | 0 | 0 | gyw | 1 | 192.168.2.99 | [["i686", "qemu", "hvm"], ["i686", "kvm", "hvm"], ["x86_64", "qemu", "hvm"], ["x86_64", "kvm", "hvm"]] | {"nova_object.name": "PciDevicePoolList", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"objects": []}, "nova_object.changes": ["objects"]} | [] | NULL | {"failed_builds": "0"} | {"nova_object.name": "NUMATopology", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"cells": [{"nova_object.name": "NUMACell", "nova_object.namespace": "nova", "nova_object.version": "1.5", "nova_object.data": {"id": 0, "cpuset": [0, 1, 2, 3], "pcpuset": [0, 1, 2, 3], "memory": 3931, "cpu_usage": 0, "memory_usage": 0, "pinned_cpus": [], "siblings": [[0], [1], [2], [3]], "mempages": [{"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 4, "total": 1006396, "used": 0, "reserved": 0}, "nova_object.changes": ["used", "reserved", "size_kb", "total"]}, {"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 2048, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["used", "reserved", "size_kb", "total"]}, {" nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 1048576, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["used", "reserved", "size_kb", "total"]}], "network_metadata": {"nova_object.name": "NetworkMetadata", "nova_object.namespace": "nova", "nova_object.version": "1.0", "nova_object.data": {"physnets": [], "tunneled": false}, "nova_object.changes": ["physnets", "tunneled"]}, "socket": null}, "nova_object.changes": ["cpuset", "pinned_cpus", "mempages", "network_metadata", "cpu_usage", "pcpuset", "memory", "id", "socket", "siblings", "memory_usage"]}]}, "nova_object.changes": ["cells"]} | gyw | 1.5 | 16 | b1bf35bd-a9ad-4f0c-9033-776a5c6d1c9b | 1 | 1 | | 2023-01-04 03:12:17 | 2023-01-31 06:36:36 | 2023-02-23 08:50:29 | 2 | NULL | 4 | 3931 | 60 | 0 | 512 | 0 | QEMU | 4002001 | {"arch": "x86_64", "model": "Broadwell-noTSX-IBRS", "vendor": "Intel", "topology": {"cells": 1, "sockets": 4, "cores": 1, "threads": 1}, "features": ["pclmuldq", "fsgsbase", "f16c", "fxsr", "ibpb", "adx", "movbe", "aes", "x2apic", "abm", "mtrr", "arat", "sse4.2", "bmi1", "stibp", "sse4.1", "pae", "vme", "msr", "skip-l1dfl-vmentry", "fma", "pcid", "avx2", "de", "ibrs-all", "ssse3", "apic", "umip", "xsavec", "3dnowprefetch", "amd-ssbd", "sse", "nx", "fpu", "pse", "smap", "smep", "lahf_lm", "pni", "spec-ctrl", "xsave", "xsaves", "rdtscp", "vmx", "avx512f", "cmov", "invpcid", "hypervisor", "erms", "rdctl-no", "cx16", "cx8", "tsc", "pge", "pdcm", "rdrand", "avx", "amd-stibp", "avx512vl", "xsaveopt", "mds-no", "popcnt", "clflushopt", "sse2", "xgetbv1", "rdseed", "pdpe1gb", "pschange-mc-no", "clwb", "avx512vnni", "mca", "tsx-ctrl", "tsc_adjust", "syscall", "pse36", "mmx", "avx512cd", "avx512bw", "pku", "tsc-deadline", "arch-capabilities", "avx512dq", "ssbd", "clflush", "mce", "ss", "pat", "bmi2", "lm", "ibrs", "sep", "md-clear"]} | 49 | 3419 | 60 | 0 | 0 | c1c1 | 2 | 192.168.2.99 | [["i686", "qemu", "hvm"], ["i686", "kvm", "hvm"], ["x86_64", "qemu", "hvm"], ["x86_64", "kvm", "hvm"]] | {"nova_object.name": "PciDevicePoolList", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"objects": []}, "nova_object.changes": ["objects"]} | [] | NULL | {"failed_builds": "0"} | {"nova_object.name": "NUMATopology", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"cells": [{"nova_object.name": "NUMACell", "nova_object.namespace": "nova", "nova_object.version": "1.5", "nova_object.data": {"id": 0, "cpuset": [0, 1, 2, 3], "pcpuset": [0, 1, 2, 3], "memory": 3931, "cpu_usage": 0, "memory_usage": 0, "pinned_cpus": [], "siblings": [[0], [1], [2], [3]], "mempages": [{"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 4, "total": 1006393, "used": 0, "reserved": 0}, "nova_object.changes": ["used", "total", "size_kb", "reserved"]}, {"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 2048, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["used", "total", "size_kb", "reserved"]}, {" nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 1048576, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["used", "total", "size_kb", "reserved"]}], "network_metadata": {"nova_object.name": "NetworkMetadata", "nova_object.namespace": "nova", "nova_object.version": "1.0", "nova_object.data": {"physnets": [], "tunneled": false}, "nova_object.changes": ["tunneled", "physnets"]}, "socket": null}, "nova_object.changes": ["memory_usage", "socket", "cpuset", "siblings", "id", "mempages", "pinned_cpus", "memory", "pcpuset", "network_metadata", "cpu_usage"]}]}, "nova_object.changes": ["cells"]} | c1c1 | 1.5 | 16 | 1eac1c8d-d96a-4eeb-9868-5a341a80c6df | 1 | 0 | | 2023-02-07 08:25:27 | 2023-02-07 08:25:27 | 2023-02-13 08:34:22 | 3 | NULL | 12 | 31890 | 456 | 0 | 512 | 0 | QEMU | 4002001 | {"arch": "x86_64", "model": "Broadwell-noTSX-IBRS", "vendor": "Intel", "topology": {"cells": 1, "sockets": 1, "cores": 6, "threads": 2}, "features": ["sha-ni", "intel-pt", "pat", "monitor", "movbe", "nx", "msr", "avx2", "md-clear", "popcnt", "rdseed", "pse36", "mds-no", "ds", "sse", "fsrm", "rdctl-no", "pse", "dtes64", "ds_cpl", "xgetbv1", "lahf_lm", "smep", "waitpkg", "smap", "fsgsbase", "sep", "tsc_adjust", "cmov", "ibrs-all", "mtrr", "cx16", "f16c", "arch-capabilities", "pclmuldq", "clflush", "erms", "umip", "xsaves", "xsavec", "ssse3", "acpi", "tsc", "movdir64b", "vpclmulqdq", "skip-l1dfl-vmentry", "xsave", "arat", "mmx", "rdpid", "sse2", "ssbd", "pdpe1gb", "spec-ctrl", "adx", "pcid", "de", "pku", "est", "pae", "tsc-deadline", "pdcm", "clwb", "vme", "rdtscp", "fxsr", "3dnowprefetch", "invpcid", "x2apic", "tm", "lm", "fma", "bmi1", "sse4.1", "abm", "xsaveopt", "pschange-mc-no", "syscall", "clflushopt", "pbe", "avx", "cx8", "vmx", "gfni", "fpu", "mce", "tm2", "movdiri", "invtsc", "apic", "bmi2", "mca", "pge", "rdrand", "xtpr", "sse4.2", "stibp", "ht", "ss", "pni", "vaes", "aes"]} | 416 | 31378 | 456 | 0 | 0 | c-MS-7D42 | 3 | 192.168.2.99 | [["alpha", "qemu", "hvm"], ["armv7l", "qemu", "hvm"], ["aarch64", "qemu", "hvm"], ["cris", "qemu", "hvm"], ["i686", "qemu", "hvm"], ["i686", "kvm", "hvm"], ["lm32", "qemu", "hvm"], ["m68k", "qemu", "hvm"], ["microblaze", "qemu", "hvm"], ["microblazeel", "qemu", "hvm"], ["mips", "qemu", "hvm"], ["mipsel", "qemu", "hvm"], ["mips64", "qemu", "hvm"], ["mips64el", "qemu", "hvm"], ["ppc", "qemu", "hvm"], ["ppc64", "qemu", "hvm"], ["ppc64le", "qemu", "hvm"], ["s390x", "qemu", "hvm"], ["sh4", "qemu", "hvm"], ["sh4eb", "qemu", "hvm"], ["sparc", "qemu", "hvm"], ["sparc64", "qemu", "hvm"], ["unicore32", "qemu", "hvm"], ["x86_64", "qemu", "hvm"], ["x86_64", "kvm", "hvm"], ["xtensa", "qemu", "hvm"], ["xtensaeb", "qemu", "hvm"]] | {" nova_object.name": "PciDevicePoolList", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"objects": []}, "nova_object.changes": ["objects"]} | [] | NULL | {"failed_builds": "0"} | {"nova_object.name": "NUMATopology", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"cells": [{"nova_object.name": "NUMACell", "nova_object.namespace": "nova", "nova_object.version": "1.5", "nova_object.data": {"id": 0, "cpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "pcpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "memory": 31890, "cpu_usage": 0, "memory_usage": 0, "pinned_cpus": [], "siblings": [[0, 1], [10, 11], [2, 3], [6, 7], [4, 5], [8, 9]], "mempages": [{"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 4, "total": 8163866, "used": 0, "reserved": 0}, "nova_object.changes": ["total", "reserved", "used", "size_kb"]}, {"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 2048, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["total", "reserved", "used", "size_kb"]}, {" nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 1048576, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["total", "reserved", "used", "size_kb"]}], "network_metadata": {"nova_object.name": "NetworkMetadata", "nova_object.namespace": "nova", "nova_object.version": "1.0", "nova_object.data": {"physnets": [], "tunneled": false}, "nova_object.changes": ["physnets", "tunneled"]}, "socket": 0}, "nova_object.changes": ["network_metadata", "cpuset", "mempages", "id", "socket", "cpu_usage", "memory", "pinned_cpus", "pcpuset", "siblings", "memory_usage"]}]}, "nova_object.changes": ["cells"]} | c-MS-7D42 | 1.5 | 16 | f115a1c2-fda3-42c6-945a-8b54fef40daf | 1 | 0 | | 2023-02-07 09:53:12 | 2023-02-13 08:38:04 | 2023-02-13 08:39:33 | 4 | NULL | 12 | 31890 | 456 | 0 | 512 | 0 | QEMU | 4002001 | {"arch": "x86_64", "model": "Broadwell-noTSX-IBRS", "vendor": "Intel", "topology": {"cells": 1, "sockets": 1, "cores": 6, "threads": 2}, "features": ["rdctl-no", "acpi", "umip", "invpcid", "bmi1", "clflushopt", "pclmuldq", "movdir64b", "ssbd", "apic", "rdpid", "ht", "fsrm", "pni", "pse", "xsaves", "cx16", "nx", "f16c", "arat", "popcnt", "mtrr", "vpclmulqdq", "intel-pt", "spec-ctrl", "syscall", "3dnowprefetch", "ds", "mce", "bmi2", "tm2", "md-clear", "fpu", "monitor", "pae", "erms", "dtes64", "tsc", "fsgsbase", "xgetbv1", "est", "mds-no", "tm", "x2apic", "xsavec", "cx8", "stibp", "clflush", "ssse3", "pge", "movdiri", "pdpe1gb", "vaes", "gfni", "mmx", "clwb", "waitpkg", "xsaveopt", "pse36", "aes", "pschange-mc-no", "sse2", "abm", "ss", "pcid", "sep", "rdseed", "mca", "skip-l1dfl-vmentry", "pat", "smap", "sse", "lahf_lm", "avx", "cmov", "sse4.1", "sse4.2", "ibrs-all", "smep", "vme", "tsc_adjust", "arch-capabilities", "fma", "movbe", "adx", "avx2", "xtpr", "pku", "pbe", "rdrand", "tsc-deadline", "pdcm", "ds_cpl", "de", "invtsc", "xsave", "msr", "fxsr", "lm", "vmx", "sha-ni", "rdtscp"]} | 416 | 31378 | 456 | 0 | 0 | c-MS-7D42 | 4 | 192.168.28.21 | [["alpha", "qemu", "hvm"], ["armv7l", "qemu", "hvm"], ["aarch64", "qemu", "hvm"], ["cris", "qemu", "hvm"], ["i686", "qemu", "hvm"], ["i686", "kvm", "hvm"], ["lm32", "qemu", "hvm"], ["m68k", "qemu", "hvm"], ["microblaze", "qemu", "hvm"], ["microblazeel", "qemu", "hvm"], ["mips", "qemu", "hvm"], ["mipsel", "qemu", "hvm"], ["mips64", "qemu", "hvm"], ["mips64el", "qemu", "hvm"], ["ppc", "qemu", "hvm"], ["ppc64", "qemu", "hvm"], ["ppc64le", "qemu", "hvm"], ["s390x", "qemu", "hvm"], ["sh4", "qemu", "hvm"], ["sh4eb", "qemu", "hvm"], ["sparc", "qemu", "hvm"], ["sparc64", "qemu", "hvm"], ["unicore32", "qemu", "hvm"], ["x86_64", "qemu", "hvm"], ["x86_64", "kvm", "hvm"], ["xtensa", "qemu", "hvm"], ["xtensaeb", "qemu", "hvm"]] | {" nova_object.name": "PciDevicePoolList", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"objects": []}, "nova_object.changes": ["objects"]} | [] | NULL | {"failed_builds": "0"} | {"nova_object.name": "NUMATopology", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"cells": [{"nova_object.name": "NUMACell", "nova_object.namespace": "nova", "nova_object.version": "1.5", "nova_object.data": {"id": 0, "cpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "pcpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "memory": 31890, "cpu_usage": 0, "memory_usage": 0, "pinned_cpus": [], "siblings": [[0, 1], [10, 11], [2, 3], [6, 7], [4, 5], [8, 9]], "mempages": [{"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 4, "total": 8163866, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "total", "used", "reserved"]}, {"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 2048, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "total", "used", "reserved"]}, {" nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 1048576, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "total", "used", "reserved"]}], "network_metadata": {"nova_object.name": "NetworkMetadata", "nova_object.namespace": "nova", "nova_object.version": "1.0", "nova_object.data": {"physnets": [], "tunneled": false}, "nova_object.changes": ["physnets", "tunneled"]}, "socket": 0}, "nova_object.changes": ["siblings", "cpuset", "mempages", "socket", "pcpuset", "memory", "memory_usage", "id", "network_metadata", "cpu_usage", "pinned_cpus"]}]}, "nova_object.changes": ["cells"]} | c1c2 | 1.5 | 16 | 10ea8254-ad84-4db9-9acd-5c783cb8600e | 1 | 0 | | 2023-02-13 08:41:21 | 2023-02-13 08:41:22 | 2023-02-13 09:56:50 | 5 | NULL | 12 | 31890 | 456 | 0 | 512 | 0 | QEMU | 4002001 | {"arch": "x86_64", "model": "Broadwell-noTSX-IBRS", "vendor": "Intel", "topology": {"cells": 1, "sockets": 1, "cores": 6, "threads": 2}, "features": ["bmi2", "ht", "pae", "pku", "monitor", "avx2", "sha-ni", "acpi", "ssbd", "syscall", "mca", "mmx", "mds-no", "erms", "fsrm", "arat", "xsaves", "movbe", "movdir64b", "fpu", "clflush", "nx", "mce", "pse", "cx8", "aes", "avx", "xsavec", "invpcid", "est", "xgetbv1", "fxsr", "rdrand", "vaes", "cmov", "intel-pt", "smep", "dtes64", "f16c", "adx", "sse2", "stibp", "rdseed", "xsave", "skip-l1dfl-vmentry", "sse4.1", "rdpid", "ds", "umip", "pni", "rdctl-no", "clwb", "md-clear", "pschange-mc-no", "msr", "popcnt", "sse4.2", "pge", "tm2", "pat", "xtpr", "fma", "gfni", "sep", "ibrs-all", "tsc", "ds_cpl", "tm", "clflushopt", "pcid", "de", "rdtscp", "vme", "cx16", "lahf_lm", "ss", "pdcm", "x2apic", "pbe", "movdiri", "tsc-deadline", "invtsc", "apic", "fsgsbase", "mtrr", "vpclmulqdq", "ssse3", "3dnowprefetch", "abm", "xsaveopt", "tsc_adjust", "pse36", "pclmuldq", "bmi1", "smap", "arch-capabilities", "lm", "vmx", "sse", "pdpe1gb", "spec-ctrl", "waitpkg"]} | 416 | 31378 | 456 | 0 | 0 | c-MS-7D42 | 5 | 192.168.28.21 | [["alpha", "qemu", "hvm"], ["armv7l", "qemu", "hvm"], ["aarch64", "qemu", "hvm"], ["cris", "qemu", "hvm"], ["i686", "qemu", "hvm"], ["i686", "kvm", "hvm"], ["lm32", "qemu", "hvm"], ["m68k", "qemu", "hvm"], ["microblaze", "qemu", "hvm"], ["microblazeel", "qemu", "hvm"], ["mips", "qemu", "hvm"], ["mipsel", "qemu", "hvm"], ["mips64", "qemu", "hvm"], ["mips64el", "qemu", "hvm"], ["ppc", "qemu", "hvm"], ["ppc64", "qemu", "hvm"], ["ppc64le", "qemu", "hvm"], ["s390x", "qemu", "hvm"], ["sh4", "qemu", "hvm"], ["sh4eb", "qemu", "hvm"], ["sparc", "qemu", "hvm"], ["sparc64", "qemu", "hvm"], ["unicore32", "qemu", "hvm"], ["x86_64", "qemu", "hvm"], ["x86_64", "kvm", "hvm"], ["xtensa", "qemu", "hvm"], ["xtensaeb", "qemu", "hvm"]] | {"nova_object.name": "PciDevicePoolList", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"objects": []}, "nova_object.changes": ["objects"]} | [] | NULL | {"failed_builds": "0"} | {"nova_object.name": "NUMATopology", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"cells": [{"nova_object.name": "NUMACell", "nova_object.namespace": "nova", "nova_object.version": "1.5", "nova_object.data": {"id": 0, "cpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "pcpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "memory": 31890, "cpu_usage": 0, "memory_usage": 0, "pinned_cpus": [], "siblings": [[0, 1], [10, 11], [2, 3], [6, 7], [4, 5], [8, 9]], "mempages": [{"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 4, "total": 8163866, "used": 0, "reserved": 0}, "nova_object.changes": ["used", "size_kb", "total", "reserved"]}, {"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 2048, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["used", "size_kb", "total", "reserved"]}, {" nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 1048576, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["used", "size_kb", "total", "reserved"]}], "network_metadata": {"nova_object.name": "NetworkMetadata", "nova_object.namespace": "nova", "nova_object.version": "1.0", "nova_object.data": {"physnets": [], "tunneled": false}, "nova_object.changes": ["tunneled", "physnets"]}, "socket": 0}, "nova_object.changes": ["pinned_cpus", "cpuset", "memory_usage", "id", "cpu_usage", "network_metadata", "siblings", "mempages", "socket", "memory", "pcpuset"]}]}, "nova_object.changes": ["cells"]} | c1c2 | 1.5 | 16 | 8efa100f-ab14-45fd-8c39-644b49772883 | 1 | 0 | | 2023-02-13 09:57:30 | 2023-02-13 09:57:31 | 2023-02-13 13:52:57 | 6 | NULL | 12 | 31890 | 456 | 0 | 512 | 0 | QEMU | 4002001 | {"arch": "x86_64", "model": "Broadwell-noTSX-IBRS", "vendor": "Intel", "topology": {"cells": 1, "sockets": 1, "cores": 6, "threads": 2}, "features": ["rdpid", "intel-pt", "fxsr", "pclmuldq", "xsaveopt", "pae", "xsave", "movdiri", "syscall", "ibrs-all", "mmx", "tsc_adjust", "abm", "ssbd", "sse", "mce", "clwb", "vmx", "dtes64", "ssse3", "fsrm", "est", "bmi1", "mtrr", "avx2", "pse36", "pat", "gfni", "mds-no", "clflushopt", "cmov", "fma", "sep", "mca", "ss", "umip", "popcnt", "skip-l1dfl-vmentry", "ht", "sha-ni", "pdcm", "pdpe1gb", "rdrand", "pge", "lahf_lm", "aes", "xsavec", "pni", "smep", "md-clear", "waitpkg", "tm", "xgetbv1", "stibp", "apic", "vaes", "fpu", "ds_cpl", "ds", "sse4.2", "3dnowprefetch", "smap", "x2apic", "vpclmulqdq", "acpi", "avx", "de", "pbe", "sse2", "xsaves", "monitor", "clflush", "tm2", "pschange-mc-no", "bmi2", "movbe", "pku", "pcid", "xtpr", "erms", "movdir64b", "cx8", "nx", "rdctl-no", "invpcid", "spec-ctrl", "tsc", "adx", "invtsc", "f16c", "rdtscp", "vme", "pse", "lm", "cx16", "fsgsbase", "rdseed", "msr", "sse4.1", "arch-capabilities", "arat", "tsc-deadline"]} | 416 | 31378 | 456 | 0 | 0 | c-MS-7D42 | 6 | 192.168.28.21 | [["alpha", "qemu", "hvm"], ["armv7l", "qemu", "hvm"], ["aarch64", "qemu", "hvm"], ["cris", "qemu", "hvm"], ["i686", "qemu", "hvm"], ["i686", "kvm", "hvm"], ["lm32", "qemu", "hvm"], ["m68k", "qemu", "hvm"], ["microblaze", "qemu", "hvm"], ["microblazeel", "qemu", "hvm"], ["mips", "qemu", "hvm"], ["mipsel", "qemu", "hvm"], ["mips64", "qemu", "hvm"], ["mips64el", "qemu", "hvm"], ["ppc", "qemu", "hvm"], ["ppc64", "qemu", "hvm"], ["ppc64le", "qemu", "hvm"], ["s390x", "qemu", "hvm"], ["sh4", "qemu", "hvm"], ["sh4eb", "qemu", "hvm"], ["sparc", "qemu", "hvm"], ["sparc64", "qemu", "hvm"], ["unicore32", "qemu", "hvm"], ["x86_64", "qemu", "hvm"], ["x86_64", "kvm", "hvm"], ["xtensa", "qemu", "hvm"], ["xtensaeb", "qemu", "hvm"]] | {" nova_object.name": "PciDevicePoolList", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"objects": []}, "nova_object.changes": ["objects"]} | [] | NULL | {"failed_builds": "0"} | {"nova_object.name": "NUMATopology", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"cells": [{"nova_object.name": "NUMACell", "nova_object.namespace": "nova", "nova_object.version": "1.5", "nova_object.data": {"id": 0, "cpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "pcpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "memory": 31890, "cpu_usage": 0, "memory_usage": 0, "pinned_cpus": [], "siblings": [[0, 1], [10, 11], [2, 3], [6, 7], [4, 5], [8, 9]], "mempages": [{"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 4, "total": 8163866, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "total", "reserved"]}, {"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 2048, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "total", "reserved"]}, {" nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 1048576, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "total", "reserved"]}], "network_metadata": {"nova_object.name": "NetworkMetadata", "nova_object.namespace": "nova", "nova_object.version": "1.0", "nova_object.data": {"physnets": [], "tunneled": false}, "nova_object.changes": ["physnets", "tunneled"]}, "socket": 0}, "nova_object.changes": ["memory_usage", "id", "mempages", "pinned_cpus", "network_metadata", "pcpuset", "cpuset", "siblings", "socket", "cpu_usage", "memory"]}]}, "nova_object.changes": ["cells"]} | c1c2 | 1.5 | 16 | 8f5b58c5-d5d7-452c-9ec7-cff24baf6c94 | 1 | 0 | | 2023-02-14 01:35:43 | 2023-02-14 01:35:43 | 2023-02-14 03:16:51 | 7 | NULL | 12 | 31890 | 456 | 0 | 512 | 0 | QEMU | 4002001 | {"arch": "x86_64", "model": "Broadwell-noTSX-IBRS", "vendor": "Intel", "topology": {"cells": 1, "sockets": 1, "cores": 6, "threads": 2}, "features": ["pcid", "pse36", "movdir64b", "apic", "nx", "vpclmulqdq", "mtrr", "popcnt", "pdcm", "fsgsbase", "lahf_lm", "sse2", "pae", "aes", "movdiri", "xsaves", "erms", "invtsc", "waitpkg", "pbe", "ht", "pni", "avx2", "rdpid", "fxsr", "tm2", "pku", "x2apic", "fma", "pge", "rdseed", "pdpe1gb", "mmx", "sse4.1", "sha-ni", "xtpr", "tsc_adjust", "cx16", "xsave", "cx8", "mce", "md-clear", "gfni", "clwb", "msr", "abm", "f16c", "ss", "xsaveopt", "ds_cpl", "pse", "syscall", "cmov", "3dnowprefetch", "ssse3", "pclmuldq", "arch-capabilities", "ibrs-all", "arat", "ds", "pat", "invpcid", "vaes", "xsavec", "mds-no", "tm", "smep", "acpi", "fsrm", "movbe", "fpu", "sse4.2", "umip", "rdtscp", "tsc-deadline", "skip-l1dfl-vmentry", "est", "rdctl-no", "clflush", "spec-ctrl", "tsc", "lm", "avx", "vmx", "clflushopt", "rdrand", "dtes64", "smap", "ssbd", "sse", "xgetbv1", "stibp", "mca", "adx", "vme", "bmi1", "pschange-mc-no", "intel-pt", "de", "monitor", "bmi2", "sep"]} | 416 | 31378 | 456 | 0 | 0 | c-MS-7D42 | 7 | 192.168.28.21 | [["alpha", "qemu", "hvm"], ["armv7l", "qemu", "hvm"], ["aarch64", "qemu", "hvm"], ["cris", "qemu", "hvm"], ["i686", "qemu", "hvm"], ["i686", "kvm", "hvm"], ["lm32", "qemu", "hvm"], ["m68k", "qemu", "hvm"], ["microblaze", "qemu", "hvm"], ["microblazeel", "qemu", "hvm"], ["mips", "qemu", "hvm"], ["mipsel", "qemu", "hvm"], ["mips64", "qemu", "hvm"], ["mips64el", "qemu", "hvm"], ["ppc", "qemu", "hvm"], ["ppc64", "qemu", "hvm"], ["ppc64le", "qemu", "hvm"], ["s390x", "qemu", "hvm"], ["sh4", "qemu", "hvm"], ["sh4eb", "qemu", "hvm"], ["sparc", "qemu", "hvm"], ["sparc64", "qemu", "hvm"], ["unicore32", "qemu", "hvm"], ["x86_64", "qemu", "hvm"], ["x86_64", "kvm", "hvm"], ["xtensa", "qemu", "hvm"], ["xtensaeb", "qemu", "hvm"]] | {"nova_object.name": "PciDevicePoolList", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"objects": []}, "nova_object.changes": ["objects"]} | [] | NULL | {"failed_builds": "0"} | {"nova_object.name": "NUMATopology", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"cells": [{"nova_object.name": "NUMACell", "nova_object.namespace": "nova", "nova_object.version": "1.5", "nova_object.data": {"id": 0, "cpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "pcpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "memory": 31890, "cpu_usage": 0, "memory_usage": 0, "pinned_cpus": [], "siblings": [[0, 1], [10, 11], [2, 3], [6, 7], [4, 5], [8, 9]], "mempages": [{"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 4, "total": 8163866, "used": 0, "reserved": 0}, "nova_object.changes": ["reserved", "total", "used", "size_kb"]}, {"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 2048, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["reserved", "total", "used", "size_kb"]}, {" nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 1048576, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["reserved", "total", "used", "size_kb"]}], "network_metadata": {"nova_object.name": "NetworkMetadata", "nova_object.namespace": "nova", "nova_object.version": "1.0", "nova_object.data": {"physnets": [], "tunneled": false}, "nova_object.changes": ["tunneled", "physnets"]}, "socket": 0}, "nova_object.changes": ["pcpuset", "mempages", "socket", "cpu_usage", "memory_usage", "siblings", "memory", "pinned_cpus", "network_metadata", "cpuset", "id"]}]}, "nova_object.changes": ["cells"]} | c1c2 | 1.5 | 16 | 9f81894d-028d-4bb0-ada0-116159b13ed8 | 1 | 0 | | 2023-02-14 03:19:40 | 2023-02-28 07:30:14 | NULL | 8 | NULL | 12 | 31890 | 456 | 0 | 512 | 0 | QEMU | 4002001 | {"arch": "x86_64", "model": "Broadwell-noTSX-IBRS", "vendor": "Intel", "topology": {"cells": 1, "sockets": 1, "cores": 6, "threads": 2}, "features": ["ds_cpl", "vpclmulqdq", "pbe", "lahf_lm", "clflush", "xsaveopt", "bmi2", "clwb", "skip-l1dfl-vmentry", "de", "cmov", "pku", "acpi", "xsavec", "3dnowprefetch", "mmx", "mtrr", "vmx", "movbe", "ssse3", "adx", "syscall", "spec-ctrl", "abm", "umip", "fxsr", "pge", "vme", "xsave", "rdctl-no", "rdrand", "sse", "cx8", "gfni", "dtes64", "invpcid", "md-clear", "fsrm", "fma", "arat", "rdtscp", "tsc", "sse4.1", "xsaves", "est", "tm", "ibrs-all", "stibp", "fsgsbase", "cx16", "arch-capabilities", "ssbd", "pse", "avx", "pse36", "pdpe1gb", "tm2", "nx", "pdcm", "sse4.2", "erms", "sep", "aes", "mca", "invtsc", "bmi1", "rdseed", "ss", "ht", "x2apic", "movdir64b", "rdpid", "lm", "tsc-deadline", "msr", "mce", "smep", "avx2", "sse2", "pschange-mc-no", "intel-pt", "tsc_adjust", "ds", "monitor", "popcnt", "f16c", "pae", "movdiri", "xgetbv1", "pcid", "pni", "pclmuldq", "xtpr", "apic", "mds-no", "sha-ni", "smap", "vaes", "waitpkg", "pat", "fpu", "clflushopt"]} | 415 | 31378 | 456 | 0 | 0 | c1c2 | 0 | 192.168.28.21 | [["alpha", "qemu", "hvm"], ["armv7l", "qemu", "hvm"], ["aarch64", "qemu", "hvm"], ["cris", "qemu", "hvm"], ["i686", "qemu", "hvm"], ["i686", "kvm", "hvm"], ["lm32", "qemu", "hvm"], ["m68k", "qemu", "hvm"], ["microblaze", "qemu", "hvm"], ["microblazeel", "qemu", "hvm"], ["mips", "qemu", "hvm"], ["mipsel", "qemu", "hvm"], ["mips64", "qemu", "hvm"], ["mips64el", "qemu", "hvm"], ["ppc", "qemu", "hvm"], ["ppc64", "qemu", "hvm"], ["ppc64le", "qemu", "hvm"], ["s390x", "qemu", "hvm"], ["sh4", "qemu", "hvm"], ["sh4eb", "qemu", "hvm"], ["sparc", "qemu", "hvm"], ["sparc64", "qemu", "hvm"], ["unicore32", "qemu", "hvm"], ["x86_64", "qemu", "hvm"], ["x86_64", "kvm", "hvm"], ["xtensa", "qemu", "hvm"], ["xtensaeb", "qemu", "hvm"]] | {"nova_object.name": "PciDevicePoolList", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"objects": []}, "nova_object.changes": ["objects"]} | [] | NULL | {"failed_builds": "0"} | {"nova_object.name": "NUMATopology", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"cells": [{"nova_object.name": "NUMACell", "nova_object.namespace": "nova", "nova_object.version": "1.5", "nova_object.data": {"id": 0, "cpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "pcpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "memory": 31890, "cpu_usage": 0, "memory_usage": 0, "pinned_cpus": [], "siblings": [[0, 1], [10, 11], [2, 3], [6, 7], [4, 5], [8, 9]], "mempages": [{"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 4, "total": 8163962, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "reserved", "total", "used"]}, {"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 2048, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "reserved", "total", "used"]}, {" nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 1048576, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "reserved", "total", "used"]}], "network_metadata": {"nova_object.name": "NetworkMetadata", "nova_object.namespace": "nova", "nova_object.version": "1.0", "nova_object.data": {"physnets": [], "tunneled": false}, "nova_object.changes": ["tunneled", "physnets"]}, "socket": 0}, "nova_object.changes": ["cpu_usage", "siblings", "id", "pcpuset", "cpuset", "network_metadata", "memory", "pinned_cpus", "memory_usage", "socket", "mempages"]}]}, "nova_object.changes": ["cells"]} | c1c2 | 1.5 | 16 | c360cc82-f0fd-4662-bccd-e1f02b27af51 | 1 | 0 | +---------------------+---------------------+---------------------+----+------------+-------+-----------+----------+------------+----------------+---------------+-----------------+--------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------+-------------+--------------+------------------+-------------+---------------------+---------+---------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+-----------------+------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+----------------------+----------------------+--------------------------------------+-----------------------+--------+ 8 rows in set (0.001 sec) ``` Then I search code, I found these infomation is upload in compute node's nova-compute, in the init process in nova/compute/resource_tracker.py:_init_compute_node Then I add some debug log, I found in _init_compute_node, there are PCI configure which read from /etc/nova/nova.conf, but in _setup_pci_tracker pop these infomation, this is code: ``` def _setup_pci_tracker(self, context, compute_node, resources): if not self.pci_tracker: LOG.debug("_setup_pci_tracker 11111") LOG.debug(resources) # 2023-02-28 14:01:33.782 443293 DEBUG nova.compute.resource_tracker [req-52743217-f0b6-4525-b9fd-371246b6c929 - - - - -] _setup_pci_tracker 11111 _setup_pci_tracker /usr/lib/python3/dist-packages/nova/compute/resource_tracker.py:772 # 2023-02-28 14:01:33.782 443293 DEBUG nova.compute.resource_tracker [req-52743217-f0b6-4525-b9fd-371246b6c929 - - - - -] {'supported_instances': [('alpha', 'qemu', 'hvm'), ('armv7l', 'qemu', 'hvm'), ('aarch64', 'qemu', 'hvm'), ('cris', 'qemu', 'hvm'), ('i686', 'qemu', 'hvm'), ('i686', 'kvm', 'hvm'), ('lm32', 'qemu', 'hvm'), ('m68k', 'qemu', 'hvm'), ('microblaze', 'qemu', 'hvm'), ('microblazeel', 'qemu', 'hvm'), ('mips', 'qemu', 'hvm'), ('mipsel', 'qemu', 'hvm'), ('mips64', 'qemu', 'hvm'), ('mips64el', 'qemu', 'hvm'), ('ppc', 'qemu', 'hvm'), ('ppc64', 'qemu', 'hvm'), ('ppc64le', 'qemu', 'hvm'), ('s390x', 'qemu', 'hvm'), ('sh4', 'qemu', 'hvm'), ('sh4eb', 'qemu', 'hvm'), ('sparc', 'qemu', 'hvm'), ('sparc64', 'qemu', 'hvm'), ('unicore32', 'qemu', 'hvm'), ('x86_64', 'qemu', 'hvm'), ('x86_64', 'kvm', 'hvm'), ('xtensa', 'qemu', 'hvm'), ('xtensaeb', 'qemu', 'hvm')], 'vcpus': 12, 'memory_mb': 31890, 'local_gb': 456.8854446411133, 'vcpus_used': 0, 'memory_mb_used': 3856, 'local_gb_used': 18.015335083007812, 'hypervisor_type': 'QEMU', 'hypervisor_version': 4002001, 'hypervisor_hostname': 'c1c2', 'cpu_info': '{"arch": "x86_64", "model": "Broadwell-noTSX-IBRS", "vendor": "Intel", "topology": {"cells": 1, "sockets": 1, "cores": 6, "threads": 2}, "features": ["sse4.2", "mds-no", "stibp", "pdpe1gb", "xsaveopt", "ht", "intel-pt", "mtrr", "abm", "tm", "lm", "umip", "mca", "pku", "ds_cpl", "rdrand", "adx", "rdseed", "lahf_lm", "xgetbv1", "nx", "invpcid", "rdtscp", "tsc", "xsavec", "pcid", "arch-capabilities", "pclmuldq", "spec-ctrl", "fsgsbase", "avx2", "md-clear", "vmx", "syscall", "mmx", "ds", "ssse3", "avx", "dtes64", "fxsr", "msr", "acpi", "vpclmulqdq", "smap", "erms", "pge", "cmov", "sha-ni", "fsrm", "x2apic", "xsaves", "cx8", "pse", "pse36", "clflushopt", "vaes", "pni", "ssbd", "movdiri", "movbe", "clwb", "xtpr", "de", "invtsc", "fpu", "tsc-deadline", "pae", "clflush", "ibrs-all", "waitpkg", "sse", "sse2", "bmi1", "3dnowprefetch", "cx16", "popcnt", "rdctl-no", "fma", "tsc_adjust", "xsave", "ss", "skip-l1dfl-vmentry", "sse4.1", "rdpid", "monitor", "vme", "tm2", "pat", "pschange-mc-no", "movdir64b", "gfni", "mce", "smep", "sep", "apic", "arat", "f16c", "bmi2", "aes", "pbe", "est", "pdcm"]}', 'disk_available_least': 415.5914993286133, 'pci_passthrough_devices': '[{"dev_id": "pci_0000_02_00_0", "address": "0000:02:00.0", "product_id": "a809", "vendor_id": "144d", "numa_node": 0, "label": "label_144d_a809", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_01_00_5", "address": "0000:01:00.5", "product_id": "101e", "vendor_id": "15b3", "numa_node": null, "label": "label_15b3_101e", "dev_type": "type-VF", "parent_addr": "0000:01:00.0", "parent_ifname": "enp1s0f0np0", "capabilities": {"network": ["rx", "tx", "sg", "tso", "gso", "gro", "rxvlan", "txvlan", "rxhash", "rdma", "txudptnl"]}}, {"dev_id": "pci_0000_00_1c_0", "address": "0000:00:1c.0", "product_id": "7ab8", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7ab8", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_08_0", "address": "0000:00:08.0", "product_id": "464f", "vendor_id": "8086", "numa_node": null, "label": "label_8086_464f", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_1f_4", "address": "0000:00:1f.4", "product_id": "7aa3", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7aa3", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_01_00_3", "address": "0000:01:00.3", "product_id": "101e", "vendor_id": "15b3", "numa_node": null, "label": "label_15b3_101e", "dev_type": "type-VF", "parent_addr": "0000:01:00.0", "parent_ifname": "enp1s0f0np0", "capabilities": {"network": ["rx", "tx", "sg", "tso", "gso", "gro", "rxvlan", "txvlan", "rxhash", "rdma", "txudptnl"]}}, {"dev_id": "pci_0000_00_06_0", "address": "0000:00:06.0", "product_id": "464d", "vendor_id": "8086", "numa_node": null, "label": "label_8086_464d", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "7ae2", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7ae2", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_1f_5", "address": "0000:00:1f.5", "product_id": "7aa4", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7aa4", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_01_00_6", "address": "0000:01:00.6", "product_id": "101e", "vendor_id": "15b3", "numa_node": null, "label": "label_15b3_101e", "dev_type": "type-VF", "parent_addr": "0000:01:00.0", "parent_ifname": "enp1s0f0np0", "capabilities": {"network": ["rx", "tx", "sg", "tso", "gso", "gro", "rxvlan", "txvlan", "rxhash", "rdma", "txudptnl"]}}, {"dev_id": "pci_0000_00_1f_0", "address": "0000:00:1f.0", "product_id": "7a86", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7a86", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_02_0", "address": "0000:00:02.0", "product_id": "4692", "vendor_id": "8086", "numa_node": null, "label": "label_8086_4692", "dev_type": "type-PF"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "7ae8", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7ae8", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_1c_1", "address": "0000:00:1c.1", "product_id": "7ab9", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7ab9", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_1c_3", "address": "0000:00:1c.3", "product_id": "7abb", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7abb", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_04_00_0", "address": "0000:04:00.0", "product_id": "8125", "vendor_id": "10ec", "numa_node": null, "label": "label_10ec_8125", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "4650", "vendor_id": "8086", "numa_node": null, "label": "label_8086_4650", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "460d", "vendor_id": "8086", "numa_node": null, "label": "label_8086_460d", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_01_00_0", "address": "0000:01:00.0", "product_id": "a2d6", "vendor_id": "15b3", "numa_node": null, "label": "label_15b3_a2d6", "dev_type": "type-PF", "mac_address": "08:c0:eb:8e:bd:f4"}, {"dev_id": "pci_0000_01_00_2", "address": "0000:01:00.2", "product_id": "c2d3", "vendor_id": "15b3", "numa_node": null, "label": "label_15b3_c2d3", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_14_2", "address": "0000:00:14.2", "product_id": "7aa7", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7aa7", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_14_3", "address": "0000:00:14.3", "product_id": "7af0", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7af0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_1f_3", "address": "0000:00:1f.3", "product_id": "7ad0", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7ad0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_01_00_4", "address": "0000:01:00.4", "product_id": "101e", "vendor_id": "15b3", "numa_node": null, "label": "label_15b3_101e", "dev_type": "type-VF", "parent_addr": "0000:01:00.0", "parent_ifname": "enp1s0f0np0", "capabilities": {"network": ["rx", "tx", "sg", "tso", "gso", "gro", "rxvlan", "txvlan", "rxhash", "rdma", "txudptnl"]}}, {"dev_id": "pci_0000_05_00_0", "address": "0000:05:00.0", "product_id": "0612", "vendor_id": "1b21", "numa_node": null, "label": "label_1b21_0612", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_01_00_1", "address": "0000:01:00.1", "product_id": "a2d6", "vendor_id": "15b3", "numa_node": null, "label": "label_15b3_a2d6", "dev_type": "type-PF", "mac_address": "08:c0:eb:8e:bd:f5"}, {"dev_id": "pci_0000_00_14_0", "address": "0000:00:14.0", "product_id": "7ae0", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7ae0", "dev_type": "type-PCI"}]', 'numa_topology': '{"nova_object.name": "NUMATopology", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"cells": [{"nova_object.name": "NUMACell", "nova_object.namespace": "nova", "nova_object.version": "1.5", "nova_object.data": {"id": 0, "cpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "pcpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "memory": 31890, "cpu_usage": 0, "memory_usage": 0, "pinned_cpus": [], "siblings": [[0, 1], [10, 11], [2, 3], [6, 7], [4, 5], [8, 9]], "mempages": [{"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 4, "total": 8163962, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "reserved", "total"]}, {"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 2048, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "reserved", "total"]}, {" nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 1048576, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "reserved", "total"]}], "network_metadata": {"nova_object.name": "NetworkMetadata", "nova_object.namespace": "nova", "nova_object.version": "1.0", "nova_object.data": {"physnets": [], "tunneled": false}, "nova_object.changes": ["physnets", "tunneled"]}, "socket": 0}, "nova_object.changes": ["cpuset", "memory_usage", "cpu_usage", "id", "pinned_cpus", "pcpuset", "socket", "network_metadata", "siblings", "mempages", "memory"]}]}, "nova_object.changes": ["cells"]}', 'host_ip': '192.168.28.21'} _setup_pci_tracker /usr/lib/python3/dist-packages/nova/compute/resource_tracker.py:773 self.pci_tracker = pci_manager.PciDevTracker(context, compute_node) if 'pci_passthrough_devices' in resources: dev_json = resources.pop('pci_passthrough_devices') self.pci_tracker.update_devices_from_hypervisor_resources( dev_json) LOG.debug("_setup_pci_tracker 22222") LOG.debug(resources) # 2023-02-28 14:01:33.798 443293 DEBUG nova.compute.resource_tracker [req-52743217-f0b6-4525-b9fd-371246b6c929 - - - - -] _setup_pci_tracker 22222 _setup_pci_tracker /usr/lib/python3/dist-packages/nova/compute/resource_tracker.py:781 # 2023-02-28 14:01:33.799 443293 DEBUG nova.compute.resource_tracker [req-52743217-f0b6-4525-b9fd-371246b6c929 - - - - -] {'supported_instances': [('alpha', 'qemu', 'hvm'), ('armv7l', 'qemu', 'hvm'), ('aarch64', 'qemu', 'hvm'), ('cris', 'qemu', 'hvm'), ('i686', 'qemu', 'hvm'), ('i686', 'kvm', 'hvm'), ('lm32', 'qemu', 'hvm'), ('m68k', 'qemu', 'hvm'), ('microblaze', 'qemu', 'hvm'), ('microblazeel', 'qemu', 'hvm'), ('mips', 'qemu', 'hvm'), ('mipsel', 'qemu', 'hvm'), ('mips64', 'qemu', 'hvm'), ('mips64el', 'qemu', 'hvm'), ('ppc', 'qemu', 'hvm'), ('ppc64', 'qemu', 'hvm'), ('ppc64le', 'qemu', 'hvm'), ('s390x', 'qemu', 'hvm'), ('sh4', 'qemu', 'hvm'), ('sh4eb', 'qemu', 'hvm'), ('sparc', 'qemu', 'hvm'), ('sparc64', 'qemu', 'hvm'), ('unicore32', 'qemu', 'hvm'), ('x86_64', 'qemu', 'hvm'), ('x86_64', 'kvm', 'hvm'), ('xtensa', 'qemu', 'hvm'), ('xtensaeb', 'qemu', 'hvm')], 'vcpus': 12, 'memory_mb': 31890, 'local_gb': 456.8854446411133, 'vcpus_used': 0, 'memory_mb_used': 3856, 'local_gb_used': 18.015335083007812, 'hypervisor_type': 'QEMU', 'hypervisor_version': 4002001, 'hypervisor_hostname': 'c1c2', 'cpu_info': '{"arch": "x86_64", "model": "Broadwell-noTSX-IBRS", "vendor": "Intel", "topology": {"cells": 1, "sockets": 1, "cores": 6, "threads": 2}, "features": ["sse4.2", "mds-no", "stibp", "pdpe1gb", "xsaveopt", "ht", "intel-pt", "mtrr", "abm", "tm", "lm", "umip", "mca", "pku", "ds_cpl", "rdrand", "adx", "rdseed", "lahf_lm", "xgetbv1", "nx", "invpcid", "rdtscp", "tsc", "xsavec", "pcid", "arch-capabilities", "pclmuldq", "spec-ctrl", "fsgsbase", "avx2", "md-clear", "vmx", "syscall", "mmx", "ds", "ssse3", "avx", "dtes64", "fxsr", "msr", "acpi", "vpclmulqdq", "smap", "erms", "pge", "cmov", "sha-ni", "fsrm", "x2apic", "xsaves", "cx8", "pse", "pse36", "clflushopt", "vaes", "pni", "ssbd", "movdiri", "movbe", "clwb", "xtpr", "de", "invtsc", "fpu", "tsc-deadline", "pae", "clflush", "ibrs-all", "waitpkg", "sse", "sse2", "bmi1", "3dnowprefetch", "cx16", "popcnt", "rdctl-no", "fma", "tsc_adjust", "xsave", "ss", "skip-l1dfl-vmentry", "sse4.1", "rdpid", "monitor", "vme", "tm2", "pat", "pschange-mc-no", "movdir64b", "gfni", "mce", "smep", "sep", "apic", "arat", "f16c", "bmi2", "aes", "pbe", "est", "pdcm"]}', 'disk_available_least': 415.5914993286133, 'numa_topology': '{"nova_object.name": "NUMATopology", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"cells": [{"nova_object.name": "NUMACell", "nova_object.namespace": "nova", "nova_object.version": "1.5", "nova_object.data": {"id": 0, "cpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "pcpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "memory": 31890, "cpu_usage": 0, "memory_usage": 0, "pinned_cpus": [], "siblings": [[0, 1], [10, 11], [2, 3], [6, 7], [4, 5], [8, 9]], "mempages": [{"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 4, "total": 8163962, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "reserved", "total"]}, {"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 2048, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "reserved", "total"]}, {" nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 1048576, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "reserved", "total"]}], "network_metadata": {"nova_object.name": "NetworkMetadata", "nova_object.namespace": "nova", "nova_object.version": "1.0", "nova_object.data": {"physnets": [], "tunneled": false}, "nova_object.changes": ["physnets", "tunneled"]}, "socket": 0}, "nova_object.changes": ["cpuset", "memory_usage", "cpu_usage", "id", "pinned_cpus", "pcpuset", "socket", "network_metadata", "siblings", "mempages", "memory"]}]}, "nova_object.changes": ["cells"]}', 'host_ip': '192.168.28.21'} _setup_pci_tracker /usr/lib/python3/dist-packages/nova/compute/resource_tracker.py:782 dev_pools_obj = self.pci_tracker.stats.to_device_pools_obj() LOG.debug("_setup_pci_tracker 44444") LOG.debug(dev_pools_obj) compute_node.pci_device_pools = dev_pools_obj LOG.debug("_setup_pci_tracker 33333") LOG.debug(resources) LOG.debug(compute_node) # 2023-02-28 14:01:33.799 443293 DEBUG nova.compute.resource_tracker [req-52743217-f0b6-4525-b9fd-371246b6c929 - - - - -] _setup_pci_tracker 33333 _setup_pci_tracker /usr/lib/python3/dist-packages/nova/compute/resource_tracker.py:786 # 2023-02-28 14:01:33.800 443293 DEBUG nova.compute.resource_tracker [req-52743217-f0b6-4525-b9fd-371246b6c929 - - - - -] {'supported_instances': [('alpha', 'qemu', 'hvm'), ('armv7l', 'qemu', 'hvm'), ('aarch64', 'qemu', 'hvm'), ('cris', 'qemu', 'hvm'), ('i686', 'qemu', 'hvm'), ('i686', 'kvm', 'hvm'), ('lm32', 'qemu', 'hvm'), ('m68k', 'qemu', 'hvm'), ('microblaze', 'qemu', 'hvm'), ('microblazeel', 'qemu', 'hvm'), ('mips', 'qemu', 'hvm'), ('mipsel', 'qemu', 'hvm'), ('mips64', 'qemu', 'hvm'), ('mips64el', 'qemu', 'hvm'), ('ppc', 'qemu', 'hvm'), ('ppc64', 'qemu', 'hvm'), ('ppc64le', 'qemu', 'hvm'), ('s390x', 'qemu', 'hvm'), ('sh4', 'qemu', 'hvm'), ('sh4eb', 'qemu', 'hvm'), ('sparc', 'qemu', 'hvm'), ('sparc64', 'qemu', 'hvm'), ('unicore32', 'qemu', 'hvm'), ('x86_64', 'qemu', 'hvm'), ('x86_64', 'kvm', 'hvm'), ('xtensa', 'qemu', 'hvm'), ('xtensaeb', 'qemu', 'hvm')], 'vcpus': 12, 'memory_mb': 31890, 'local_gb': 456.8854446411133, 'vcpus_used': 0, 'memory_mb_used': 3856, 'local_gb_used': 18.015335083007812, 'hypervisor_type': 'QEMU', 'hypervisor_version': 4002001, 'hypervisor_hostname': 'c1c2', 'cpu_info': '{"arch": "x86_64", "model": "Broadwell-noTSX-IBRS", "vendor": "Intel", "topology": {"cells": 1, "sockets": 1, "cores": 6, "threads": 2}, "features": ["sse4.2", "mds-no", "stibp", "pdpe1gb", "xsaveopt", "ht", "intel-pt", "mtrr", "abm", "tm", "lm", "umip", "mca", "pku", "ds_cpl", "rdrand", "adx", "rdseed", "lahf_lm", "xgetbv1", "nx", "invpcid", "rdtscp", "tsc", "xsavec", "pcid", "arch-capabilities", "pclmuldq", "spec-ctrl", "fsgsbase", "avx2", "md-clear", "vmx", "syscall", "mmx", "ds", "ssse3", "avx", "dtes64", "fxsr", "msr", "acpi", "vpclmulqdq", "smap", "erms", "pge", "cmov", "sha-ni", "fsrm", "x2apic", "xsaves", "cx8", "pse", "pse36", "clflushopt", "vaes", "pni", "ssbd", "movdiri", "movbe", "clwb", "xtpr", "de", "invtsc", "fpu", "tsc-deadline", "pae", "clflush", "ibrs-all", "waitpkg", "sse", "sse2", "bmi1", "3dnowprefetch", "cx16", "popcnt", "rdctl-no", "fma", "tsc_adjust", "xsave", "ss", "skip-l1dfl-vmentry", "sse4.1", "rdpid", "monitor", "vme", "tm2", "pat", "pschange-mc-no", "movdir64b", "gfni", "mce", "smep", "sep", "apic", "arat", "f16c", "bmi2", "aes", "pbe", "est", "pdcm"]}', 'disk_available_least': 415.5914993286133, 'numa_topology': '{"nova_object.name": "NUMATopology", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"cells": [{"nova_object.name": "NUMACell", "nova_object.namespace": "nova", "nova_object.version": "1.5", "nova_object.data": {"id": 0, "cpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "pcpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "memory": 31890, "cpu_usage": 0, "memory_usage": 0, "pinned_cpus": [], "siblings": [[0, 1], [10, 11], [2, 3], [6, 7], [4, 5], [8, 9]], "mempages": [{"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 4, "total": 8163962, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "reserved", "total"]}, {"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 2048, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "reserved", "total"]}, {" nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 1048576, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "reserved", "total"]}], "network_metadata": {"nova_object.name": "NetworkMetadata", "nova_object.namespace": "nova", "nova_object.version": "1.0", "nova_object.data": {"physnets": [], "tunneled": false}, "nova_object.changes": ["physnets", "tunneled"]}, "socket": 0}, "nova_object.changes": ["cpuset", "memory_usage", "cpu_usage", "id", "pinned_cpus", "pcpuset", "socket", "network_metadata", "siblings", "mempages", "memory"]}]}, "nova_object.changes": ["cells"]}', 'host_ip': '192.168.28.21'} _setup_pci_tracker /usr/lib/python3/dist-packages/nova/compute/resource_tracker.py:787 # 2023-02-28 14:01:33.800 443293 DEBUG nova.compute.resource_tracker [req-52743217-f0b6-4525-b9fd-371246b6c929 - - - - -] ComputeNode(cpu_allocation_ratio=16.0,cpu_info='{"arch": "x86_64", "model": "Broadwell-noTSX-IBRS", "vendor": "Intel", "topology": {"cells": 1, "sockets": 1, "cores": 6, "threads": 2}, "features": ["sse4.2", "mds-no", "stibp", "pdpe1gb", "xsaveopt", "ht", "intel-pt", "mtrr", "abm", "tm", "lm", "umip", "mca", "pku", "ds_cpl", "rdrand", "adx", "rdseed", "lahf_lm", "xgetbv1", "nx", "invpcid", "rdtscp", "tsc", "xsavec", "pcid", "arch-capabilities", "pclmuldq", "spec-ctrl", "fsgsbase", "avx2", "md-clear", "vmx", "syscall", "mmx", "ds", "ssse3", "avx", "dtes64", "fxsr", "msr", "acpi", "vpclmulqdq", "smap", "erms", "pge", "cmov", "sha-ni", "fsrm", "x2apic", "xsaves", "cx8", "pse", "pse36", "clflushopt", "vaes", "pni", "ssbd", "movdiri", "movbe", "clwb", "xtpr", "de", "invtsc", "fpu", "tsc-deadline", "pae", "clflush", "ibrs-all", "waitpkg", "sse", "sse2", "bmi1", "3dnowprefetch", "cx16", "popcnt", "rdctl-no", "fma", "tsc_adjust", "xsave", "ss", "skip-l1dfl-vmentry", "sse4.1", "rdpid", "monitor", "vme", "tm2", "pat", "pschange-mc-no", "movdir64b", "gfni", "mce", "smep", "sep", "apic", "arat", "f16c", "bmi2", "aes", "pbe", "est", "pdcm"]}',created_at=2023-02-14T03:19:40Z,current_workload=0,deleted=False,deleted_at=None,disk_allocation_ratio=1.0,disk_available_least=415,free_disk_gb=456,free_ram_mb=31378,host='c1c2',host_ip=192.168.28.21,hypervisor_hostname='c1c2',hypervisor_type='QEMU',hypervisor_version=4002001,id=8,local_gb=456,local_gb_used=18,mapped=0,memory_mb=31890,memory_mb_used=3856,metrics='[]',numa_topology='{" nova_object.name": "NUMATopology", "nova_object.namespace": "nova", "nova_object.version": "1.2", "nova_object.data": {"cells": [{" nova_object.name": "NUMACell", "nova_object.namespace": "nova", "nova_object.version": "1.5", "nova_object.data": {"id": 0, "cpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "pcpuset": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], "memory": 31890, "cpu_usage": 0, "memory_usage": 0, "pinned_cpus": [], "siblings": [[0, 1], [10, 11], [2, 3], [6, 7], [4, 5], [8, 9]], "mempages": [{"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 4, "total": 8163962, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "reserved", "total"]}, {" nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 2048, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "reserved", "total"]}, {"nova_object.name": "NUMAPagesTopology", "nova_object.namespace": "nova", "nova_object.version": "1.1", "nova_object.data": {"size_kb": 1048576, "total": 0, "used": 0, "reserved": 0}, "nova_object.changes": ["size_kb", "used", "reserved", "total"]}], "network_metadata": {"nova_object.name": "NetworkMetadata", "nova_object.namespace": "nova", "nova_object.version": "1.0", "nova_object.data": {"physnets": [], "tunneled": false}, "nova_object.changes": ["physnets", "tunneled"]}, "socket": 0}, "nova_object.changes": ["cpuset", "memory_usage", "cpu_usage", "id", "pinned_cpus", "pcpuset", "socket", "network_metadata", "siblings", "mempages", "memory"]}]}, "nova_object.changes": ["cells"]}',pci_device_pools=PciDevicePoolList,ram_allocation_ratio=1.5,running_vms=0,service_id=None,stats={failed_builds='0'},supported_hv_specs=[HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec,HVSpec],updated_at=2023-02-28T06:00:15Z,uuid=c360cc82-f0fd-4662-bccd-e1f02b27af51,vcpus=12,vcpus_used=0) _setup_pci_tracker /usr/lib/python3/dist-packages/nova/compute/resource_tracker.py:788 LOG.debug(compute_node.pci_device_pools.fields) # 2023-02-28 15:27:57.213 447003 DEBUG nova.compute.resource_tracker [req-7f09d102-95de-4b6f-a0f7-d6ee2fc26e6d - - - - -] {'objects': List(default=<class 'oslo_versionedobjects.fields.UnspecifiedDefault'>,nullable=False)} _setup_pci_tracker /usr/lib/python3/dist-packages/nova/compute/resource_tracker.py:789 ``` As above, this code "dev_json = resources.pop('pci_passthrough_devices')" pop these infomation. So my question is 1) is this BUG ? 2) what update_devices_from_hypervisor_resources do ? 3) how to use PCI passthough in openstack Yoga ? ---- Simon Jones

4 13

[cinder] Error when creating backups from iscsi volume
by Rishat Azizov 06 Mar '23

06 Mar '23

Hello! We have an error with creating backups from iscsi volume. Usually, this happens with large backups over 100GB. 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server [req-f6619913-6f96-4226-8d75-2da3fca722f1 23de1b92e7674cf59486f07ac75b886b a7585b47d1f143e9839c49b4e3bbe1b4 - - -] Exception during message handling: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. Command: multipath -f 3624a93705842cfae35d7483200015ec6 Exit code: 1 Stdout: '' Stderr: 'Feb 16 00:22:45 | 3624a93705842cfae35d7483200015ec6 is not a multipath device\n' 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/utils.py", line 890, in wrapper 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server return func(self, *args, **kwargs) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/backup/manager.py", line 410, in create_backup 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server volume_utils.update_backup_error(backup, str(err)) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server self.force_reraise() 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server raise self.value 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/backup/manager.py", line 399, in create_backup 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server updates = self._run_backup(context, backup, volume) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/backup/manager.py", line 493, in _run_backup 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server ignore_errors=True) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/cinder/backup/manager.py", line 1066, in _detach_device 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server force=force, ignore_errors=ignore_errors) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/utils.py", line 141, in trace_logging_wrapper 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 360, in inner 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 880, in disconnect_volume 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server is_disconnect_call=True) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 942, in _cleanup_connection 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server self._linuxscsi.flush_multipath_device(multipath_name) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/initiator/linuxscsi.py", line 382, in flush_multipath_device 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server root_helper=self._root_helper) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/executor.py", line 52, in _execute 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server result = self.__execute(*args, **kwargs) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/os_brick/privileged/rootwrap.py", line 172, in execute 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server return execute_root(*cmd, **kwargs) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 247, in _wrap 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server return self.channel.remote_call(name, args, kwargs) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 224, in remote_call 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server raise exc_type(*result[2]) 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server Command: multipath -f 3624a93705842cfae35d7483200015ec6 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server Exit code: 1 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server Stdout: '' 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server Stderr: 'Feb 16 00:22:45 | 3624a93705842cfae35d7483200015ec6 is not a multipath device\n' 2023-02-16 00:23:25.298 1920 ERROR oslo_messaging.rpc.server Could you please help with this error?

2 1

[kolla] [train] [cinder] Volume multiattach exposed to non-admin users via API
by Albert Braden 03 Mar '23

03 Mar '23

According to this document [1] multiattach volumes can only be setup if explicitly allowed by creating a “multiattach” volume type. “Starting from the Queens release the ability to attach a volume to multiple hosts/servers requires that the volume is of a special type that includes an extra-spec capability setting of multiattach=<is> True… Creating a new volume type is an admin-only operation by default. One of our customers appears to have used TerraForm to create a volume with the multiattach flag set and it worked, and that volume has multiple attachments. When I look here [2] it appears that the default is: #"volume:multiattach": "rule:xena_system_admin_or_project_member" So it looks like, by default, any project member can create a multiattach volume. What am I missing? [1]: https://docs.openstack.org/cinder/latest/admin/volume-multiattach.html [2]: https://docs.openstack.org/cinder/latest/configuration/block-storage/sample…

5 10