- openstack-discuss - lists.openstack.org

[security][ptg] Virtual PTG Planning
by Gage Hugo 30 Apr '20

30 Apr '20

Hello, As discussed in today's meeting, the Security SIG has scheduled some tentative timeslots for the virtual PTG to discuss any topics. We also have an etherpad to fill out if anyone has anything they want to discuss. Hope to see you there! Current Timeslots: - 1500 - 1700 UTC Mon June 1st 2020 - 2100 - 2300 UTC Mon June 1st 2020 https://etherpad.opendev.org/p/security-sig-ptg-victoria

1 0

[ironic][redfish] Breaking (for us) underlying schema/behavior change and what do we do?
by Julia Kreger 30 Apr '20

30 Apr '20

Greetings fellow awesome sentient beings, and the occasional text indexers! We have a growing conundrum with Redfish. A long time ago, in a red shifted galaxy inhabited by Redfish, a feature was defined early on in the DMTF specification defining a "BootSourceOverrideEnabled" field as part of the ComputerSystem Boot object[0]. This setting, in concert with the "BootSourceOverrideTarget" and "BootSourceOverrideMode" setting allowed Ironic to signal an override preference in regards to what device the machine should boot to. As time as moved forward, schema field descriptions were added, and notes further detailed (observable by looking at [1], see ComputerSystem 1.0.0, 1.5.0, 1.6.0, 1.8.0). One such case, at least depending on what field you looked at also noted that "BootSourceOverrideEnabled" was not possible with the "Continuous" setting, however that was only noted under the one time override option "UefiTargetBootSourceOverride" which is leveraged by the explicit "UEFITarget" target option, at least until ComputerSystem 1.8.0. In ComputerSystem 1.8.0, an explicit note was added into the description text for "BootSourceOverrideEnabled" detailing that one time override use was the only option for UEFI. The published documents also have not been very clear on this, but there are numerous documents on the subject so it seems very easy for the schema details note being added and enhanced in 1.8.0 drove vendors to begin to change the logic around their "BootSourceOverrideEnabled" option to represent what the current documentation detailed, regardless of their schema version. Also, the schema/api versions are disjointed and BMCs out there exist without some of the defined fields. It just kind of is our life with hardware. So why now? The ironic team has had reports recently, across multiple vendors, where with newer firmware versions beginning to implement the changes in schema 1.8.0, where it seems we will no longer be able to use the option we were using, which seemed logical at least when looking at the BMC conformance to the standard. And while some may view the fact it worked on multiple vendors as a regression, I'm sure they see it as a bug as it was seemingly never intended to work that way when we look at the "UEFITargetBootSourceOverride" field notes from the early schemas. tl;dr: As BMC's vendors add the newer schema fields and resulting behavior, the existing redfish driver's ability to assert persistent boot devices breaks. With all of that having been said, I think we should remove our persistent setting capability in our redfish driver, and back port the fix where possible. That being said, this is also a behavior change, so it is time to get everyone's thoughts, since we will need to stress that machines by default should boot from persistent storage instead of network by default. We also need to launch a long term effort to try and duplicate the setting capabilities through the other BMC interfaces/methods/settings, but the variation by vendor is what is going to make it very difficult until the latest redfish schema version is implemented by vendors. Of course, that too may be a while. Thoughts? -Julia [0]: DMTF Redfish Schema Supplement 2019.1a, Page 71: https://www.dmtf.org/sites/default/files/standards/documents/DSP0268_2019.1… or the 2016.2 version of the document, Page 16: https://www.dmtf.org/sites/default/files/standards/documents/DSP2046_2016.2… And the same information can be found in the and 2019.2, 2019.3 schema document versions. [1]: Latest schema bundle pack: https://www.dmtf.org/sites/default/files/standards/documents/DSP8010_2019.4…

2 3

[MURANO] DevStack Murano "yaql_function" Error
by İzzettin Erdem 30 Apr '20

30 Apr '20

Hello folk, Is there anyone who used murano before? I am trying to install Murano on devstack stable/train release. When i try to deploy an environment, it throws an error. At murona-engine logs error is: "func.__yaql_function__ = fd: murano.dsl.dsl_exception.MuranoPlException: [AttributeError]: 'method' object has no attribute '__yaql_function__'" Any suggestioıns? Thanks. Regards.

3 4

[puppet][packaging] Switching to policy.yaml (over policy.json)
by Thomas Goirand 30 Apr '20

30 Apr '20

Hi team! In the light of the discussion that recently happened in #openstack-nova, it looks like switching from a .json file to a .yaml file is what we should do. Indeed, the file generated in .json format, if used pristine with the nova.conf and without enforcing scope, makes a nova-api service that simply doesn't work. Please read https://bugs.launchpad.net/nova/+bug/1875418 for more insights. The issue is that both packages are generating a .json (at least in Debian and Ubuntu) and puppet expect a policy.json, not a policy.yaml. With the policy.yaml, we don't have the same problem as by default, all policies are commented out. Operators just need to uncomment to activate. So, my proposal is the following: before the final release of OpenStack, I will modify all Debian OpenStack packages to generate (and package) both policy.json and policy.yaml. Then puppet-openstack can switch over to the .yaml file, and uncomment only the parts that the operator sees as relevant. I also would like to add a policy.d folder by default in each package, where operators can override stuff. Just having the folder will be a sign to operators that they are invited to write stuff in there, and that it will not be overwritten by a package upgrade. Then what I would like to do, is get puppet-openstack to only write there, for example in /etc/nova/policy.d/my-custom-policy.yaml. The file in /etc/nova/policy.yaml will be marked as "CONFFILE" in Debian, meaning that dpkg will prompt for changes on upgrades, while what's in the policy.d will remain. Last, I do believe that the yaml files are a way more easy to handle with puppet than the .json counterpart. Indeed, we could use something like the .ini management thing, with the : (Semicolumn) sign replacing the = (equal) sign. Moreover, the .yaml files contain comments which the .json files are lacking, making it auto-documented for operators. The only reason why I didn't use .yaml files earlier in the Debian packages was that, somehow, loading them with the API didn't work. I confirm that now it looks like working (though I'd have to test if changing a value is taken into account, I didn't do that yet). Your thoughts everyone? Cheers, Thomas Goirand (zigo)

2 1

[all] Curating the openstack org on GitHub
by Thierry Carrez 30 Apr '20

30 Apr '20

Hi everyone, As you know, for practical and "code marketing" reasons, we maintain mirrors of the opendev.org/openstack repositories on GitHub: https://github.com/openstack/ A few months ago jroll and I took on the task of maintaining the "openstack" organization on GitHub... In particular pushing "moved" final commits to repositories that are now maintained elsewhere on Opendev, and marking them retired. It's all a bit of a challenge. GitHub does not let you set the "mirror" flag by API (you have to ask GitHub support to manually do it, and they don't like it when you ask for it to be set on 1,831 repositories). The "pinned repositories" UI fails miserably and does not let us select "Nova" (again, probably too many repositories, GitHub support does not have a solution for us). There are limited opportunities to describe the fact that we actually only use GitHub as a mirror, leading to confusion. For inactive repositories (no longer under OpenStack governance), the situation is even more confusing. Some are noted "MOVED" or "RETIRED" in their description, some have a closing commit (some others don't), some have the "archived" flag. Some don't have anything and appear active while they are not (see attached CSV for those who like data). There is also the openstack-dev and openstack-attic organizations, which are leftovers from ancient times. In summary, for a "code marketing" opportunity, it does not paint a great picture. I'd like to make the following suggestion: - aggressively delete all non-openstack things from the openstack org, keeping only official, active repositories - delete the openstack-dev and openstack-attic organizations We shied away from doing that in the past, mostly to not break people who may have cloned those repositories... But I think the benefits of cleaning up now outweigh the drawbacks. The reference code always exists at opendev.org anyway. Also maybe once we are back to a more reasonable number of visible repositories, the pins UI will work again. And GitHub will actually be OK with setting the mirror flag. Thoughts, comments ? -- Thierry Carrez (ttx)

13 27

[neutron] network converting from flat to vlan
by Ionut Biru 30 Apr '20

30 Apr '20

Hi guys, I made a mistake a long time ago creating a flat network instead of vlan. Is there a way to convert this network to vlan? Did anyone try to do that and have a procedure in place? I'm running rocky. -- Ionut Biru - https://fleio.com

1 0

[neutron] CI meeting cancelled
by Slawek Kaplonski 30 Apr '20

30 Apr '20

Hi, As 1st of May is Labor Day and public holiday in Poland, I will not be available to chair drivers meeting. Lets cancel it this week. See You all on the meeting next week. Have a great (long) weekend :) -- Slawek Kaplonski Senior software engineer Red Hat

1 0

[TC] Ussuri Virtual Celebration!
by Kendall Nelson 29 Apr '20

29 Apr '20

Hello :) So! Since we won't be able to see each other in June and celebrate the recent release together, is there interest in doing a virtual cheers as a community? We could do something fun like.. trivia or a baking contest or something? I am reaching out to the TC specifically because we would probably want to do more than one of these and spread across all timezones so we could have a TC member or two 'host' the cheers. Can't wait to hear what you all think! -Kendall (diablo_rojo)

9 12

[openstack-helm-infra][elasticsearch]Error deploying chart in k8s cluster with local storage
by Sanjay K 29 Apr '20

29 Apr '20

Hello everyone, I have been trying to deploy elasticsearch helm chart in my k8s cluster consisting of 4 nodes (1 k8s master, 1 controller and 2 computes) after deploying fluentbit and fluentd helm charts which are deployed with success. I have plan to make the EFK (Elasticsearch-Fluentd-Kibana) stack running in my k8s cluster setup for logs management. However I face issues while deploying elasticsearch helm chart using the below command : $ helm install elasticsearch local/elasticsearch --values values_overrides/local-storage.yaml -n openstack Error: unable to build kubernetes objects from release manifest: error validating "": error validating data: ValidationError(StatefulSet.spec): unknown field "strategy" in io.k8s.api.apps.v1.StatefulSetSpec Can anyone point me if I missing anything? What are the prerequisites before deploying elasticsearch chart which intended to use local storage/pv in my case avoiding the default S3 storage as per the chart? Thanks for your pointers and suggestions. ' best regards, SanjayK

2 7

[requirements][all] - requirements is now frozen for non-critical updates
by Matthew Thode 29 Apr '20

29 Apr '20

After the Ussuri-3 milestone, only critical requirements and constraints changes will be allowed. Freezing our requirements list gives packagers downstream an opportunity to catch up and prepare packages for everything necessary for distributions of the upcoming release. The requirements remain frozen until the stable branches are created, with the release candidates. All future changes not previously approved will need a FFE request set to the list, please prefix it with [FFE][requirements][PROJECTNAME]. -- Matthew Thode

1 1

Re: [TripleO][train][rdo] installation of undercloud fails during Run container-puppet tasks step1
by Ruslanas Gžibovskis 29 Apr '20

29 Apr '20

During installation, I can see them running fro 25sec max. not sure if it is as expected, Can I check somehow, how to get them up? if I do podman run -it Image_ID will it be enough to test if images are good? On Wed, 29 Apr 2020 at 12:01, Wu, Heng <wuh.fnst(a)cn.fujitsu.com> wrote: > > I am new to containers, can I somehow transfer these images between, as > I understand, it might help? > > > > The image will be downloaded automatically with the installation, you > don't need to do it manually. > > The problem seems to be that the container failed to start, not the errors > in the images themselves. > > You may need to check the configuration of undercloud.conf and proxy > settings (if they exist). > > > > *From:* Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > *Sent:* 2020年4月29日 15:48 > *To:* Alex Schultz <aschultz(a)redhat.com> > *Cc:* Openstack-discuss <openstack-discuss(a)lists.openstack.org>; > kent.martin(a)ericsson.com > *Subject:* Re: [TripleO][train][rdo] installation of undercloud fails > during Run container-puppet tasks step1 > > > > On the other installation I have, I see a bit more images: > > > > > I am new to containers, can I somehow transfer these images between, as I > understand, it might help? > > > > On Wed, 29 Apr 2020 at 09:45, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > wrote: > > I just now realized, that I have seen in some log, messages, missing some > puppet-dependencies... Cannot find log again... > > > > On Wed, 29 Apr 2020 at 09:39, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > wrote: > > podman ps -a = clean, no containers at all. > > I have a watch -d "sudo podman ps -a ; sudo podman images -a ; sudo df -h" > > > > paunch.log is empty. (I did several reinstallations). > > > > > > > Later I saw connectionpool retrying, but I have not seen " > tripleo_common.image.image_uploader" with same. > > > > > > > On Tue, 28 Apr 2020 at 20:10, Alex Schultz <aschultz(a)redhat.com> wrote: > > On Tue, Apr 28, 2020 at 11:57 AM Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > wrote: > > > > Hi all, > > > > I am running a fresh install of rdo train, on centos7 > > I almost a week I am facing error at this step: > > TASK [Run container-puppet tasks (generate config) during step 1] > > > > So I have ansible.log attached, I cannot find anything, where it is > failing. > > According to some understanding in ansible, it fails if it finds stderr > output. > > I cannot find error/fail or smth, I see Notices and Warnings, but I > believe it is not stderr? > > > > I see containers running and removed after some time. > > (as it should be I think)... > > > > Could you help me, where to dig? > > > > 2020-04-27 22:27:46,147 p=132230 u=root | TASK [Start containers for > step 1 using paunch] > > ***************************************************************************************************************************** > 2020-04-27 22:27:46,148 p=132230 u=root | Monday 27 April 2020 > 22:27:46 +0200 (0:00:00.137) 0:04:44.326 **********? > 2020-04-27 22:27:46,816 p=132230 u=root | ok: [remote-u] > 2020-04-27 22:27:46,914 p=132230 u=root | TASK [Debug output for > task: Start containers for step 1] > > ******************************************************************************************************************* > 2020-04-27 22:27:46,915 p=132230 u=root | Monday 27 April 2020 > 22:27:46 +0200 (0:00:00.767) 0:04:45.093 **********? > 2020-04-27 22:27:46,977 p=132230 u=root | fatal: [remote-u]: FAILED! => { > "failed_when_result": true,? > "outputs.stdout_lines | default([]) | union(outputs.stderr_lines | > default([]))": [] > > Check /var/log/paunch.log. It probably has additional information as > to why the containers didn't start. You might also check the output > of 'sudo podman ps -a' to see if any containers exited with errors. > > > -- > > Ruslanas Gžibovskis > > +370 6030 7030 > > > > > -- > > Ruslanas Gžibovskis > +370 6030 7030 > > > > > -- > > Ruslanas Gžibovskis > +370 6030 7030 > > > > > -- > > Ruslanas Gžibovskis > +370 6030 7030 > -- Ruslanas Gžibovskis +370 6030 7030

1 1

[neutron] CI meeting
by Slawek Kaplonski 29 Apr '20

29 Apr '20

Hi, Sorry for late notice but I totally forgot about it. I need to cancel today's Neutron CI meeting. There is RH Summit going on and I have some sessions which I want to attend during our CI meeting. I checked grafana, and I don't see any urgent issues related to our CI so lets cancel this meeting and see You all next week :) -- Slawek Kaplonski Senior software engineer Red Hat

1 0

[Neutron] Cancel neutron L3 meeting
by LIU Yulong 29 Apr '20

29 Apr '20

Hi there, Because I have to some local workload, I will not be able to run the L3 meeting tonight. And I'm going to take a vacation next week. So the next L3 meeting will be set on 13th May 2020. See you guys online then. Regards, LIU Yulong

1 0

Re: [TripleO][train][rdo] installation of undercloud fails during Run container-puppet tasks step1
by Ruslanas Gžibovskis 29 Apr '20

29 Apr '20

Moving output to pastebin I have found it very interesting, that I was able to download images by running some pull commands (below), podman images -a show it, but the installation still gets 401 errors. http://paste.debian.net/1143698/ > > On Wed, 29 Apr 2020 at 09:47, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> > wrote: > >> On the other installation I have, I see a bit more images: >> >> I am new to containers, can I somehow transfer these images between, as I >> understand, it might help? >> >> On Wed, 29 Apr 2020 at 09:45, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> >> wrote: >> >>> I just now realized, that I have seen in some log, messages, missing >>> some puppet-dependencies... Cannot find log again... >>> >>> On Wed, 29 Apr 2020 at 09:39, Ruslanas Gžibovskis <ruslanas(a)lpic.lt> >>> wrote: >>> >>>> podman ps -a = clean, no containers at all. >>>> I have a watch -d "sudo podman ps -a ; sudo podman images -a ; sudo df >>>> -h" >>>> >>>> paunch.log is empty. (I did several reinstallations). >>>> >>>> I found in image logs: >>>> 2020-04-29 08:52:49,854 140572 DEBUG urllib3.connectionpool [ ] >>>> https://registry-1.docker.io:443 "GET /v2/ HTTP/1.1" 401 87 >>>> 2020-04-29 08:52:49,855 140572 DEBUG >>>> tripleo_common.image.image_uploader [ ] >>>> https://registry-1.docker.io/v2/ status code 401 >>>> 2020-04-29 08:52:49,855 140572 DEBUG >>>> tripleo_common.image.image_uploader [ ] Token parameters: params {'scope': >>>> 'repository:tripleotrain/centos-binary-zaqar-wsgi:pull', 'service': ' >>>> registry.docker.io'} >>>> >>>> Later I saw connectionpool retrying, but I have not seen " >>>> tripleo_common.image.image_uploader" with same. >>>> >>>> On Tue, 28 Apr 2020 at 20:10, Alex Schultz <aschultz(a)redhat.com> wrote: >>>> >>>>> On Tue, Apr 28, 2020 at 11:57 AM Ruslanas Gžibovskis <ruslanas(a)lpic.lt> >>>>> wrote: >>>>> > >>>>> > Hi all, >>>>> > >>>>> > I am running a fresh install of rdo train, on centos7 >>>>> > I almost a week I am facing error at this step: >>>>> > TASK [Run container-puppet tasks (generate config) during step 1] >>>>> > >>>>> > So I have ansible.log attached, I cannot find anything, where it is >>>>> failing. >>>>> > According to some understanding in ansible, it fails if it finds >>>>> stderr output. >>>>> > I cannot find error/fail or smth, I see Notices and Warnings, but I >>>>> believe it is not stderr? >>>>> > >>>>> > I see containers running and removed after some time. >>>>> > (as it should be I think)... >>>>> > >>>>> > Could you help me, where to dig? >>>>> > >>>>> >>>>> 2020-04-27 22:27:46,147 p=132230 u=root | TASK [Start containers for >>>>> step 1 using paunch] >>>>> >>>>> ***************************************************************************************************************************** >>>>> 2020-04-27 22:27:46,148 p=132230 u=root | Monday 27 April 2020 >>>>> 22:27:46 +0200 (0:00:00.137) 0:04:44.326 **********? >>>>> 2020-04-27 22:27:46,816 p=132230 u=root | ok: [remote-u] >>>>> 2020-04-27 22:27:46,914 p=132230 u=root | TASK [Debug output for >>>>> task: Start containers for step 1] >>>>> >>>>> ******************************************************************************************************************* >>>>> 2020-04-27 22:27:46,915 p=132230 u=root | Monday 27 April 2020 >>>>> 22:27:46 +0200 (0:00:00.767) 0:04:45.093 **********? >>>>> 2020-04-27 22:27:46,977 p=132230 u=root | fatal: [remote-u]: FAILED! >>>>> => { >>>>> "failed_when_result": true,? >>>>> "outputs.stdout_lines | default([]) | union(outputs.stderr_lines | >>>>> default([]))": [] >>>>> >>>>> Check /var/log/paunch.log. It probably has additional information as >>>>> to why the containers didn't start. You might also check the output >>>>> of 'sudo podman ps -a' to see if any containers exited with errors. >>>>> >>>>> -- Ruslanas Gžibovskis +370 6030 7030

1 0

[all][TC][PTL][release] Ussuri Community Meeting Proposal
by Allison Price 29 Apr '20

29 Apr '20

Hi everyone, As we prepare for the upcoming Ussuri release on May 13, I wanted to see if there was interest among the community to do a community meeting with folks presenting release highlights / answering questions that people may have. We were thinking that atleast one member from the TC or release management team could moderate the discussion and call on various PTLs to present what their teams accomplished during this cycle. If possible, we could do one that is around 1400 UTC to hit Europe and the United States and potentially another one around 0200 UTC for APAC, so we may need two volunteers. We would also like to promote Ussuri demos in the release comms, including the landing page and promotional materials. If you are planning to do a demo or would like to, please let me know and we can include it on the Ussuri landing page. Let me know what y’all think about doing a community meeting and if particular folks would like to volunteer to moderate and / or present. Looking forward to hearing feedback, and, in particular, if anyone is interested in volunteering to help moderate or participate in the Ussuri focused community meeting. Thanks! Allison Allison Price OpenStack Foundation allison(a)openstack.org <mailto:allison@openstack.org>

7 12

[magnum] issues with cloud-init and user-data on fedora-atomic image
by Jean-Philippe Méthot 28 Apr '20

28 Apr '20

Hi, I’ve been testing out magnum (rocky) in my staging environment and I’ve been running into an issue I can’t really explain. Essentially, Magnum is able to provision all the VMs it needs according to the template. However, when it comes time to install kubernetes through cloud-init, cloud-init is unable to fetch the user-data. I must specify that ONLY the user-data is affected. Everything else that’s read from the metadata gets through without any issue. I can also do 'curl http://169.254.169.254/2009-04-04/' <http://169.254.169.254/2009-04-04/'> from the VM without any issue. When I try to curl http://169.254.169.254/2009-04-04/user-data <http://169.254.169.254/2009-04-04/user-data> , curl just hangs without any data showing up : sudo curl -v http://169.254.169.254/2009-04-04/user-data * Trying 169.254.169.254... * TCP_NODELAY set * Connected to 169.254.169.254 (169.254.169.254) port 80 (#0) > GET /2009-04-04/user-data HTTP/1.1 > Host: 169.254.169.254 > User-Agent: curl/7.59.0 > Accept: */* Tcpdump -vvv looks like this : sudo tcpdump -vvv -i eth0 host 169.254.169.254 tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes 20:36:22.284195 IP (tos 0x0, ttl 64, id 11248, offset 0, flags [DF], proto TCP (6), length 60) kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896 > 169.254.169.254.http: Flags [S], cksum 0x5e34 (incorrect -> 0x51a5), seq 2359174217, win 26730, options [mss 8910,sackOK,TS val 583688334 ecr 0,nop,wscale 7], length 0 20:36:22.284793 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 169.254.169.254.http > kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896: Flags [S.], cksum 0x9ed1 (correct), seq 2099498427, ack 2359174218, win 26694, options [mss 8910,sackOK,TS val 4068504964 ecr 583688334,nop,wscale 9], length 0 20:36:22.284875 IP (tos 0x0, ttl 64, id 11249, offset 0, flags [DF], proto TCP (6), length 52) kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896 > 169.254.169.254.http: Flags [.], cksum 0x5e2c (incorrect -> 0x522e), seq 1, ack 1, win 209, options [nop,nop,TS val 583688335 ecr 4068504964], length 0 20:36:22.284962 IP (tos 0x0, ttl 64, id 11250, offset 0, flags [DF], proto TCP (6), length 151) kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896 > 169.254.169.254.http: Flags [P.], cksum 0x5e8f (incorrect -> 0xb692), seq 1:100, ack 1, win 209, options [nop,nop,TS val 583688335 ecr 4068504964], length 99: HTTP, length: 99 GET /2009-04-04/user-data HTTP/1.1 Host: 169.254.169.254 User-Agent: curl/7.59.0 Accept: */* 20:36:22.325212 IP (tos 0x0, ttl 64, id 47160, offset 0, flags [DF], proto TCP (6), length 52) 169.254.169.254.http > kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896: Flags [.], cksum 0x523e (correct), seq 1, ack 100, win 53, options [nop,nop,TS val 4068505005 ecr 583688335], length 0 20:36:33.546523 IP (tos 0x0, ttl 64, id 11251, offset 0, flags [DF], proto TCP (6), length 52) kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896 > 169.254.169.254.http: Flags [F.], cksum 0x5e2c (incorrect -> 0x25a4), seq 100, ack 1, win 209, options [nop,nop,TS val 583699596 ecr 4068505005], length 0 20:36:33.586154 IP (tos 0x0, ttl 64, id 47169, offset 0, flags [DF], proto TCP (6), length 52) 169.254.169.254.http > kubernetes-cluster-f4anhakchpkj-minion-0.staging.planethoster.ne.56896: Flags [.], cksum 0x91fc (correct), seq 26695, ack 101, win 53, options [nop,nop,TS val 4068516266 ecr 583699596], length 0 So the instance does receive packets as a reply to the curl, but they’re empty. I have tested on both Fedora-Atomic-27-20180212.2.x86_64.qcow2 and Fedora-AtomicHost-28-20180425.0.x86_64.qcow2 images and reproduced the issue both times. Nova-api and neutron-metadata logs show http 200 for each of my requests to nova. What could be blocking the user-data from reaching the instance in this case? Is there a way to set magnum-created intances to use configdrive instead? I must also specify that user-data works on all regular instances. It’s only affecting instances created by magnum. Best regards, Jean-Philippe Méthot Senior Openstack system administrator Administrateur système Openstack sénior PlanetHoster inc.

1 0

[all][PTL] U Community Goal: Project PTL & Contrib Docs Update #8
by Kendall Nelson 28 Apr '20

28 Apr '20

Hello Everyone! More excellent progress[1] has been made since last week! Again, thank you for all the hard work! Projects missing progress/merged status according to the story[4] are as follows: - Adjutant - Barbican - Cloudkitty - Congress - Cyborg - Ec2-API - Freezer - Heat - Horizon - Karbor - Kuryr - LOCI - Masakari - Mistral - Monasca - Murano - OpenStack Charms - openstack-chef - OpenStackAnsible - OpenStackClient - Oslo - Packaging-RPM - Placement - Puppet OpenStack - Rally - Release Management - Requirements - Senlin - Solum - Storlets - Swift - Tacker - Telemetry - Tricircle - TripleO - Winstackers - Zaqar If you are on this list, please get started IMMEDIATELY. All told, if you are doing the bare minimum of only filling out the template[2] it shouldn't take more than 5-10 minutes per repo. Please add me if you need help with reviews and let me know if there is any other help I can provide. If you have questions about the goal itself, here is the link to that[3]. And as you push patches, please be sure to update the StoryBoard story[4]. Thanks! -Kendall (diablo_rojo) [1] Progress: https://review.opendev.org/#/q/topic:project-ptl-and-contrib-docs+(status:o… [2] Cookiecutter Template: https://opendev.org/openstack/cookiecutter/src/branch/master/%7b%7bcookiecu… [3] Accepted Goal: https://governance.openstack.org/tc/goals/selected/ussuri/project-ptl-and-c… [4] Task Tracking: https://storyboard.openstack.org/#!/story/2007236

1 0

[scientific-sig] IRC Meeting today 2100 UTC - Open Infra labs, Train upgrade experiences
by Stig Telfer 28 Apr '20

28 Apr '20

Hi All, We have a Scientific SIG IRC meeting in #openstack-meeting at 2100 UTC (about 45 minutes). Everyone is welcome. Today’s agenda is here: https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_April_28th_2020 <https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_April_28th_2020> This week we have a discussion on the Open Infra Labs project, plus we are looking to share operational experiences on OpenStack Train. Cheers, Stig

1 0

[tripleo] ptg
by Wesley Hayutin 28 Apr '20

28 Apr '20

Greetings, Just a reminder to review the TripleO time slots for PTG [1] Looking for feedback on the times. We'll start a schedule etherpad for folks to align their topics to a particular time slot soon. Note, the time slot for Q&A or really anything for folks not able to join the 13-17UTC time slot. Thanks [1] https://ethercalc.openstack.org/126u8ek25noy [2] https://etherpad.opendev.org/p/tripleo-victoria-topics

1 0

Announcing the general availability of Debian packages for Ussuri
by Thomas Goirand 28 Apr '20

28 Apr '20

Hi, Even after 9 years packaging OpenStack, I'm still excited to announce the availability of the last release of OpenStack in Debian. Indeed, today, I was able to launch my first VM using Ussuri, running on top of Debian Buster. Even if we haven't run Tempest on this release yet, it's already usable, and I wanted everyone to know about it. Packages are available either from Debian Experimental (they will be uploaded to Unstable the day of the official release, then hopefully migrate automatically to Debian Testing) or from the debian.net backports to Buster, from both of these repos: deb http://buster-ussuri.debian.net/debian buster-ussuri-backports main deb-src http://buster-ussuri.debian.net/debian buster-ussuri-backports main deb http://buster-ussuri.debian.net/debian buster-ussuri-backports-nochange main deb-src http://buster-ussuri.debian.net/debian buster-ussuri-backports-nochange main *OR* from this server (which contains all release of OpenStack since the Kilo release): deb http://osbpo.debian.net/debian buster-ussuri-backports main deb-src http://osbpo.debian.net/debian buster-ussuri-backports main deb http://osbpo.debian.net/debian buster-ussuri-backports-nochange main deb-src http://osbpo.debian.net/debian buster-ussuri-backports-nochange main Please make sure to configure your servers to use *both* buster-ussuri-backports and buster-ussuri-backports-nochange suites. The first one contains packages maintained by the Debian OpenStack team, while the later are backports of packages maintained by other people in Debian (like those maintained in the Debian Python Module Team, but not only). Still missing from this release... - vitrage-dashboard: There's some new XStatic packages to make, work is ongoing. - some neutron plugins: Many aren't released yet... - heat: It just got released a few hours ago, and I didn't have time yet to figure out what's wrong with these 3 unit tests: http://paste.openstack.org/show/792781/ Hopefully, this will be addressed tomorrow. Note that OCI, the Debian cluster installer for OpenStack, already supports that version (that's how I launched my first instance), and is running the latest version of all puppet modules of puppet-openstack: https://salsa.debian.org/openstack-team/debian/openstack-cluster-installer Some fun statistics: 187 packages were updated in the OpenStack team's Git repository to make this release (plus some updates of packages not within the team). Note that the last piece of software (cinder) was converted to uwsgi, so this release comes with absolutely all API servers running under uwsgi. Please try these packages and report bugs to the Debian BTS as usual, so we can fix as many bugs as possible before the final release. Some packages are still waiting for the FTP master's approval before they can reach Debian proper. Noticeably, many Tempest plugins (as I decided to package them all in Debian). 6 of these are pending, and probably more will be packaged. Stay tuned. Cheers, Thomas Goirand (zigo)

5 5

[neutron] Low-throughput over high-latency connections
by Eric K. Miller 28 Apr '20

28 Apr '20

Hi, I'm at a loss after quite a bit of testing, so thought I'd see if someone has any ideas... We have a Stein cluster that was/is deployed with Kolla Ansible with DVR without OVS+DPDK - rather just the standard kernel OVS. Everything appears to work well with low-latency local and Internet destinations, but as the latency increases, throughput drops significantly. I would have blamed the standard bandwidth-delay product and TCP window size not being large enough, etc., but that definitely isn't the issue, since we have shown that VMs on our VMware clusters (VLAN-based, not NSX-based) do not have the issue and use the same many-10Gbps-connected ISPs to our routers as the OpenStack configuration. I am using a Vultr machine in London as a test destination since the latency is around 86ms - a good high-latency test. The good environment we are testing against: Using iperf3 with 1 thread on a VMware VM results in an average of 155Mbits/sec sent to Vultr London with a few retries recorded that lower the window size to around 1.78MiBytes. This is roughly what we expect, if not more, with faster machines. The problem: Performing the same test on a "much" faster server (E-2288G based, without hyperthreading enabled, so 4.5GHz on all 8 cores), with no other VMs on it, a VM with iperf3 with 1 thread results in an average of only around 32Mbits/sec with a window size that drops significantly at the start and increases to only about 386KiBytes before retries start to happen. This average does not change when using other machines in our cluster, including some Xeon Gold and EPYC processor machines. The only thing I can point to is OVS and/or something in Linux. However, both tests (VMware and KVM) were performed with the same fresh installation and configuration (CentOS 7 with all updates and the standard kernel). So, my doubts are that it has something to do with Linux. On the physical host, the vhost process barely registers any CPU usage, and neither does the VM - as expected on such a fast machine. I even tried to enable multiqueue, since we had it disabled. With a 2 vCPU VM, ethtool -l eth0 displays that 2 queues are in use. Same results - 32Mbits/sec throughput to Vultr London. All machines are connected via 100Gbps or 25Gbps connections with Mellanox Connect-X 4 and 5 cards with Mellanox switches, and iperf can saturate these links between the physical hosts when run on the physical hosts' CentOS. No errors (tx errors, rx errors, packet loss, etc.) are being reported on any of the logical or physical links. I thought a queue got full somewhere, so I looked for where queues were used and found that virtio has a tx_queue_size that can be adjusted, but unfortunately, it appears this can only be done with vhost-user, which I believe is specific to OVS+DPDK. The default tx_queue_size in virtio is 256 elements. I'm not sure what this represents, though, much less how to measure the queue usage statistics. Running iperf3 in "Reverse mode", so the Vultr London machine sends data back to the KVM VM, bandwidth averages around 254Mbits/sec! So the issue appears to be only in one direction - sending network traffic outbound. Running the same iperf3 test from the KVM VM to a Vultr machine that is very close to us (latency of 2ms or so), results in an average bandwidth of 373Mbits/sec, but with 39 retries per second on average. The window size never adjusts low enough to stop the retries - it averages 146KiBytes in size. Using "Reverse mode" results in an average of 1.21Gbits/sec! So, the OVS software isn't slow, but something is causing issues when transmitting data when latency is large - something that doesn't happen with the VMware VMs with the same network and Internet connections. Also, pings from the VMware and KVM VMs are nearly identical - so there isn't some crazy amount of latency caused by OVS - at least when it is essentially idle. I'm not sure where to go from here in terms of diagnostics. Any ideas? Thanks for any help, in advance! Eric

1 0

[tc][puppet][Glare] Propose retirement of puppet-glare
by Takashi Kajinami 28 Apr '20

28 Apr '20

Hi, Currently Glare[1] project is located under x instead of openstack, and IIUC this means that Glare is no longer maintained and has been retired already. [1] https://opendev.org/x/glare Regarding its status, I'll propose to retire puppet-glare[2] because it doesn't make sense to maintain a puppet module for the retired project. [2] https://github.com/openstack/puppet-glare May I ask TC to confirm my understanding about Glare project status ? Also, please let me know if anybody has concern with retiring puppet-glare. Thank you, Takashi Kajinami

4 4

[all] Virtual PTG Tooling Requirements & Suggestions
by Kendall Nelson 28 Apr '20

28 Apr '20

Hello Everyone :) We want to get your input on tooling for the PTG. We have a general idea of what we have for requirements, but wanted to get the community's input- i.e. make sure we aren't missing any. Here is the list of requirements we have right now[1]. Also, if anyone has suggestions of tools that meet these criteria, we would love to hear them! We are gathering and vetting options for the event and want to make sure nothing gets overlooked. -the Kendalls (diablo_rojo & wendallkaters) [1] https://etherpad.opendev.org/p/virt-ptg-requirements

4 3

Openstacksdk
by Tom Schoonjans 28 Apr '20

28 Apr '20

Hi all, I am trying to use the openstacksdk to get a list of my projects. I have not been successful at doing this, as it always tries to get all projects, which doesn’t work for me as I am not an admin: the error message I get is "Policy doesn't allow os_compute_api:servers:detail:get_all_tenants to be performed." This is certainly possible though, because it works fine with the client: openstack project list --my-projects Any thoughts on how I can accomplish this? Many thanks in advance, Tom

1 0

trove image
by 会水的鸟 28 Apr '20

28 Apr '20

您好：     我在使用openstack trove镜像时，一直处于build状态，我应该如何解决。      镜像链接：      https://tarballs.opendev.org/openstack/trove/images/trove-train-mysql-ubuntu-xenial-dev.qcow2                                                                                                                         谢谢 hello：     When I was using the openstack trove image，Always in build state，What should I do。     https://tarballs.opendev.org/openstack/trove/images/trove-train-mysql-ubunt…                                                                                                                             thank you

2 4

[ironic]: multiple compute nodes
by fsbiz＠yahoo.com 28 Apr '20

28 Apr '20

Hi folks, We are seeing some weird issues with multiple compute nodes and would appreciate your thoughts. Background:We are on stable Queens.As part of an upgrade to accomodate 3X more servers, we decided to add three more compute nodes+ three more ICs for a total of 6 compute nodes and 6 ICs.As soon as we added these in preparation for the 3X increase in servers I am seeing weirdbehaviour. A general question to everyone:How many of you run your baremetal clouds with 5+ computes and ICs?Are things stable with the setup ? Logs and Analysis:all compute and conductor services are up and running. 1) Baremetal node c1bda753-d46c-4379-8d07-7787c2a4a7f2 mapped to sc-ironic08root@stg-cl1-dev-001:~# openstack hypervisor show c1bda753-d46c-4379-8d07-7787c2a4a7f2 | grep ironic | | service_host | sc-ironic08.nvc.nvidia.com 2)Mac address is 6c:b3:11:4f:8a:c0root@stg-cl1-dev-001:~# openstack baremetal port list --node c1bda753-d46c-4379-8d07-7787c2a4a7f2+--------------------------------------+-------------------+| UUID | Address |+--------------------------------------+-------------------+| a517fb41-f977-438d-8c0d-21046e2918d9 | 6c:b3:11:4f:8a:c0 |+--------------------------------------+-------------------+ 2)Provisioning starts: ironic06 receives the VIF update: WHY ?2020-04-21 15:05:47.509 71431 INFO ironic.conductor.manager VIF 657fea31-3218-4f10-b6ad-8b6a0fa7bab8 successfully attached to node c1bda753-d46c-4379-8d07-7787c2a4a7f2 ironic08 (correct one) also receives updates.[root@sc-ironic08 master_images]# tail -f /var/log/ironic/ironic-conductor.log | grep c1bda753-d46c-4379-8d07-7787c2a4a7f22020-04-21 15:08:04.943 27542 INFO ironic.conductor.task_manager [req-259b0175-65bc-4707-8c88-a65189a29954 - - - - -] Node c1bda753-d46c-4379-8d07-7787c2a4a7f2 moved to provision state "deploying" from state "wait call-back"; target provision state is "active" For now we have backed down to 3 and are stable again but I would really like to overprovision our computes and conductors if possible. Please let me know your thoughts and if anything rings a bell. thanks,Fred.

3 4

[glance][ptg[ Virtual PTG
by Abhishek Kekane 27 Apr '20

27 Apr '20

Hi Stackers, As you have noticed due to the current Covid-19 situation we will be having a virtual PTG for Victoria [1]. The PTG will take place from 1st to 5th June 2020 with various slots [2]. So I created a doodle to ask You what would be the best time slots for You to participate - it's available at [3]. If You are planning to attend this event, please mark what time slots are best for You. I will try to book some slots to fit the needs of the team. If You can attend only on some specific slots, and You will have some topics to discuss, we can try to book some slots in time which will work for You and schedule discussions about Your topics on that slot. As we should book slots in about next 2 weeks, it would be great if You could do this doodle before e.g. end of next week (Friday 24th of April). We also have our etherpad [4] - please add Your name to it (at top) if You are planning to attend this virtual event. And please also add proposals of the topics which You want to be discussed there. If you have any suggestions or doubt you can ping me on IRC (abhishekk) or any glance core member at #openstack-glance. I will compile the responses on 24th April and will book the slots accordingly. [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014126.ht… [2] https://ethercalc.openstack.org/126u8ek25noy [3] https://doodle.com/poll/6a46283tii7gt2aa [4] https://etherpad.opendev.org/p/Glance-Victoria-PTG-planning Thanks & Best Regards, Abhishek Kekane

1 2

[release] Ussuri for vitarge
by Eyal B 27 Apr '20

27 Apr '20

Hi, Is there a reason Vitrage, vitrage client and vitrage-dashboard didn't get a patch for creating the Ussuri branch ? Eyal

2 1

[neutron] Bug deputy report for week of April 20th
by Nate Johnston 27 Apr '20

27 Apr '20

Nate bug deputy notes - 2020-04-20 to 2020-04-27 ------------------------------------------------ Untriaged: - "[VPNaas-DVR]The Traffic is unreachable after the virtual machine is bound to the floating IP" * URL: https://bugs.launchpad.net/bugs/1874392 * Note: Need a VPNaaS expert to look at this one. - "After the vm unbinds the floating ip, there are residual ip rules in the qrouter namespace" * URL: https://bugs.launchpad.net/bugs/1874420 * Version: stable/stein * Unable to reproduce locally Critical: - "[stable/stein] Neutron-tempest-plugin jobs are failing due to wrong python version" * URL: https://bugs.launchpad.net/bugs/1873776 * Version: stable/stein * Assignee: bcafarel * Fix: https://review.opendev.org/721277 - "[L3HA] Keepalived 2.x.x tracks state of virtual_ipaddresses interfaces and router now" * URL: https://bugs.launchpad.net/bugs/1874211 * Assignee: slaweq * Fix: https://review.opendev.org/721799 https://review.opendev.org/721805 High: - "[OVN] Tempest test neutron_tempest_plugin.scenario.test_trunk.TrunkTest.test_trunk_subport_lifecycle fails randomly" * URL: https://bugs.launchpad.net/bugs/1874447 * Unassigned - "[OVN] Stale ports can be present in OVN NB leading to metadata errors" * URL: https://bugs.launchpad.net/bugs/1874733 * Assignee: maciejjozefczyk/lucasgomes * Fix: https://review.opendev.org/722789 https://review.opendev.org/723404 Medium: - "test_multicast_between_vms_on_same_network failures" * URL: https://bugs.launchpad.net/bugs/1873798 * Assignee: lucasgomes * Fix: https://review.opendev.org/721262 - "[OVN] extra_dhcp_opts does not validate any options" * URL: https://bugs.launchpad.net/bugs/1874282 * Assignee: lucasgomes * Fix: https://review.opendev.org/722025 - "Cleanup in neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON may fail in dvr deployments" * URL: https://bugs.launchpad.net/bugs/1875344 * Assignee: slaweq * Fix: https://review.opendev.org/723387 Low: - "[agent][dhcp] When revision_number matches the one in cache it is not considered stale" * URL: https://bugs.launchpad.net/bugs/1874400 * Unassigned Invalid/Incomplete: - "Cannot create fip namespace when dhcp pool is insufficient" * URL: https://bugs.launchpad.net/bugs/1875153 Last week's report by tidwellr: http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014332.ht…

1 0

[ops][cinder] Policy for volume attach/detach
by Massimo Sgaravatto 27 Apr '20

27 Apr '20

I would like to set a policy so that attachments operations can be done only by the user who created that volume. To do that I created this [*] policy.yaml file. I verified that with such policy file: - I am able to attach volumes only for the volumes I created - I can attach my volumes also to instances owned by other users - I can not attach volumes belonging to other users to my instances So far so good. But I am allowed to detach any volume from any instance, even if I am not the owner of that volume, and this is not what I want What am I doing wrong ? Thanks, Massimo [*] # # To be used when another member of the same project can't change something # created by another user of the same project "admin_or_user": "is_admin:True or (role:admin and is_admin_project:True) or user_id:%(user_id)s" # Create attachment. # POST /attachments "volume:attachment_create": "rule:admin_or_user" # Update attachment. # PUT /attachments/{attachment_id} "volume:attachment_update": "rule:admin_or_user" # Delete attachment. # DELETE /attachments/{attachment_id} "volume:attachment_delete": "rule:admin_or_user" # Mark a volume attachment process as completed (in-use) # POST /attachments/{attachment_id}/action (os-complete) "volume:attachment_complete": "rule:admin_or_user"

2 2

[neutron] Virtual PTG planning
by Slawek Kaplonski 26 Apr '20

26 Apr '20

Hi neutrinos, As You probably noticed, there is already plan for Virtual PTG in June. See mail from Kendall [1] for details. So I created doodle to ask You what would be the best time slots for You to participate - it's available at [2]. If You are planning to attend this event, please mark what time slots are best for You. I will try to book some slots to fit needs of the team. If You can attend only on some specific slots, and You will have some topics to discuss, we can try to book some slot in time which will work for You and schedule discussions about Your topics on that slot. As we should book slots in about next 2 weeks, it would be great if You could do this doodle before e.g. end of next week (Friday 24th of April). BTW, we also have our etherpad [3] - please add Your name to it if You are planning to attend this virtual event. And please also add proposals of the topics which You want to be discussed there. [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014126.ht… [2] https://doodle.com/poll/xcd82ea6svs5dben [3] https://etherpad.opendev.org/p/neutron-victoria-ptg -- Slawek Kaplonski Senior software engineer Red Hat

2 2

[release] Release countdown for week R-2, April 27 - May 1
by Sean McGinnis 26 Apr '20

26 Apr '20

Development Focus ----------------- At this point we should have release candidates (RC1 or recent intermediary release) for all the Ussuri deliverables. Teams should be working on any release-critical bugs that would require another RC or intermediary release before the final release. Actions ------- Early in the week, the release team will be proposing stable/ussuri branch creation for all deliverables that have not branched yet, using the latest available Ussuri release as the branch point. If your team is ready to go for creating that branch, please let us know by leaving a +1 on these patches. If you would like to wait for another release before branching, you can -1 the patch and update it later in the week with the new release you would like to use. By the end of the week the release team will merge those patches though, unless an exception is granted. Once stable/ussuri branches are created, if a release-critical bug is detected, you will need to fix the issue in the master branch first, then backport the fix to the stable/ussuri branch before releasing out of the stable/ussuri branch. After all of the cycle-with-rc projects have branched we will branch devstack, grenade, and the requirements repos. This will effectively open them up for Victoria development, though the focus should still be on finishing up Ussuri until the final release. For projects with translations, watch for any translation patches coming through and merge them quickly. A new release should be produced so that translations are included in the final Ussuri release. Finally, now is a good time to finalize release notes. In particular, consider adding any relevant "prelude" content. Release notes are targeted for the downstream consumers of your project, so it would be great to include any useful information for those that are going to pick up and use or deploy the Ussuri version of your project. Upcoming Deadlines & Dates -------------------------- Final RC deadline: May 7 (R-1 week) Final Ussuri release: May 13 Virtual Victoria PTG: June 1-5

2 1

[goals][Drop Python 2.7 Support] Week R-3 Update
by Ghanshyam Mann 26 Apr '20

26 Apr '20

Hello Everyone, Below is the progress on "Drop Python 2.7 Support" at end of R-3 week. Schedule: https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html#sc… Highlights: ======== * Storlets is also keeping py2.7 support. py2.7 drop patches are reverted. * Aduit is finished and missing repos patches are also proposed. * Requesting projects listed below to merge the open patches asap. Project not completed the py2.7 drop: ============================ * adjutant-ui: https://review.opendev.org/#/c/714636/ * pythonclient: ** python-barbicanclient: https://review.opendev.org/#/c/699096/ *** gate is already broken. The last patch merged in this repo was 6 months ago so I am not sure it is maintained or not. ** python-zaqarclient: https://review.opendev.org/#/c/692011/ *** This also the same story, not maintained and job is broken already. ** python-adjutantclient https://review.opendev.org/#/c/723128/ * Deployment projects: *** Openstack Charms - Most of them merged, few failing on func job. *** Openstackansible - Centos7 jobs are failing on a few projects. *** Openstack-Helm - No conclusion yet on what else to do from Helm team. *** TripleO: Below repo are found to drop py2.7 during audit **** paunch: https://review.opendev.org/#/c/700560/ **** https://review.opendev.org/#/c/723125/ **** https://review.opendev.org/#/c/723126/ **** https://review.opendev.org/#/c/723127/ * Open review: https://review.opendev.org/#/q/topic:drop-py27-support+status:open Phase-3 status: I did the audit and found a few repos were missed. I have proposed the patches for those (listed above). All other projects have complete the work. I will discuss with the requirement team is anything needs to be done on the requirement side. I feel we should keep going with py2.7 cap in u-c. -gmann

1 0

[openstack-dev] [Tacker]
by Ivandro 25 Apr '20

25 Apr '20

Hello, I have been trying to install the tacker project in a VM, following the instruction from this url: https://docs.openstack.org/tacker/train/install/devstack.html based on devstack but unsuccessful. Is there a link to download an already installed VM with tacker? BR

6 15

Re: [release-announce] keystone_tempest_plugin 0.4.0 (ussuri)
by Thomas Goirand 24 Apr '20

24 Apr '20

On 4/23/20 9:32 PM, no-reply(a)openstack.org wrote: > We are amped to announce the release of: > > keystone_tempest_plugin 0.4.0: Tempest plugin keystone_tempest_plugin > > This release is part of the ussuri release series. > > The source is available from: > > https://opendev.org/openstack/keystone_tempest_plugin This one also has underscores instead of dashes... :/ Thomas

2 1

[user committee] Survey tool research
by Dincer Celik 24 Apr '20

24 Apr '20

Hi everyone! As Kolla project, we are looking for a survey tool to get feedback from our users. We are now on the comparison stage of some popular tools and would like to ask what user committee uses for the user surveys. The current tool seems hard coded to website but I would like to ask if it could be used for a custom survey. Best regards, Dincer

2 1

[octavia] Re: [Release-job-failures] release-post job for openstack/releases for ref refs/heads/master failed
by Sean McGinnis 24 Apr '20

24 Apr '20

> Build failed. > > - tag-releases https://zuul.opendev.org/t/openstack/build/8831fb0606114aae904ba66386b0b0eb : FAILURE in 4m 26s > - publish-tox-docs-releases https://zuul.opendev.org/t/openstack/build/None : SKIPPED > This was a failure during the tag processing for the Octavia repo. There are a few steps that are done when a new branch is created for a repo. These are typically: 1. Update the .gitreview file on the new stable branch to include the setting pointing at the right branch. 2. Updating any upper-constraints links to point to the stable branch version of those constraints. 3. Add a new release notes landing page for the stable branch in the master release notes index. 4. Switch the python3 series jobs on master to the new series (ussuri > victoria). The first step completed successfully. The second step failed due to some repo specific files checks for constraints urls. The job has typically only updated constraints in tox.ini, but handling was added for other files used in this repo. Those files were updated, but the change missed the step of adding those files before committing and attempting to push up the review. Since git review does a rebase by default when submitting, git errored out due to the presence of unstaged files. That step has now bee manually submitted: https://review.opendev.org/#/c/722489/ And the fix for the job that caused the failure has been submitted with: https://review.opendev.org/#/c/722491/ The third step has been manually submitted with: https://review.opendev.org/#/c/722492/ And step four with: https://review.opendev.org/#/c/722493/ Everything should be set now. The skipped docs job will be caught up with the next merged release, so we should be good. If there are any issues that crop up from this, just let me know. Sean

2 1

Re: [release-announce] telemetry_tempest_plugin 1.0.0 (ussuri)
by Thomas Goirand 24 Apr '20

24 Apr '20

On 4/22/20 3:28 PM, no-reply(a)openstack.org wrote: > We are happy to announce the release of: > > telemetry_tempest_plugin 1.0.0: Tempest plugin for Telemetry Projects > > This release is part of the ussuri release series. > > The source is available from: > > https://opendev.org/openstack/telemetry_tempest_plugin Wrong link, it's with dashes, not underscores. I'm not sure where this gets fixed... Thomas Goirand (zigo)

3 2

[tc][tacker]Poll for Tacker PTL role
by Rico Lin 24 Apr '20

24 Apr '20

Hi all As Tacker team decided in meeting today [1] We will run an election inner Tacker team and provide one name for TCs to appoint as PTL The candidate list includes: * Hyunsik Yang ( http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014045.ht… ) * Yasufumi Ogawa ( http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014044.ht… ) The election will end at 2020/04/23 23:59 UTC. Please vote before that time. If you commit to Tacker project within a year, you should receive an email with civs link for voting. If you're a valid voter but didn't receive any email. Please contact me with links to your contribution and I'm more than happy to fix that for you. [1] http://eavesdrop.openstack.org/meetings/tacker/2020/tacker.2020-04-21-08.00…

2 2

[cinder] victoria virtual PTG time slots
by Brian Rosmaita 23 Apr '20

23 Apr '20

I've signed up Cinder for the following timeslots in the "Havana" room for the Victoria virtual PTG: Tuesday, June 2: 13:00-16:00 UTC Wednesday, June 3: 13:00-16:00 UTC Thursday, June 4: 13:00-16:00 UTC Friday, June 5: 13:00-16:00 UTC Not very creative (same 3 hour period each day), but I couldn't find a better solution. Details on the planning etherpad if you want to see if you can come up with something better: https://etherpad.opendev.org/p/cinder-victoria-ptg-planning cheers, brian

1 0

Re: [Release-job-failures] Pre-release of openstack/mistral-dashboard for ref refs/tags/10.0.0.0rc1 failed
by Thierry Carrez 23 Apr '20

23 Apr '20

zuul(a)openstack.org wrote: > Build failed. > > - release-openstack-python https://zuul.opendev.org/t/openstack/build/dacb9f52eaa345b397a28e6dcdf54187 : SUCCESS in 3m 49s > - announce-release https://zuul.opendev.org/t/openstack/build/254ac00652174e60869fd1caa11ab261 : FAILURE in 6m 34s > - propose-update-constraints https://zuul.opendev.org/t/openstack/build/1a94db3e1ad5473ab4814555ac9b2df4 : SUCCESS in 3m 27s Transient failure ("Temporary local problem") sending email: + send-mail -v /home/zuul/relnotes/mistral-dashboard-10.0.0.0rc1 send: 'ehlo [127.0.0.1]\r\n' reply: b'250-lists.openstack.org Hello [127.0.0.1] [108.44.198.35]\r\n' reply: b'250-SIZE 52428800\r\n' reply: b'250-8BITMIME\r\n' reply: b'250-PIPELINING\r\n' reply: b'250 HELP\r\n' reply: retcode (250); Msg: b'lists.openstack.org Hello [127.0.0.1] [108.44.198.35]\nSIZE 52428800\n8BITMIME\nPIPELINING\nHELP' send: 'mail FROM:<no-reply(a)openstack.org> size=1098\r\n' reply: b'250 OK\r\n' reply: retcode (250); Msg: b'OK' send: 'rcpt TO:<release-announce(a)lists.openstack.org>\r\n' reply: b'451 Temporary local problem - please try later\r\n' reply: retcode (451); Msg: b'Temporary local problem - please try later' This was for mistral-dashboard rc1, I think we can survive that announce not being sent. -- Thierry Carrez (ttx)

2 2

RDO | OpenStack VMs | XFS Metadata Corruption
by Pradeep Antil 23 Apr '20

23 Apr '20

Hi Techies, I have below RDO setup, - RDO 13 - Base OS for Controllers & Compute is Ubuntu - Neutron with vxlan + VLAN (for provider N/W) - Cinder backend is CHEF - HugePages and CPU Pinning for VNF's VMs I am trying to deploy a stack which is suppose to create 18 VMs across 11 computes nodes internal disk, but every time 3 to 4 VMs out of 18 doesn't spawned properly. At console of these VMs i am getting below errors, Any idea and suggestion how to troubleshoot this? and resolve the issue. [ 100.681552] ffff8b37f8f86020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 100.681553] ffff8b37f8f86030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 100.681560] XFS (vda1): Metadata corruption detected at xfs_inode_buf_verify+0x79/0x100 [xfs], xfs_inode block 0x179b800 [ 100.681561] XFS (vda1): Unmount and run xfs_repair [ 100.681561] XFS (vda1): First 64 bytes of corrupted metadata buffer: [ 100.681562] ffff8b37f8f86000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 100.681562] ffff8b37f8f86010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 100.681563] ffff8b37f8f86020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 100.681564] ffff8b37f8f86030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 100.681596] XFS (vda1): metadata I/O error: block 0x179b800 ("xfs_trans_read_buf_map") error 117 numblks 32 [ 100.681599] XFS (vda1): xfs_imap_to_bp: xfs_trans_read_buf() returned error -117. [ 99.585766] [cloud-init[32m OK [0m[2530]: ] Cloud-init v. 18.2 running 'init-local' at Thu, 16 Apr 2020 10:44:21 +0000. Up 99.55 seconds.Started oVirt Guest Agent. [ 101.086566] XFS (vda1): Metadata corruption detected at xfs_inode_buf_verify+0x79/0x100 [xfs], xfs_inode block 0x179b800 [ 101.092093] XFS (vda1): Unmount and run xfs_repair [ 101.094660] XFS (vda1): First 64 bytes of corrupted metadata buffer: [ 101.097787] ffff8b37fef07000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.105959] ffff8b37fef07010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.110718] ffff8b37fef07020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.115412] ffff8b37fef07030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.120166] XFS (vda1): Metadata corruption detected at xfs_inode_buf_verify+0x79/0x100 [xfs], xfs_inode block 0x179b800 [ 101.125644] XFS (vda1): Unmount and run xfs_repair [ 101.128229] XFS (vda1): First 64 bytes of corrupted metadata buffer: [ 101.131370] ffff8b37fef07000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.138671] ffff8b37fef07010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.143427] ffff8b37fef07020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.148235] ffff8b37fef07030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.152999] XFS (vda1): Metadata corruption detected at xfs_inode_buf_verify+0x79/0x100 [xfs], xfs_inode block 0x179b800 [ 101.158479] XFS (vda1): Unmount and run xfs_repair [ 101.161068] XFS (vda1): First 64 bytes of corrupted metadata buffer: [ 101.169883] ffff8b37fef07000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.174751] ffff8b37fef07010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.179639] ffff8b37fef07020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.184285] ffff8b37fef07030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.189104] XFS (vda1): Metadata corruption detected at xfs_inode_buf_verify+0x79/0x100 [xfs], xfs_inode block 0x179b800 [ 101.194619] XFS (vda1): Unmount and run xfs_repair [ 101.197228] XFS (vda1): First 64 bytes of corrupted metadata buffer: [ 101.201109] ffff8b37fef07000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.205976] ffff8b37fef07010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.210709] ffff8b37fef07020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.215442] ffff8b37fef07030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.220196] XFS (vda1): Metadata corruption detected at xfs_inode_buf_verify+0x79/0x100 [xfs], xfs_inode block 0x179b800 [ 101.225708] XFS (vda1): Unmount and run xfs_repair [ 101.228296] XFS (vda1): First 64 bytes of corrupted metadata buffer: [ 101.232058] ffff8b37fef07000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.236803] ffff8b37fef07010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.241538] ffff8b37fef07020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.246252] ffff8b37fef07030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.250997] XFS (vda1): Metadata corruption detected at xfs_inode_buf_verify+0x79/0x100 [xfs], xfs_inode block 0x179b800 [ 101.256518] XFS (vda1): Unmount and run xfs_repair [ 101.259105] XFS (vda1): First 64 bytes of corrupted metadata buffer: [ 101.262912] ffff8b37fef07000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.267649] ffff8b37fef07010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.272360] ffff8b37fef07020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.277088] ffff8b37fef07030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.281831] XFS (vda1): Metadata corruption detected at xfs_inode_buf_verify+0x79/0x100 [xfs], xfs_inode block 0x179b800 [ 101.287322] XFS (vda1): Unmount and run xfs_repair [ 101.295401] XFS (vda1): First 64 bytes of corrupted metadata buffer: [ 101.298546] ffff8b37fef07000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.303283] ffff8b37fef07010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.308009] ffff8b37fef07020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.312747] ffff8b37fef07030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.317460] XFS (vda1): Metadata corruption detected at xfs_inode_buf_verify+0x79/0x100 [xfs], xfs_inode block 0x179b800 [ 101.322960] XFS (vda1): Unmount and run xfs_repair [ 101.326233] XFS (vda1): First 64 bytes of corrupted metadata buffer: [ 101.329383] ffff8b37fef07000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.334100] ffff8b37fef07010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.338822] ffff8b37fef07020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.343549] ffff8b37fef07030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 101.348297] XFS (vda1): Metadata corruption detected at xfs_inode_buf_verify+0x79/0x100 [xfs], xfs_inode block 0x179b800 [ 101.353793] XFS (vda1): Unmount and run xfs_repair [ 101.357102] XFS (vda1): First 64 bytes of corrupted metadata buffer: Below are the Nova Compute logs of the hypervisor where it is scheduled to spawned, 3T06:04:55Z,direct_url=<?>,disk_format='qcow2',id=c255bbbc-c8c3-462e-b827-1d35db08d283,min_disk=0,min_ram=0,name='vnf-scef-18.5',owner='36c70ae400e74fc2859f44815d0c9afb',properties=ImageMetaProps,protected=<?>,size=7143292928,status='active',tags=<?>,updated_at=2020-03-03T06:05:49Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'swap': None, 'root_device_name': u'/dev/vda', 'ephemerals': [], 'block_device_mapping': []} _get_guest_xml /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:5419 2020-04-16 16:12:28.310 219284 DEBUG oslo_concurrency.processutils [req-5a53263c-928c-4a0c-a03c-8b698339efca cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/dfa80e78-ee02-46e5-ba7a-0874fa37da56/disk --force-share" returned: 0 in 0.031s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:12:28.310 219284 DEBUG oslo_concurrency.processutils [req-5a53263c-928c-4a0c-a03c-8b698339efca cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Running cmd (subprocess): qemu-img resize /var/lib/nova/instances/dfa80e78-ee02-46e5-ba7a-0874fa37da56/disk 64424509440 execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:12:28.322 219284 DEBUG oslo_concurrency.processutils [req-5a53263c-928c-4a0c-a03c-8b698339efca cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] CMD "qemu-img resize /var/lib/nova/instances/dfa80e78-ee02-46e5-ba7a-0874fa37da56/disk 64424509440" returned: 0 in 0.012s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:12:28.323 219284 DEBUG oslo_concurrency.lockutils [req-5a53263c-928c-4a0c-a03c-8b698339efca cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Lock "86692cd1e738b8df7cf1f951967c61e92222fc4c" released by "nova.virt.libvirt.imagebackend.copy_qcow2_image" :: held 0.092s inner /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:285 2020-04-16 16:12:28.323 219284 DEBUG oslo_concurrency.processutils [req-5a53263c-928c-4a0c-a03c-8b698339efca cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/_base/86692cd1e738b8df7cf1f951967c61e92222fc4c --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:12:28.338 219284 DEBUG nova.virt.libvirt.driver [req-a7ee4c3e-ea3a-4237-ba75-4c85411c9889 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] CPU mode 'host-model' model '' was chosen, with extra flags: '' _get_guest_cpu_model_config /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:3909 2020-04-16 16:12:28.338 219284 DEBUG nova.virt.hardware [req-a7ee4c3e-ea3a-4237-ba75-4c85411c9889 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Getting desirable topologies for flavor Flavor(created_at=2020-03-23T11:20:41Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw:cpu_policy='dedicated',hw:mem_page_size='1048576'},flavorid='03e45d45-f4f4-4c24-8b70-678c3703402f',id=102,is_public=False,memory_mb=49152,name='dmdc-traffic-flavor',projects=<?>,root_gb=60,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=16) and image_meta ImageMeta(checksum='69f8c18e59db9669d669d04824507a82',container_format='bare',created_at=2020-03-03T06:07:18Z,direct_url=<?>,disk_format='qcow2',id=d31e39bc-c2b7-42ad-968f-7e782dd72943,min_disk=0,min_ram=0,name='vnf-dmdc-18.5.0',owner='36c70ae400e74fc2859f44815d0c9afb',properties=ImageMetaProps,protected=<?>,size=5569380352,status='active',tags=<?>,updated_at=2020-03-03T06:08:03Z,virtual_size=<?>,visibility=<?>), allow threads: True _get_desirable_cpu_topologies /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:551 2020-04-16 16:12:28.339 219284 DEBUG nova.virt.hardware [req-a7ee4c3e-ea3a-4237-ba75-4c85411c9889 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Flavor limits 65536:65536:65536 _get_cpu_topology_constraints /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:297 2020-04-16 16:12:28.339 219284 DEBUG nova.virt.hardware [req-a7ee4c3e-ea3a-4237-ba75-4c85411c9889 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Image limits 65536:65536:65536 _get_cpu_topology_constraints /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:308 2020-04-16 16:12:28.339 219284 DEBUG nova.virt.hardware [req-a7ee4c3e-ea3a-4237-ba75-4c85411c9889 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Flavor pref -1:-1:-1 _get_cpu_topology_constraints /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:331 2020-04-16 16:12:28.340 219284 DEBUG nova.virt.hardware [req-a7ee4c3e-ea3a-4237-ba75-4c85411c9889 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Image pref -1:-1:-1 _get_cpu_topology_constraints /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:350 2020-04-16 16:12:28.340 219284 DEBUG nova.virt.hardware [req-a7ee4c3e-ea3a-4237-ba75-4c85411c9889 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Chosen -1:-1:-1 limits 65536:65536:65536 _get_cpu_topology_constraints /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:379 2020-04-16 16:12:28.340 219284 DEBUG nova.virt.hardware [req-a7ee4c3e-ea3a-4237-ba75-4c85411c9889 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Topology preferred VirtCPUTopology(cores=-1,sockets=-1,threads=-1), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) _get_desirable_cpu_topologies /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:555 2020-04-16 16:12:28.340 219284 DEBUG nova.virt.hardware [req-a7ee4c3e-ea3a-4237-ba75-4c85411c9889 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Build topologies for 16 vcpu(s) 16:16:16 _get_possible_cpu_topologies /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:418 2020-04-16 16:12:28.341 219284 DEBUG nova.virt.hardware [req-a7ee4c3e-ea3a-4237-ba75-4c85411c9889 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Got 15 possible topologies _get_possible_cpu_topologies /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:445 2020-04-16 16:12:28.341 219284 DEBUG nova.virt.hardware [req-a7ee4c3e-ea3a-4237-ba75-4c85411c9889 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Possible topologies [VirtCPUTopology(cores=1,sockets=16,threads=1), VirtCPUTopology(cores=2,sockets=8,threads=1), VirtCPUTopology(cores=4,sockets=4,threads=1), VirtCPUTopology(cores=8,sockets=2,threads=1), VirtCPUTopology(cores=16,sockets=1,threads=1), VirtCPUTopology(cores=1,sockets=8,threads=2), VirtCPUTopology(cores=2,sockets=4,threads=2), VirtCPUTopology(cores=4,sockets=2,threads=2), VirtCPUTopology(cores=8,sockets=1,threads=2), VirtCPUTopology(cores=1,sockets=4,threads=4), VirtCPUTopology(cores=2,sockets=2,threads=4), VirtCPUTopology(cores=4,sockets=1,threads=4), VirtCPUTopology(cores=1,sockets=2,threads=8), VirtCPUTopology(cores=2,sockets=1,threads=8), VirtCPUTopology(cores=1,sockets=1,threads=16)] _get_desirable_cpu_topologies /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:560 2020-04-16 16:12:28.341 219284 DEBUG nova.virt.hardware [req-a7ee4c3e-ea3a-4237-ba75-4c85411c9889 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Filtering topologies best for 2 threads _get_desirable_cpu_topologies /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:578 2020-04-16 16:12:28.342 219284 DEBUG nova.virt.hardware [req-a7ee4c3e-ea3a-4237-ba75-4c85411c9889 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Remaining possible topologies [VirtCPUTopology(cores=1,sockets=8,threads=2), VirtCPUTopology(cores=2,sockets=4,threads=2), VirtCPUTopology(cores=4,sockets=2,threads=2), VirtCPUTopology(cores=8,sockets=1,threads=2)] _get_desirable_cpu_topologies /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:583 2020-04-16 16:12:28.342 219284 DEBUG nova.virt.hardware [req-a7ee4c3e-ea3a-4237-ba75-4c85411c9889 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=8,threads=2), VirtCPUTopology(cores=2,sockets=4,threads=2), VirtCPUTopology(cores=4,sockets=2,threads=2), VirtCPUTopology(cores=8,sockets=1,threads=2)] _get_desirable_cpu_topologies /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:586 2020-04-16 16:12:28.344 219284 DEBUG nova.virt.libvirt.driver [req-48d96e5d-f071-44e7-94d2-e9fcb2a13087 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] CPU mode 'host-model' model '' was chosen, with extra flags: '' _get_guest_cpu_model_config /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:3909 2020-04-16 16:12:28.345 219284 DEBUG nova.virt.hardware [req-48d96e5d-f071-44e7-94d2-e9fcb2a13087 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Getting desirable topologies for flavor Flavor(created_at=2020-03-23T11:20:34Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw:cpu_policy='dedicated',hw:mem_page_size='1048576'},flavorid='d60b66d4-c0e0-4292-9113-1df2d94d57a5',id=90,is_public=False,memory_mb=57344,name='scef-traffic-flavor',projects=<?>,root_gb=60,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=16) and image_meta ImageMeta(checksum='3fe0e06194e0b5327ba38bb2367f760d',container_format='bare',created_at=2020-03-03T06:04:55Z,direct_url=<?>,disk_format='qcow2',id=c255bbbc-c8c3-462e-b827-1d35db08d283,min_disk=0,min_ram=0,name='vnf-scef-18.5',owner='36c70ae400e74fc2859f44815d0c9afb',properties=ImageMetaProps,protected=<?>,size=7143292928,status='active',tags=<?>,updated_at=2020-03-03T06:05:49Z,virtual_size=<?>,visibility=<?>), allow threads: True _get_desirable_cpu_topologies /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:551 2020-04-16 16:12:28.345 219284 DEBUG nova.virt.hardware [req-48d96e5d-f071-44e7-94d2-e9fcb2a13087 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Flavor limits 65536:65536:65536 _get_cpu_topology_constraints /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:297 2020-04-16 16:12:28.345 219284 DEBUG nova.virt.hardware [req-48d96e5d-f071-44e7-94d2-e9fcb2a13087 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Image limits 65536:65536:65536 _get_cpu_topology_constraints /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:308 2020-04-16 16:12:28.346 219284 DEBUG nova.virt.hardware [req-48d96e5d-f071-44e7-94d2-e9fcb2a13087 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Flavor pref -1:-1:-1 _get_cpu_topology_constraints /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:331 2020-04-16 16:12:28.346 219284 DEBUG nova.virt.hardware [req-48d96e5d-f071-44e7-94d2-e9fcb2a13087 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Image pref -1:-1:-1 _get_cpu_topology_constraints /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:350 2020-04-16 16:12:28.346 219284 DEBUG nova.virt.hardware [req-48d96e5d-f071-44e7-94d2-e9fcb2a13087 cbabd9368dc24fea84fd2e43935fddfa 975a7d3840a141b0a20a9dc60e3da6cd - default default] Chosen -1:-1:-1 limits 65536:65536:65536 _get_cpu_topology_constraints /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/hardware.py:379 packages/nova/network/base_api.py:48 2020-04-16 16:34:48.580 219284 DEBUG oslo_concurrency.lockutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Releasing semaphore "refresh_cache-f33b2602-ac5f-491e-bdb8-7e7f9376bcad" lock /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:228 2020-04-16 16:34:48.580 219284 DEBUG nova.compute.manager [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] [instance: f33b2602-ac5f-491e-bdb8-7e7f9376bcad] Updated the network info_cache for instance _heal_instance_info_cache /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/compute/manager.py:6827 2020-04-16 16:34:50.580 219284 DEBUG oslo_service.periodic_task [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running periodic task ComputeManager._run_image_cache_manager_pass run_periodic_tasks /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_service/periodic_task.py:215 2020-04-16 16:34:50.581 219284 DEBUG oslo_concurrency.lockutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Lock "storage-registry-lock" acquired by "nova.virt.storage_users.do_register_storage_use" :: waited 0.000s inner /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:273 2020-04-16 16:34:50.581 219284 DEBUG oslo_concurrency.lockutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Lock "storage-registry-lock" released by "nova.virt.storage_users.do_register_storage_use" :: held 0.000s inner /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:285 2020-04-16 16:34:50.581 219284 DEBUG oslo_concurrency.lockutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Lock "storage-registry-lock" acquired by "nova.virt.storage_users.do_get_storage_users" :: waited 0.000s inner /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:273 2020-04-16 16:34:50.582 219284 DEBUG oslo_concurrency.lockutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Lock "storage-registry-lock" released by "nova.virt.storage_users.do_get_storage_users" :: held 0.000s inner /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:285 2020-04-16 16:34:50.628 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Verify base images _age_and_verify_cached_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:348 2020-04-16 16:34:50.628 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Image id yields fingerprint da39a3ee5e6b4b0d3255bfef95601890afd80709 _age_and_verify_cached_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:355 2020-04-16 16:34:50.628 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Image id b8783f95-138b-4265-a09d-55ec9d9ad35d yields fingerprint b40b27e04896d063bc591b19642da8910da3eb1f _age_and_verify_cached_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:355 2020-04-16 16:34:50.628 219284 INFO nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] image b8783f95-138b-4265-a09d-55ec9d9ad35d at (/var/lib/nova/instances/_base/b40b27e04896d063bc591b19642da8910da3eb1f): checking 2020-04-16 16:34:50.628 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] image b8783f95-138b-4265-a09d-55ec9d9ad35d at (/var/lib/nova/instances/_base/b40b27e04896d063bc591b19642da8910da3eb1f): image is in use _mark_in_use /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:329 2020-04-16 16:34:50.629 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Image id c255bbbc-c8c3-462e-b827-1d35db08d283 yields fingerprint 86692cd1e738b8df7cf1f951967c61e92222fc4c _age_and_verify_cached_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:355 2020-04-16 16:34:50.630 219284 INFO nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] image c255bbbc-c8c3-462e-b827-1d35db08d283 at (/var/lib/nova/instances/_base/86692cd1e738b8df7cf1f951967c61e92222fc4c): checking 2020-04-16 16:34:50.630 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] image c255bbbc-c8c3-462e-b827-1d35db08d283 at (/var/lib/nova/instances/_base/86692cd1e738b8df7cf1f951967c61e92222fc4c): image is in use _mark_in_use /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:329 2020-04-16 16:34:50.630 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Image id d31e39bc-c2b7-42ad-968f-7e782dd72943 yields fingerprint 5c538ead16d8375e4890e8b9bb1aa080edc75f33 _age_and_verify_cached_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:355 2020-04-16 16:34:50.630 219284 INFO nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] image d31e39bc-c2b7-42ad-968f-7e782dd72943 at (/var/lib/nova/instances/_base/5c538ead16d8375e4890e8b9bb1aa080edc75f33): checking 2020-04-16 16:34:50.630 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] image d31e39bc-c2b7-42ad-968f-7e782dd72943 at (/var/lib/nova/instances/_base/5c538ead16d8375e4890e8b9bb1aa080edc75f33): image is in use _mark_in_use /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:329 2020-04-16 16:34:50.631 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Image id b3af2bf0-055b-48fb-aedc-4683468a3f74 yields fingerprint 7af98c4d49b766d82eec8169a5c87be4eb56e5eb _age_and_verify_cached_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:355 2020-04-16 16:34:50.631 219284 INFO nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] image b3af2bf0-055b-48fb-aedc-4683468a3f74 at (/var/lib/nova/instances/_base/7af98c4d49b766d82eec8169a5c87be4eb56e5eb): checking 2020-04-16 16:34:50.631 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] image b3af2bf0-055b-48fb-aedc-4683468a3f74 at (/var/lib/nova/instances/_base/7af98c4d49b766d82eec8169a5c87be4eb56e5eb): image is in use _mark_in_use /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:329 2020-04-16 16:34:50.632 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] f33b2602-ac5f-491e-bdb8-7e7f9376bcad is a valid instance name _list_backing_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:169 2020-04-16 16:34:50.632 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] f33b2602-ac5f-491e-bdb8-7e7f9376bcad has a disk file _list_backing_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:172 2020-04-16 16:34:50.632 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/f33b2602-ac5f-491e-bdb8-7e7f9376bcad/disk --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:34:50.663 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/f33b2602-ac5f-491e-bdb8-7e7f9376bcad/disk --force-share" returned: 0 in 0.031s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:34:50.664 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Instance f33b2602-ac5f-491e-bdb8-7e7f9376bcad is backed by b40b27e04896d063bc591b19642da8910da3eb1f _list_backing_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:187 2020-04-16 16:34:50.665 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] f117eb96-06a9-4c91-9c5c-111228e24d66 is a valid instance name _list_backing_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:169 2020-04-16 16:34:50.665 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] f117eb96-06a9-4c91-9c5c-111228e24d66 has a disk file _list_backing_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:172 2020-04-16 16:34:50.665 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/f117eb96-06a9-4c91-9c5c-111228e24d66/disk --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:34:50.694 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/f117eb96-06a9-4c91-9c5c-111228e24d66/disk --force-share" returned: 0 in 0.029s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:34:50.695 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Instance f117eb96-06a9-4c91-9c5c-111228e24d66 is backed by 7af98c4d49b766d82eec8169a5c87be4eb56e5eb _list_backing_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:187 2020-04-16 16:34:50.695 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] 5ba39de3-f5f8-46a2-908d-c43b901e1696 is a valid instance name _list_backing_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:169 2020-04-16 16:34:50.695 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] 5ba39de3-f5f8-46a2-908d-c43b901e1696 has a disk file _list_backing_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:172 2020-04-16 16:34:50.695 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/5ba39de3-f5f8-46a2-908d-c43b901e1696/disk --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:34:50.723 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/5ba39de3-f5f8-46a2-908d-c43b901e1696/disk --force-share" returned: 0 in 0.028s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:34:50.724 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Instance 5ba39de3-f5f8-46a2-908d-c43b901e1696 is backed by 86692cd1e738b8df7cf1f951967c61e92222fc4c _list_backing_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:187 2020-04-16 16:34:50.724 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] d3d2837b-49c3-4822-b26b-4b3c03d344ae is a valid instance name _list_backing_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:169 2020-04-16 16:34:50.724 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] d3d2837b-49c3-4822-b26b-4b3c03d344ae has a disk file _list_backing_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:172 2020-04-16 16:34:50.725 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/d3d2837b-49c3-4822-b26b-4b3c03d344ae/disk --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:34:50.752 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/d3d2837b-49c3-4822-b26b-4b3c03d344ae/disk --force-share" returned: 0 in 0.028s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:34:50.753 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Instance d3d2837b-49c3-4822-b26b-4b3c03d344ae is backed by 5c538ead16d8375e4890e8b9bb1aa080edc75f33 _list_backing_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:187 2020-04-16 16:34:50.753 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] dfa80e78-ee02-46e5-ba7a-0874fa37da56 is a valid instance name _list_backing_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:169 2020-04-16 16:34:50.754 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] dfa80e78-ee02-46e5-ba7a-0874fa37da56 has a disk file _list_backing_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:172 2020-04-16 16:34:50.754 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/dfa80e78-ee02-46e5-ba7a-0874fa37da56/disk --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:34:50.781 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/dfa80e78-ee02-46e5-ba7a-0874fa37da56/disk --force-share" returned: 0 in 0.027s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:34:50.782 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Instance dfa80e78-ee02-46e5-ba7a-0874fa37da56 is backed by 86692cd1e738b8df7cf1f951967c61e92222fc4c _list_backing_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:187 2020-04-16 16:34:50.782 219284 INFO nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Active base files: /var/lib/nova/instances/_base/b40b27e04896d063bc591b19642da8910da3eb1f /var/lib/nova/instances/_base/86692cd1e738b8df7cf1f951967c61e92222fc4c /var/lib/nova/instances/_base/5c538ead16d8375e4890e8b9bb1aa080edc75f33 /var/lib/nova/instances/_base/7af98c4d49b766d82eec8169a5c87be4eb56e5eb 2020-04-16 16:34:50.783 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Verification complete _age_and_verify_cached_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:384 2020-04-16 16:34:50.783 219284 DEBUG nova.virt.libvirt.imagecache [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Verify swap images _age_and_verify_swap_images /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/imagecache.py:333 2020-04-16 16:35:01.887 219284 DEBUG oslo_service.periodic_task [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running periodic task ComputeManager.update_available_resource run_periodic_tasks /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_service/periodic_task.py:215 2020-04-16 16:35:01.910 219284 DEBUG nova.compute.resource_tracker [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Auditing locally available compute resources for KO1A3D02O131106CM07 (node: KO1A3D02O131106CM07.openstack.local) update_available_resource /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/compute/resource_tracker.py:689 2020-04-16 16:35:02.009 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/f33b2602-ac5f-491e-bdb8-7e7f9376bcad/disk --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:35:02.040 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/f33b2602-ac5f-491e-bdb8-7e7f9376bcad/disk --force-share" returned: 0 in 0.031s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:35:02.041 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/f33b2602-ac5f-491e-bdb8-7e7f9376bcad/disk --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:35:02.070 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/f33b2602-ac5f-491e-bdb8-7e7f9376bcad/disk --force-share" returned: 0 in 0.029s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:35:02.070 219284 DEBUG nova.virt.libvirt.driver [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] skipping disk for instance-00000636 as it does not have a path _get_instance_disk_info_from_config /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:7840 2020-04-16 16:35:02.071 219284 DEBUG nova.virt.libvirt.driver [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] skipping disk for instance-00000636 as it does not have a path _get_instance_disk_info_from_config /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:7840 2020-04-16 16:35:02.073 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/d3d2837b-49c3-4822-b26b-4b3c03d344ae/disk --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:35:02.101 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/d3d2837b-49c3-4822-b26b-4b3c03d344ae/disk --force-share" returned: 0 in 0.028s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:35:02.101 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/d3d2837b-49c3-4822-b26b-4b3c03d344ae/disk --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:35:02.129 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/d3d2837b-49c3-4822-b26b-4b3c03d344ae/disk --force-share" returned: 0 in 0.028s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:35:02.132 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/dfa80e78-ee02-46e5-ba7a-0874fa37da56/disk --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:35:02.159 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/dfa80e78-ee02-46e5-ba7a-0874fa37da56/disk --force-share" returned: 0 in 0.028s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:35:02.160 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/dfa80e78-ee02-46e5-ba7a-0874fa37da56/disk --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:35:02.187 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/dfa80e78-ee02-46e5-ba7a-0874fa37da56/disk --force-share" returned: 0 in 0.027s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:35:02.190 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/f117eb96-06a9-4c91-9c5c-111228e24d66/disk --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:35:02.217 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/f117eb96-06a9-4c91-9c5c-111228e24d66/disk --force-share" returned: 0 in 0.027s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:35:02.218 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/f117eb96-06a9-4c91-9c5c-111228e24d66/disk --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:35:02.245 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/f117eb96-06a9-4c91-9c5c-111228e24d66/disk --force-share" returned: 0 in 0.027s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:35:02.247 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/5ba39de3-f5f8-46a2-908d-c43b901e1696/disk --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:35:02.274 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/5ba39de3-f5f8-46a2-908d-c43b901e1696/disk --force-share" returned: 0 in 0.027s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:35:02.275 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running cmd (subprocess): /openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/5ba39de3-f5f8-46a2-908d-c43b901e1696/disk --force-share execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:372 2020-04-16 16:35:02.302 219284 DEBUG oslo_concurrency.processutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CMD "/openstack/venvs/nova-17.1.12/bin/python -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/5ba39de3-f5f8-46a2-908d-c43b901e1696/disk --force-share" returned: 0 in 0.027s execute /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/processutils.py:409 2020-04-16 16:35:02.669 219284 DEBUG nova.compute.resource_tracker [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Hypervisor/Node resource view: name=KO1A3D02O131106CM07.openstack.local free_ram=72406MB free_disk=523GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_3a_0a_7", "product_id": "2047", /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/compute/resource_tracker.py:828 2020-04-16 16:35:02.670 219284 DEBUG oslo_concurrency.lockutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Lock "compute_resources" acquired by "nova.compute.resource_tracker._update_available_resource" :: waited 0.000s inner /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:273 2020-04-16 16:35:02.729 219284 DEBUG nova.compute.resource_tracker [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Compute driver doesn't require allocation refresh and we're on a compute host in a deployment that only has compute hosts with Nova versions >=16 (Pike). Skipping auto-correction of allocations. _update_usage_from_instances /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/compute/resource_tracker.py:1247 2020-04-16 16:35:02.784 219284 DEBUG nova.compute.resource_tracker [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Instance 5ba39de3-f5f8-46a2-908d-c43b901e1696 actively managed on this compute host and has allocations in placement: {u'resources': {u'VCPU': 16, u'MEMORY_MB': 57344, u'DISK_GB': 60}}. _remove_deleted_instances_allocations /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/compute/resource_tracker.py:1269 2020-04-16 16:35:02.785 219284 DEBUG nova.compute.resource_tracker [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Instance dfa80e78-ee02-46e5-ba7a-0874fa37da56 actively managed on this compute host and has allocations in placement: {u'resources': {u'VCPU': 12, u'MEMORY_MB': 24576, u'DISK_GB': 60}}. _remove_deleted_instances_allocations /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/compute/resource_tracker.py:1269 2020-04-16 16:35:02.785 219284 DEBUG nova.compute.resource_tracker [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Instance f33b2602-ac5f-491e-bdb8-7e7f9376bcad actively managed on this compute host and has allocations in placement: {u'resources': {u'VCPU': 16, u'MEMORY_MB': 49152, u'DISK_GB': 40}}. _remove_deleted_instances_allocations /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/compute/resource_tracker.py:1269 2020-04-16 16:35:02.785 219284 DEBUG nova.compute.resource_tracker [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Instance d3d2837b-49c3-4822-b26b-4b3c03d344ae actively managed on this compute host and has allocations in placement: {u'resources': {u'VCPU': 16, u'MEMORY_MB': 49152, u'DISK_GB': 60}}. _remove_deleted_instances_allocations /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/compute/resource_tracker.py:1269 2020-04-16 16:35:02.785 219284 DEBUG nova.compute.resource_tracker [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Instance f117eb96-06a9-4c91-9c5c-111228e24d66 actively managed on this compute host and has allocations in placement: {u'resources': {u'VCPU': 2, u'MEMORY_MB': 4096, u'DISK_GB': 20}}. _remove_deleted_instances_allocations /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/compute/resource_tracker.py:1269 2020-04-16 16:35:02.785 219284 DEBUG nova.compute.resource_tracker [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Total usable vcpus: 72, total allocated vcpus: 62 _report_final_resource_view /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/compute/resource_tracker.py:844 2020-04-16 16:35:02.786 219284 INFO nova.compute.resource_tracker [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Final resource view: name=KO1A3D02O131106CM07.openstack.local phys_ram=385391MB used_ram=192512MB phys_disk=548GB used_disk=250GB total_vcpus=72 used_vcpus=62 pci_stats=[] 2020-04-16 16:35:02.814 219284 DEBUG nova.compute.resource_tracker [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Compute_service record updated for KO1A3D02O131106CM07:KO1A3D02O131106CM07.openstack.local _update_available_resource /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/compute/resource_tracker.py:784 2020-04-16 16:35:02.814 219284 DEBUG oslo_concurrency.lockutils [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Lock "compute_resources" released by "nova.compute.resource_tracker._update_available_resource" :: held 0.144s inner /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:285 2020-04-16 16:35:37.612 219284 DEBUG oslo_service.periodic_task [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running periodic task ComputeManager._reclaim_queued_deletes run_periodic_tasks /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_service/periodic_task.py:215 2020-04-16 16:35:37.613 219284 DEBUG nova.compute.manager [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] CONF.reclaim_instance_interval <= 0, skipping... _reclaim_queued_deletes /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/nova/compute/manager.py:7438 2020-04-16 16:35:38.685 219284 DEBUG oslo_service.periodic_task [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running periodic task ComputeManager._poll_rebooting_instances run_periodic_tasks /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_service/periodic_task.py:215 2020-04-16 16:35:39.685 219284 DEBUG oslo_service.periodic_task [req-dd4a2032-bbbd-4c0b-87ac-11605ffbf6c2 - - - - -] Running periodic task ComputeManager._instance_usage_audit run_periodic_tasks /openstack/venvs/nova-17.1.12/lib/python2.7/site-packages/oslo_service/periodic_task.py:215 -- Best Regards Pradeep Kumar

4 8

[nova] [libvirt] New libvirt subteam
by Sylvain Bauza 23 Apr '20

23 Apr '20

Hi, As proposed and agreed during the last Nova meeting [1], a new libvirt subteam is about to be created. The overall idea of this subteam is to gather contributors from various companies working on the same Nova component (here, libvirt) and help them sharing their efforts. A common pattern is to have all the subteam members to actively engage themselves on reviewing changes from the other subteam members. See the Nova subteam principles [2] for more details. If you feel interested with this idea and you'd like to join this, I just created a new etherpad for tracking contributors and review requests : https://etherpad.opendev.org/p/nova-libvirt-subteam Feel free to put your name on the board and please also provide your timezone. Ideally, I'd love to have some kind of IRC discussion between all subteam members so let's see who's in and then when we can discuss. I'll just sit around in the nova channel if you have questions. Thanks, -Sylvain [1] http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-04-23-16.00.log… [2] https://wiki.openstack.org/wiki/Nova#Nova_subteams

1 0

[kolla] Kolla klub
by Mark Goddard 23 Apr '20

23 Apr '20

Hi, Thanks to everyone who attended today's Kolla Klub meeting. There is a recording [1] for anyone who missed it. I've updated the agenda [2] for the next meeting to include the case studies agreed today. So far I've got Gaël Therond and Дмитрий Грачев (Dmitry Grachev) signed up to present on 7th May, with a few others listed as interested in presenting during a future session. Please add your name if you are interested. No cloud too small! Please also add meeting topic ideas if you have them. [1] https://drive.google.com/open?id=1TLabWeOK65nTrZ278qN4fWLt0Az_zUFT [2] https://docs.google.com/document/d/1EwQs2GXF-EvJZamEx9vQAOSDB5tCjsDCJyHQN5_… Cheers, Mark

1 0

[ansible][tripleo][openstack-ansible-modules] Virtual PTG planning for Openstack Ansilbe modules
by Sagi Shnaidman 23 Apr '20

23 Apr '20

Hi, all As we discussed in the last meeting, I've created an etherpad for Virtual PTG session of Openstack Ansible modules project. Please feel free to add there topics for discussion and add yourself to a list if you plan to attend: https://etherpad.opendev.org/p/openstack-ansible-modules-ptg Thanks -- Best regards Sagi Shnaidman

1 0

[tempest] regex failing
by Eric Barrera 23 Apr '20

23 Apr '20

Hi, I started seeing an error yesterday when doing this command: tempest UPPER_CONSTRAINTS_FILE= https://releases.openstack.org/constraints/upper/master tox -eall -- baremetal --concurrency=1 The specified regex doesn't match with anythingError: InvocationError for command /opt/stack/new/tempest/.tox/tempest/bin/tempest run --regex baremetal --concurrency=1 Any ideas how could I resolve this? Thanks!

4 7

[qa][ptg] Virtual PTG planning
by Masayuki Igawa 23 Apr '20

23 Apr '20

Hi team, Short summary: 1. Please take a look at the QA PTG etherpad[2] and put your name and topics there 2. Please vote on the doodle poll[3] Details: As we discussed in the office hours, we're now collecting topics[3] for the PTG[1]. And, we need to fix our slots for the PTG. We have now 5 topics[2], and we'll have some more, possibly. So, we'll need 2 or 3 hours slots for 2 days or more if we have more topics. Anyway, please fill out the doodle poll[3] if you'd like to participate in the PTG. And, please add your name and topics to the etherpad so that we can discuss them during the following office hours. [1] http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014126.ht… [2] https://etherpad.opendev.org/p/qa-victoria-ptg [3] https://doodle.com/poll/5awstyshtquuwcq3 -- Masayuki

1 0

[goals][Drop Python 2.7 Support][storlets] Request to hold on dropping python2.7 support in Storlets
by Takashi Kajinami 23 Apr '20

23 Apr '20

Hi, Sorry to be late to raise this up, but I'd like to request to postpone completion of "Dropping Python2.7 Support" work in Storlets during this cycle. Storlets is heavily dependent on swift and swift won't drop python 2.7 support during this cycle. This means that we still expect some users who want to run storlets with their swift running on python2 during Ussuri. Currently we've completed all python3 compatibility work, and py2 tests are already dropped. However we would like to postpone marking storlets as python3 only (and dropping all py2-3 compatibility codes) so that we don't prohibit users from using it in swift with python2. See the following patch to remove python3 only mark at the moment. (It was initially added, but we want to revert it) [1] https://review.opendev.org/#/c/720131/2 Thank you, Takashi -- ---------- Takashi Kajinami Senior Software Maintenance Engineer Customer Experience and Engagement Red Hat e-mail: tkajinam(a)redhat.com

2 8

[all][PTL] U Community Goal: Project PTL & Contrib Docs Update #7
by Kendall Nelson 23 Apr '20

23 Apr '20

Hello Everyone! It looks like a lot of good progress[1] has been made since my last update. Thank you for all the hard work people have been putting into their contributor docs! We still have a number of projects missing any progress though. If you have not started yet, PLEASE do so. If there is anything I can do to help, please let me know ASAP. Also, feel free to add me as a reviewer once you have patches pushed. All told, if you are doing the bare minimum of only filling out the template[2] it shouldn't take more than 5-10 minutes per repo. If you have questions about the goal itself, here is the link to that[3]. And as you push patches, please be sure to update the StoryBoard story[4]. Thanks! -Kendall (diablo_rojo) [1] Progress: https://review.opendev.org/#/q/topic:project-ptl-and-contrib-docs+(status:o… [2] Cookiecutter Template: https://opendev.org/openstack/cookiecutter/src/branch/master/%7b%7bcookiecu… [3] Accepted Goal: https://governance.openstack.org/tc/goals/selected/ussuri/project-ptl-and-c… [4] Task Tracking: https://storyboard.openstack.org/#!/story/2007236

2 2

RH-OSP 13 | Queens | Troubleshooting transient 403 in RBAC networks.
by Laurent Dumont 22 Apr '20

22 Apr '20

Hi! I'm currently troubleshooting some transient Keystone/Neutron 403 issues when creating ports in RBAC-ed networks. - As part of our CI process,we are running a few sanity tests in virtual Openstack setups with 3 controllers and 2 computes. - One of those test validates the following scenarios (everything is done through the Openstack python sdk) : - Create projectA / UserA / ProjectB / UserB - Create Network in project A - RBAC Network in projectA to ProjectB - With a token issued to a user in projectB, try to create the following ports in project. - regular port - update regular port with a fixed IP - update regular port with an allowed_address_pair - create port with a fixed IP - create port with an allowed address pair. - The behavior that I'm seeing the most is that the port creation will fail with the following error (the logs are from the sdk with logging/debug enabled. - It fails maybe 1/3 - and I'm unable to see a common pattern. - This is testing against the same Openstack env and the failures/success can happen within a few seconds of each others. Sun Apr 19 01:54:06 2020 -- From ProjectB - create a port with a fixed > address in the network shared from ProjectA > REQ: curl -g -i -X GET > https://OPENSTACK_URL_HERE:13696/v2.0/subnets/77c3fad2-bf78-4b4e-bcfb-8114f… > -H "User-Agent: openstacksdk/0.37.0 keystoneauth1/3.18.0 > python-requests/2.22.0 CPython/3.7.5" -H "X-Auth-Token: > {SHA256}d80fe300742786033376f0591f98d3b4c342103cda1f802087809963694a7935" > RESP: [200] Content-Length: 619 Content-Type: application/json Date: Sun, > 19 Apr 2020 01:54:06 GMT X-Openstack-Request-Id: > req-fcee8345-7daa-499e-a985-92d59a31d0dd > RESP BODY: > {"subnet":{"service_types":[],"description":"","enable_dhcp":true,"tags":[],"network_id":"4ba08203-4328-418b-89e6-b82d474275d5","tenant_id":"472631dbfc5f4dfda9293fb93b5e493f","created_at":"2020-04-19T01:54:03Z","dns_nameservers":[],"updated_at":"2020-04-19T01:54:03Z","gateway_ip":"1.1.1.1","ipv6_ra_mode":null,"allocation_pools":[{"start":"1.1.1.2","end":"1.1.1.254"}],"host_routes":[],"revision_number":0,"ip_version":4,"ipv6_address_mode":null,"cidr":" > 1.1.1.0/24 > ","project_id":"472631dbfc5f4dfda9293fb93b5e493f","id":"77c3fad2-bf78-4b4e-bcfb-8114f95d7884","subnetpool_id":null,"name":"neutronpolicy_shared_network"}} > GET call to network for > https://OPENSTACK_URL_HERE:13696/v2.0/subnets/77c3fad2-bf78-4b4e-bcfb-8114f… > used request id req-fcee8345-7daa-499e-a985-92d59a31d0dd > REQ: curl -g -i -X POST https://OPENSTACK_URL_HERE:13696/v2.0/ports -H > "Content-Type: application/json" -H "User-Agent: openstacksdk/0.37.0 > keystoneauth1/3.18.0 python-requests/2.22.0 CPython/3.7.5" -H > "X-Auth-Token: > {SHA256}d80fe300742786033376f0591f98d3b4c342103cda1f802087809963694a7935" > -d '{"port": {"name": "neutronpolicy_portb-fixed-ip", "network_id": > "4ba08203-4328-418b-89e6-b82d474275d5", "fixed_ips": [{"subnet_id": > "77c3fad2-bf78-4b4e-bcfb-8114f95d7884", "ip_address": "1.1.1.100"}]}}' > RESP: [403] Content-Length: 151 Content-Type: application/json Date: Sun, > 19 Apr 2020 01:54:07 GMT X-Openstack-Request-Id: > req-00277dbc-faca-479f-8736-563595d2729b > RESP BODY: {"NeutronError": {"message": "(rule:create_port and > rule:create_port:fixed_ips) is disallowed by policy", "type": > "PolicyNotAuthorized", "detail": ""}} > POST call to network for https://OPENSTACK_URL_HERE:13696/v2.0/ports used > request id req-00277dbc-faca-479f-8736-563595d2729b When it works, it looks the same, just without the 403 during the port creation. Sun Apr 19 02:29:54 2020 -- From ProjectB - create a port with a fixed > address in the network shared from ProjectA > REQ: curl -g -i -X GET > https://OPENSTACK_URL_HERE:13696/v2.0/subnets/6d2231b6-3ce2-4121-a9a6-72848… > -H "User-Agent: openstacksdk/0.37.0 keystoneauth1/3.18.0 > python-requests/2.22.0 CPython/3.7.5" -H "X-Auth-Token: > {SHA256}d39995434f6a0f106cae7408b07eee93846230f2bb5c922a8e16cfc7c8b26599" > RESP: [200] Content-Length: 619 Content-Type: application/json Date: Sun, > 19 Apr 2020 02:29:54 GMT X-Openstack-Request-Id: > req-bd41b51f-b6e9-454c-8c9a-ee444140459e > RESP BODY: > {"subnet":{"service_types":[],"description":"","enable_dhcp":true,"tags":[],"network_id":"e0d7e7a1-2863-45ec-bd7b-1767a1d02822","tenant_id":"4530b70143ec4cd0ad929a4ca571bdd8","created_at":"2020-04-19T02:29:51Z","dns_nameservers":[],"updated_at":"2020-04-19T02:29:51Z","gateway_ip":"1.1.1.1","ipv6_ra_mode":null,"allocation_pools":[{"start":"1.1.1.2","end":"1.1.1.254"}],"host_routes":[],"revision_number":0,"ip_version":4,"ipv6_address_mode":null,"cidr":" > 1.1.1.0/24 > ","project_id":"4530b70143ec4cd0ad929a4ca571bdd8","id":"6d2231b6-3ce2-4121-a9a6-7284823c48ab","subnetpool_id":null,"name":"neutronpolicy_shared_network"}} > GET call to network for > https:/OPENSTACK_URL_HERE:13696/v2.0/subnets/6d2231b6-3ce2-4121-a9a6-7284823c48ab > used request id req-bd41b51f-b6e9-454c-8c9a-ee444140459e > REQ: curl -g -i -X POST https://OPENSTACK_URL_HERE:13696/v2.0/ports -H > "Content-Type: application/json" -H "User-Agent: openstacksdk/0.37.0 > keystoneauth1/3.18.0 python-requests/2.22.0 CPython/3.7.5" -H > "X-Auth-Token: > {SHA256}d39995434f6a0f106cae7408b07eee93846230f2bb5c922a8e16cfc7c8b26599" > -d '{"port": {"name": "neutronpolicy_portb-fixed-ip", "network_id": > "e0d7e7a1-2863-45ec-bd7b-1767a1d02822", "fixed_ips": [{"subnet_id": > "6d2231b6-3ce2-4121-a9a6-7284823c48ab", "ip_address": "1.1.1.100"}]}}' > RESP: [201] Content-Length: 769 Content-Type: application/json Date: Sun, > 19 Apr 2020 02:29:56 GMT X-Openstack-Request-Id: > req-898dadfb-366d-4a78-8e89-c9d5e9a512f6 > RESP BODY: > {"port":{"allowed_address_pairs":[],"extra_dhcp_opts":[],"updated_at":"2020-04-19T02:29:55Z","device_owner":"","revision_number":6,"port_security_enabled":true,"fixed_ips":[{"subnet_id":"6d2231b6-3ce2-4121-a9a6-7284823c48ab","ip_address":"1.1.1.100"}],"id":"9999784b-c38d-4cf2-b951-e8255d3483f4","security_groups":["4c022e69-b575-435b-bfb7-9cc81f50d49e"],"mac_address":"fa:16:3e:3e:22:fb","device_id":"","status":"DOWN","description":"","tags":[],"name":"neutronpolicy_portb-fixed-ip","project_id":"253ee9b6b5964185ab0818580129c011","qos_policy_id":null,"admin_state_up":true,"network_id":"e0d7e7a1-2863-45ec-bd7b-1767a1d02822","tenant_id":"253ee9b6b5964185ab0818580129c011","created_at":"2020-04-19T02:29:55Z","binding:vnic_type":"normal","ip_allocation":"immediate"}} > POST call to network for https://OPENSTACK_URL_HERE:13696/v2.0/ports used > request id req-898dadfb-366d-4a78-8e89-c9d5e9a512f6 > Sun Apr 19 02:29:56 2020 -- PORT-OK This is what I see in the Neutron server logs (with debug enabled) DEBUG neutron.api.v2.base [req-00277dbc-faca-479f-8736-563595d2729b > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default > default] Request body: {u'port': {u'network_id': > u'4ba08203-4328-418b-89e6-b82d474275d5', u'fixed_ips': [{u'subnet_id': > u'77c3fad2-bf78-4b4e-bcfb-8114f95d7884', u'ip_address': u'1.1.1.100'}], > u'name': u'neutronpolicy_portb-fixed-ip'}} prepare_request_body > /usr/lib/python2.7/site-packages/neutron/api/v2/base.py:690 > DEBUG neutron.db.quota.driver [req-00277dbc-faca-479f-8736-563595d2729b > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default > default] Resources subnetpool,trunk have unlimited quota limit. It is not > required to calculate headroom make_reservation > /usr/lib/python2.7/site-packages/neutron/db/quota/driver.py:223 > DEBUG neutron.quota.resource [req-00277dbc-faca-479f-8736-563595d2729b > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default > default] Usage tracker for resource:port and > tenant:4bae655884b44e8e9c8dbb5983fb3cd8 is out of sync, need to count used > quota count_used > /usr/lib/python2.7/site-packages/neutron/quota/resource.py:272 > DEBUG neutron.quota.resource [req-00277dbc-faca-479f-8736-563595d2729b > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default > default] Quota usage for port was recalculated. Used quota:0. count_used > /usr/lib/python2.7/site-packages/neutron/quota/resource.py:292 > DEBUG neutron.db.quota.driver [req-00277dbc-faca-479f-8736-563595d2729b > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default > default] Attempting to reserve 1 items for resource port. Total usage: 0; > quota limit: 500; headroom:500 make_reservation > /usr/lib/python2.7/site-packages/neutron/db/quota/driver.py:255 > DEBUG neutron.pecan_wsgi.hooks.quota_enforcement > [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989 > 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] Made reservation on > behalf of 4bae655884b44e8e9c8dbb5983fb3cd8 for: {'port': 1} before > /usr/lib/python2.7/site-packages/neutron/pecan_wsgi/hooks/quota_enforcement.py:55 > DEBUG neutron.policy [req-00277dbc-faca-479f-8736-563595d2729b > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default > default] Unable to find requested field: shared in target: > {'binding:host_id': <neutron_lib.constants.Sentinel object at > 0x7f2a3b214d50>, u'name': u'neutronpolicy_portb-fixed-ip', > 'allowed_address_pairs': <neutron_lib.constants.Sentinel object at > 0x7f2a3b214d50>, 'admin_state_up': True, u'network_id': > u'4ba08203-4328-418b-89e6-b82d474275d5', 'tenant_id': > u'4bae655884b44e8e9c8dbb5983fb3cd8', 'extra_dhcp_opts': None, > 'mac_address': <neutron_lib.constants.Sentinel object at 0x7f2a3b214d50>, > 'binding:vnic_type': 'normal', 'device_owner': '', 'qos_policy_id': None, > 'device_id': '', 'binding:profile': <neutron_lib.constants.Sentinel object > at 0x7f2a3b214d50>, 'port_security_enabled': > <neutron_lib.constants.Sentinel object at 0x7f2a3b214d50>, 'project_id': > u'4bae655884b44e8e9c8dbb5983fb3cd8', u'fixed_ips': [{u'subnet_id': > u'77c3fad2-bf78-4b4e-bcfb-8114f95d7884', u'ip_address': u'1.1.1.100'}], > u'network:tenant_id': u'472631dbfc5f4dfda9293fb93b5e493f', > 'security_groups': <neutron_lib.constants.Sentinel object at > 0x7f2a3b214d50>, 'description': ''} __call__ > /usr/lib/python2.7/site-packages/neutron/policy.py:314 > DEBUG neutron.policy [req-00277dbc-faca-479f-8736-563595d2729b > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default > default] Enforcing rules: ['create_port', 'create_port:fixed_ips'] > log_rule_list /usr/lib/python2.7/site-packages/neutron/policy.py:334 > DEBUG neutron.policy [req-00277dbc-faca-479f-8736-563595d2729b > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default > default] Failed policy check for 'create_port' enforce > /usr/lib/python2.7/site-packages/neutron/policy.py:405 > INFO neutron.pecan_wsgi.hooks.translation > [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989 > 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] POST failed (client > error): Access was denied to this resource. > DEBUG neutron.pecan_wsgi.hooks.notifier > [req-00277dbc-faca-479f-8736-563595d2729b c905a7d0011e4bb4ae388cfd699e5989 > 4bae655884b44e8e9c8dbb5983fb3cd8 - default default] No notification will be > sent due to unsuccessful status code: 403 after > /usr/lib/python2.7/site-packages/neutron/pecan_wsgi/hooks/notifier.py:79 > INFO neutron.wsgi [req-00277dbc-faca-479f-8736-563595d2729b > c905a7d0011e4bb4ae388cfd699e5989 4bae655884b44e8e9c8dbb5983fb3cd8 - default > default] 192.168.20.29 "POST /v2.0/ports HTTP/1.1" status: 403 len: 333 > time: 0.3625510 > I haven't found a whole lot online - just a couple of possibly related bugs https://bugs.launchpad.net/neutron/+bug/1808112 https://bugzilla.redhat.com/show_bug.cgi?id=1683385 Does anyone have any good tips/debugging methods? Thanks!

2 3