November 2019 - openstack-discuss

RE: [nova][neutron][cyborg] Bandwidth (and accel) providers are broken if CONF.host is set
by Balázs Gibizer 22 Nov '19

22 Nov '19

On Fri, Nov 22, 2019 at 02:11, "Nadathur, Sundar" <sundar.nadathur(a)intel.com> wrote: >> -----Original Message----- >> From: Eric Fried <openstack(a)fried.cc> >> Sent: Thursday, November 21, 2019 3:28 PM > >> Action Summary >> ============== >> If the above sounds reasonable, it would entail the following >> actions: >> - Neutron(/Cyborg?): backportable patch to >> s/CONF.host/socket.gethostname()/ >> - Nova: GET /os-hypervisors*?service_host=X in a new microversion. >> - Neutron/Cyborg: master-only patch to do the logic described in >> `Upgrade >> Concerns`_ (though for now without the `elif` branch). > > Cyborg does use CONF.host today. We can use socket.gethostname() > instead. > > On a related node, the patch for ARQ binding [1] uses instance.host, > which comes from CONF.host. I'll change it to instance.node, which > comes from [2], which in turn comes from get_hostname() [3]. I'm not sure that for binding you need to use the instance.node. See my neutron port binding discussion in my response to Eric's original mail. > > [1] > https://review.opendev.org/#/c/631244/46/nova/compute/manager.py@2634 > [2] > https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L9137 > [3] > https://github.com/openstack/nova/blob/1cd5563f2dd2b218db2422397c8aab394d48… > >> Thanks, >> efried > > > Thanks & Regards, > Sundar >

1 0

[nova][ops] heads up: change in behavior for resize and migrate server action APIs
by melanie witt 21 Nov '19

21 Nov '19

Hey all, Just wanted to give everyone a heads up about a recent change [1] in behavior for the resize [2] and migrate [3] server action APIs: https://docs.openstack.org/releasenotes/nova/unreleased.html#other-notes The scheduling step for resize and migrate has been changed from synchronous to asynchronous, so users will no longer receive a 400 error for NoValidHost. To discover whether the resize or migrate has failed during scheduling, users must use the server actions API [4], which is called "server events" in the openstackclient [5]. There is a WIP proposal [6] to add exception messages to events in the server actions API: https://review.opendev.org/694428 in order to communicate additional failure info to non-admin users. I think we'll want this as the server actions API becomes more of a source of information for the end user. Feel free reply here or visit the review to add your thoughts. Cheers, -melanie [1] https://review.opendev.org/693937 [2] https://docs.openstack.org/api-ref/compute/#resize-server-resize-action [3] https://docs.openstack.org/api-ref/compute/#migrate-server-migrate-action [4] https://docs.openstack.org/api-ref/compute/#servers-actions-servers-os-inst… [5] https://docs.openstack.org/python-openstackclient/latest/cli/command-object… [6] background: http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010775…

1 0

Re: All VMs fail when --max exceeds available resources
by Albert Braden 21 Nov '19

21 Nov '19

The other symptom is that the scheduler will send single VMs to a full hypervisor and overload it even though we have cpu_allocation_ratio and ram_allocation_ratio set to 1: root@us01odc-dev1-ctrl1:~# os hypervisor list --long +----+------------------------------------------+-----------------+---------------+-------+------------+-------+----------------+-----------+ | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State | vCPUs Used | vCPUs | Memory MB Used | Memory MB | +----+------------------------------------------+-----------------+---------------+-------+------------+-------+----------------+-----------+ | 1 | us01odc-dev1-hv003.internal.synopsys.com | QEMU | 10.195.116.16 | up | 42 | 16 | 161792 | 128888 | | 3 | us01odc-dev1-hv002.internal.synopsys.com | QEMU | 10.195.116.15 | up | 43 | 16 | 165888 | 128888 | | 4 | us01odc-dev1-hv001.internal.synopsys.com | QEMU | 10.195.116.14 | up | 38 | 16 | 161792 | 128888 | +----+------------------------------------------+-----------------+---------------+-------+------------+-------+----------------+-----------+ In the logs I see the scheduler returning 1 host: /var/log/nova/nova-scheduler.log:2019-11-19 16:38:20.930 895454 DEBUG nova.filters [req-0703f1f8-a52a-4fb6-a402-226dd25e9988 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Filter NUMATopologyFilter returned 1 host(s) get_filtered_objects /usr/lib/python2.7/dist-packages/nova/filters.py:104 It weighs the host and reports negative RAM: /var/log/nova/nova-scheduler.log:2019-11-19 16:38:20.930 895454 DEBUG nova.scheduler.filter_scheduler [req-0703f1f8-a52a-4fb6-a402-226dd25e9988 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Weighed [WeighedHost [host: (us01odc-dev1-hv002, us01odc-dev1-hv002.internal.synopsys.com) ram: -6280MB disk: 683008MB io_ops: 1 instances: 5, weight: 0.0]] _get_sorted_hosts /usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py:454 Then it selects that host: /var/log/nova/nova-scheduler.log:2019-11-19 16:38:20.931 895454 DEBUG nova.scheduler.utils [req-0703f1f8-a52a-4fb6-a402-226dd25e9988 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Attempting to claim resources in the placement API for instance 834dd112-26f8-424c-a92b-23423baa185a claim_resources /usr/lib/python2.7/dist-packages/nova/scheduler/utils.py:935 /var/log/nova/nova-scheduler.log:2019-11-19 16:38:21.567 895454 DEBUG nova.scheduler.filter_scheduler [req-0703f1f8-a52a-4fb6-a402-226dd25e9988 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] [instance: 834dd112-26f8-424c-a92b-23423baa185a] Selected host: (us01odc-dev1-hv002, us01odc-dev1-hv002.internal.synopsys.com) ram: -6280MB disk: 683008MB io_ops: 1 instances: 5 _consume_selected_host /usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py:346 The VM builds successfully and goes to ACTIVE. What should I be looking for here? Obviously I broke the scheduler, but my nova config is the same as the working cluster.

4 11

[nova][gate] Thoughts on working around bug 1853453?
by Matt Riedemann 21 Nov '19

21 Nov '19

I've been noticing these shelve/unshelve guest ssh fail due to dhcp lease issues quite a bit recently and wrote a bug and e-r query for it this morning: http://status.openstack.org/elastic-recheck/#1853453 The problem seems to stem from when these shelve tests run on multinode jobs and we shelve on one host and unshelve on another. I have a patch up to nova to force config drive in the nova-next job where this hits the most: https://review.opendev.org/#/c/695431 But that's just kind of a stab in the dark to take the metadata API out of the picture for cloud-init. If that doesn't help, and we don't know what is causing this or have ideas to debug it, we might need to consider making a change to shelve/unshelve testing in tempest such that we try to unshelve on original host. Now I realize that is unfortunate since the whole point of shelve offloading and unshelving is that you can land on another host and things are good, but if these tests continue to be a high failure rate in multinode jobs we probably need to consider workarounds if no one is going to dig into the failures and figure out what is going wrong. Thoughts? -- Thanks, Matt

2 1

[monasca] New team meeting time poll
by Witek Bedyk 21 Nov '19

21 Nov '19

Hello everyone, We would like to find the new time slot for the Monasca Team Meeting which suites you best. Please fill in the times which work for you in that poll [1] until next Wednesday. Thanks Witek [1] https://doodle.com/poll/ey6brvmbsubkxpp9

1 1

Re: [nova][ptg] Allow compute nodes to use DISK_GB from shared storage RP by using aggregate relationship
by Balázs Gibizer 21 Nov '19

21 Nov '19

On Thu, Nov 14, 2019 at 02:58, "Patil, Tushar" <Tushar.Patil(a)nttdata.com> wrote: > On 11/13/2019 8:34 AM, Sylvain Bauza wrote: >>> Me too. To be clear, I don't think operators would modify the >>> above but >>> if so, they would need reshapes. > >> Maybe not, but this is the kind of detail that should be in the >> spec and >> functional tests to make sure it's solid since this is a big >> architectural change in nova. > > It depends on how the aggregates are created on the nova and > placement side. > > A) From placement point of view, operator can create a new aggregate > and add shared storage RP to it (tag MISC_SHARES_VIA_AGGREGATE trait > to this RP). The newly created valid UUID would then be set in the > config option ``sharing_disk_aggregate`` on the compute node side. > This aggregate UUID wouldn't be present in the nova aggregate. so > it's not possible to add host to the nova aggregate unless a new > aggregate is created on nova side. > > B) If nova aggregates are synced to the placement service and say > below is the picture: > > Nova: > > Agg1 - metadata (pinned=True) > - host1 > - host2 > > Now, operator adds a new shared storage RP to Agg1 on placement side > and then set Agg1 UUID in ``sharing_disk_aggregate`` on compute nodes > along with ``using_shared_disk_provider`=True``, then it would add > compute node RP to the Agg1 on the placement without any issues but > when you want to reverse the configuration, > using_shared_disk_provider=False, then it not that straight to remove > the host from the placement/nova aggregate because there would be > other traits set to compute RPs which could cause those functions > stop working. For me from the sharing disk provider feature perspective the placement aggregate that is needed for the sharing to work, and any kind of nova host aggregate (either synced to placement or not) is independent. The placement aggregate is a must for the feature. On top of that if the operator wants to create a nova host aggregate as well and sync it to placement then at the end there will be two, independent placement aggregates. One to express the sharing relationship and one to express a host aggregate from nova. These two aggregate will not be the same as the first one will have the sharing provider in it while the second one doesn't. gibi > > We had same kind of discussion [1] when implementing forbidden > aggregates where we want to sync traits set to the aggregates but > later it was concluded that operator will do it manually. > > I will include the details Matt has pointed out in this email in my > next patchset. > > [1] : > http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006950.html > > Regards, > tpatil > > > > ________________________________________ > From: Matt Riedemann <mriedemos(a)gmail.com> > Sent: Wednesday, November 13, 2019 11:41 PM > To: openstack-discuss(a)lists.openstack.org > Subject: Re: [nova][ptg] Allow compute nodes to use DISK_GB from > shared storage RP by using aggregate relationship > > On 11/13/2019 8:34 AM, Sylvain Bauza wrote: >> Me too. To be clear, I don't think operators would modify the above >> but >> if so, they would need reshapes. > > Maybe not, but this is the kind of detail that should be in the spec > and > functional tests to make sure it's solid since this is a big > architectural change in nova. > > -- > > Thanks, > > Matt > > Disclaimer: This email and any attachments are sent in strictest > confidence for the sole use of the addressee and may contain legally > privileged, confidential, and proprietary data. If you are not the > intended recipient, please advise the sender by replying promptly to > this email and then delete and destroy this email and any attachments > without any further use, copying or forwarding. >

3 2

[cinder] meeting time change poll results
by Brian Rosmaita 20 Nov '19

20 Nov '19

Here are the results of the meeting time change poll: 16 responses - current time (1600 UTC): 19% unlikely to attend, 25% love it - one hour earlier (1500 UTC): 13% unlikely to attend, 19% love it - two hours earlier (1400 UTC): 0% unlikely to attend, 56% love it Thus, we will reschedule the Cinder weekly meeting to occur each Wednesday at 1400 UTC beginning with the meeting on 4 December 2019. Our current meeting channel is occupied at that time. I've submitted a patch to hold the meeting in #openstack-meeting-4: https://review.opendev.org/695339 I'll send out another email when that patch has been approved and the new meeting location has been verified. cheers, brian

1 0

[all][tc] Planning for dropping the Python2 support in OpenStack
by Ghanshyam Mann 20 Nov '19

20 Nov '19

Hello Everyone, Python 2.7 is going to retire in Jan 2020 [1] and we planned to drop the python 2 support from OpenStack during the start of the Ussuri cycle[2]. Time has come now to start the planning on dropping the Python2. It needs to be coordinated among various Projects, libraries, vendors driver, third party CI and testing frameworks. * Preparation for the Plan & Schedule: Etherpad: https://etherpad.openstack.org/p/drop-python2-support We discussed it in TC to come up with the plan, execute it smoothly and avoid breaking any dependent projects. I have prepared an etherpad[3](mentioned above also) to capture all the points related to this topic and most importantly the draft schedule about who can drop the support and when. The schedule is in the draft state and not final yet. The most important points are if you are dropping the support then all your consumers (OpenStack Projects, Vendors drivers etc) are ready for that. For example, oslo, os-bricks, client lib, testing framework projects will keep the python2 support until we make sure all the consumers of those projects do not require py2 support. If anyone require then how long they can support py2. These libraries, testing frameworks will be the last one to drop py2. We have planned to have a dedicated discussion in TC office hours on the 24th Thursday #openstack-tc channel. We will discuss what all need to be done and the schedules. You do not have to drop it immediately and keep eyes on this ML thread till we get the consensus on the community-level plan and schedule. Meanwhile, you can always start pre-planning for your projects, for example, stephenfin has started for Nova[4] to migrate the third party CI etc. Cinder has coordinated with all vendor drivers & their CI to migrate from py2 to py3. * Projects want to keep the py2 support? There is no mandate that projects have to drop the py2 support right now. If you want to keep the support then key things to discuss are what all you need and does all your dependent projects/libs provide the support of py2. This is something needs to be discussed case by case. If any project wants to keep the support, add that in the etherpad with a brief reason which will be helpful to discuss the need and feasibility. Feel free to provide feedback or add the missing point on the etherpad. Do not forget to attend the 24th Oct 2019, TC office hour on Thursday at 1500 UTC in #openstack-tc. [1] https://pythonclock.org/ [2] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecatio… [3] https://etherpad.openstack.org/p/drop-python2-support [4] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010109.… -gmann

7 18

[cinder] Ussuri Virtual PTG next week
by Brian Rosmaita 20 Nov '19

20 Nov '19

As announced at today's Cinder weekly meeting, the Virtual PTG will be held next week on two days: - Monday 25 November 1500-1700 UTC - Wednesday 27 November 1500-1700 UTC The planning etherpad is: https://etherpad.openstack.org/p/cinder-ussuri-virtual-ptg-planning Feel free to add topics. They can be new topics or something you want to follow up on from the Shanghai PTG. Format will be the usual, that is, we'll start at the beginning, give each topic what it takes, and finish when we are done. We'll use BlueJeans video conferencing: https://bluejeans.com/3228528973 If you haven't used BlueJeans before, ping me in #openstack-cinder and I can do a quick meeting with you to make sure you can connect OK. cheers, brian

1 0

All VMs fail when --max exceeds available resources
by Albert Braden 20 Nov '19

20 Nov '19

I was experimenting in our dev cluster with CPU pinning and filters. After I was done I ran Ansible to put everything back like it was, but the scheduler is broken in 2 ways and I can't find the problem in my config. The first symptom is that if I use -max to create more VMs than the hypervisors can support, all of them go to ERROR. Before I changed things, --max would fill the hypervisors and only the extra VMs would go to ERROR. I'll email separately about the other one; this is already getting long. When I look at the logs, I see the "Starting to schedule" line with the list of instance UUIDs, and then "Attempting to claim resources" log entries. Hosts are selected; they are starting with 0 instances: us01odc-dev1-ctrl3:/var/log/nova/nova-scheduler.log:2019-11-19 11:39:56.880 1409571 DEBUG nova.scheduler.filter_scheduler [req-892201d0-6b04-4e92-a347-00285725fbed 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] [instance: ef1f7493-f792-4c3b-bf50-8e68b3d553ac] Selected host: (us01odc-dev1-hv003, us01odc-dev1-hv003.internal.synopsys.com) ram: 120696MB disk: 986112MB io_ops: 0 instances: 0 _consume_selected_host /usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py:346 Scheduler starting with the correct number of HV: us01odc-dev1-ctrl3:/var/log/nova/nova-scheduler.log:2019-11-19 11:39:56.888 1409571 DEBUG nova.filters [req-892201d0-6b04-4e92-a347-00285725fbed 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Starting with 3 host(s) get_filtered_objects /usr/lib/python2.7/dist-packages/nova/filters.py:70 I see the HV being weighed and the number of instances on each HV increasing from 0: us01odc-dev1-ctrl3:/var/log/nova/nova-scheduler.log:2019-11-19 11:39:57.106 1409571 DEBUG nova.scheduler.filter_scheduler [req-892201d0-6b04-4e92-a347-00285725fbed 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Filtered [(us01odc-dev1-hv002, us01odc-dev1-hv002.internal.synopsys.com) ram: 85880MB disk: 880640MB io_ops: 1 instances: 2, (us01odc-dev1-hv001, us01odc-dev1-hv001.internal.synopsys.com) ram: 89976MB disk: 932864MB io_ops: 0 instances: 1, (us01odc-dev1-hv003, us01odc-dev1-hv003.internal.synopsys.com) ram: 89976MB disk: 929792MB io_ops: 1 instances: 1] _get_sorted_hosts /usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py:435 us01odc-dev1-ctrl3:/var/log/nova/nova-scheduler.log:2019-11-19 11:39:57.107 1409571 DEBUG nova.scheduler.filter_scheduler [req-892201d0-6b04-4e92-a347-00285725fbed 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Weighed [WeighedHost [host: (us01odc-dev1-hv001, us01odc-dev1-hv001.internal.synopsys.com) ram: 89976MB disk: 932864MB io_ops: 0 instances: 1, weight: 2.40576402895], WeighedHost [host: (us01odc-dev1-hv003, us01odc-dev1-hv003.internal.synopsys.com) ram: 89976MB disk: 929792MB io_ops: 1 instances: 1, weight: 1.11693447844], WeighedHost [host: (us01odc-dev1-hv002, us01odc-dev1-hv002.internal.synopsys.com) ram: 85880MB disk: 880640MB io_ops: 1 instances: 2, weight: 0.961725168625]] _get_sorted_hosts /usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py:454 The number of instances goes beyond the HV capacity: us01odc-dev1-ctrl3:/var/log/nova/nova-scheduler.log:2019-11-19 11:39:57.602 1409571 DEBUG nova.scheduler.filter_scheduler [req-892201d0-6b04-4e92-a347-00285725fbed 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Weighed [WeighedHost [host: (us01odc-dev1-hv001, us01odc-dev1-hv001.internal.synopsys.com) ram: 59256MB disk: 876544MB io_ops: 1 instances: 2, weight: 1.17395901159], WeighedHost [host: (us01odc-dev1-hv003, us01odc-dev1-hv003.internal.synopsys.com) ram: 59256MB disk: 873472MB io_ops: 2 instances: 2, weight: 0.435549629144], WeighedHost [host: (us01odc-dev1-hv002, us01odc-dev1-hv002.internal.synopsys.com) ram: 55160MB disk: 824320MB io_ops: 2 instances: 3, weight: 0.292945361349]] _get_sorted_hosts /usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py:454 Hosts are still being selected, but they are way over capacity already: us01odc-dev1-ctrl3:/var/log/nova/nova-scheduler.log:2019-11-19 11:39:58.696 1409571 DEBUG nova.scheduler.filter_scheduler [req-892201d0-6b04-4e92-a347-00285725fbed 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] [instance: 579dd4e2-d5d5-445f-905f-84cfd93146f6] Selected host: (us01odc-dev1-hv002, us01odc-dev1-hv002.internal.synopsys.com) ram: -6280MB disk: 711680MB io_ops: 4 instances: 5 _consume_selected_host /usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py:346 Then we start to see the warnings: us01odc-dev1-ctrl3:/var/log/nova/nova-scheduler.log:2019-11-19 11:39:58.817 1409571 WARNING nova.scheduler.client.report [req-892201d0-6b04-4e92-a347-00285725fbed 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Unable to submit allocation for instance 5de08185-1ef2-4c92-8a19-5f09ec27be71 (409 {"errors": [{"status": 409, "request_id": "req-6cdd0b7a-bdbd-486f-9792-20840ea4a72e", "code": "placement.undefined_code", "detail": "There was a conflict when trying to complete your request.\n\n Unable to allocate inventory: Unable to create allocation for 'MEMORY_MB' on resource provider 'f20fa03d-18f4-486b-9b40-ceaaf52dabf8'. The requested amount would exceed the capacity. ", "title": "Conflict"}]}) And then they all fail and are deleted: us01odc-dev1-ctrl3:/var/log/nova/nova-scheduler.log:2019-11-19 11:39:59.057 1409571 DEBUG nova.scheduler.filter_scheduler [req-892201d0-6b04-4e92-a347-00285725fbed 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Unable to successfully claim against any host. _schedule /usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py:242 us01odc-dev1-ctrl3:/var/log/nova/nova-scheduler.log:2019-11-19 11:39:59.058 1409571 DEBUG nova.scheduler.filter_scheduler [req-892201d0-6b04-4e92-a347-00285725fbed 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Cleaning up allocations for [u'ef1f7493-f792-4c3b-bf50-8e68b3d553ac', u'80afcfd7-ce25-4fc4-8b0d-b581a31b87bd', u'22cf9509-39e8-456a-b32c-e950cc597266', u'd6fc0b66-cd44-410d-96c9-590c22f1e21b', u'713a12fb-8bbf-4eac-ae02-68cb007fa34e', u'30c70c4d-1447-4481-bdc5-816835180ac6', u'be955f43-ac56-4531-ada8-16ce966211c7', u'b91820f6-775e-4abb-b28e-5b9b065819c2', u'83472954-2197-4e65-b7e7-1fe892f28458', u'd862bd74-494b-4c29-94ad-b80fc6c113e8', u'80a69dba-8011-403f-87fb-c39ef17ba467', u'c440d8d1-4da2-4c27-af9f-dd5afa19d083', u'5b301d69-a4f1-4393-8d67-4286c9873490', u'579dd4e2-d5d5-445f-905f-84cfd93146f6'] _cleanup_allocations /usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py:299 us01odc-dev1-ctrl3:/var/log/nova/nova-scheduler.log:2019-11-19 11:39:59.124 1409571 INFO nova.scheduler.client.report [req-892201d0-6b04-4e92-a347-00285725fbed 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Deleted allocation for instance ef1f7493-f792-4c3b-bf50-8e68b3d553ac us01odc-dev1-ctrl3:/var/log/nova/nova-scheduler.log:2019-11-19 11:39:59.190 1409571 INFO nova.scheduler.client.report [req-892201d0-6b04-4e92-a347-00285725fbed 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Deleted allocation for instance 80afcfd7-ce25-4fc4-8b0d-b581a31b87bd And the errors: us01odc-dev1-ctrl1:/var/log/nova/nova-conductor.log:2019-11-19 11:40:00.201 1801903 WARNING nova.scheduler.utils [req-892201d0-6b04-4e92-a347-00285725fbed 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. us01odc-dev1-ctrl1:/var/log/nova/nova-conductor.log:2019-11-19 11:40:00.205 1801903 WARNING nova.scheduler.utils [req-892201d0-6b04-4e92-a347-00285725fbed 2cb6757679d54a69803a5b6e317b3a93 474ae347d8ad426f8118e55eee47dcfd - default 7d3a4deab35b434bba403100a6729c81] [instance: 5b301d69-a4f1-4393-8d67-4286c9873490] Setting instance to ERROR state.: NoValidHost_Remote: No valid host was found. There are not enough hosts available.

1 0