[Openstack-operators] Problems with AggregateMultiTenancyIsolation while migrating an instance

Jay Pipes jaypipes at gmail.com
Tue May 29 16:10:00 UTC 2018


The hosts you are attempting to migrate *to* do not have the 
filter_tenant_id property set to the same tenant ID as the compute host 
2 that originally hosted the instance.

That is why you see this in the scheduler logs when evaluating the 
fitness of compute host 1 and compute host 3:

"fails tenant id"

Best,
-jay

On 05/29/2018 05:41 AM, Massimo Sgaravatto wrote:
> I have a small testbed OpenStack cloud (running Ocata) where I am trying 
> to debug a problem with Nova scheduling.
> 
> 
> In short: I see different behaviors when I create a new VM and when I 
> try to migrate a VM
> 
> 
> Since I want to partition the Cloud so that each project uses only 
> certain compute nodes, I created one host aggregate per project (see 
> also this thread:
> http://lists.openstack.org/pipermail/openstack-operators/2018-February/014831.html)
> 
> 
> The host-aggregate for my project is:
> 
> # nova  aggregate-show 52
> +----+-----------+-------------------+--------------------------------------------------------------+----------------------------------------------------------------------------------------------+--------------------------------------+
> | Id | Name      | Availability Zone | Hosts                            
>                              | Metadata                                  
>                                                     | UUID              
>                     |
> +----+-----------+-------------------+--------------------------------------------------------------+----------------------------------------------------------------------------------------------+--------------------------------------+
> | 52 | SgaraPrj1 | nova              | 'compute-01.cloud.pd.infn.it 
> <http://compute-01.cloud.pd.infn.it>', 'compute-02.cloud.pd.infn.it 
> <http://compute-02.cloud.pd.infn.it>' | 'availability_zone=nova', 
> 'filter_tenant_id=ee1865a76440481cbcff08544c7d580a', 'size=normal' | 
> 675f6291-6997-470d-87e1-e9ea199a379f |
> +----+-----------+-------------------+--------------------------------------------------------------+----------------------------------------------------------------------------------------------+--------------------------------------+
> 
> The same compute nodes are shared by other projects  (for which specific 
> host-aggregates, as this one, have been created)
> The other compute node (I have only 3 compute nodes in this small 
> testbed) is targeted to other projects (for which specific 
> host-aggregates exist)
> 
> 
> This is what I have in nova.conf wrt scheduling filters:
> 
> enabled_filters = 
> AggregateInstanceExtraSpecsFilter,AggregateMultiTenancyIsolation,RetryFilter,AvailabilityZoneFilter,RamFilter,CoreFilter,AggregateRamFilter,AggregateCo
> reFilter,DiskFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter
> 
> 
> 
> If I try to create a VM, I see from the scheduler log [*] that 
> the AggregateMultiTenancyIsolation selects only 2 compute nodes, as 
> expected.
> 
> 
> But if I then try to migrate the very same VM, it reports that no valid 
> host was found:
> 
> # nova migrate afaf2a2d-7ff8-4e52-a89a-031ee079a9ba
> ERROR (BadRequest): No valid host was found. No valid host found for 
> cold migrate (HTTP 400) (Request-ID: 
> req-45b8afd5-9683-40a6-8416-295563e37e34)
> 
> 
> And according to the scheduler log the problem is with the 
> AggregateMultiTenancyIsolation which returned 0 hosts (while I would 
> have expected one):
> 
> 2018-05-29 11:12:56.375 19428 INFO nova.scheduler.host_manager 
> [req-45b8afd5-9683-40a6-8416-295563e37e34 
> 9bd03f63fa9d4beb8de31e6c2f2c8d12 56c3f5c047e74a78a714\
> 38c4412e6e13 - - -\
> ] Host filter ignoring hosts: compute-02.cloud.pd.infn.it 
> <http://compute-02.cloud.pd.infn.it>
> 2018-05-29 11:12:56.375 19428 DEBUG nova.filters 
> [req-45b8afd5-9683-40a6-8416-295563e37e34 
> 9bd03f63fa9d4beb8de31e6c2f2c8d12 56c3f5c047e74a78a71438c4412e6e13 -\
>   - -] Starting wit\
> h 2 host(s) get_filtered_objects 
> /usr/lib/python2.7/site-packages/nova/filters.py:70
> 2018-05-29 11:12:56.376 19428 DEBUG nova.filters 
> [req-45b8afd5-9683-40a6-8416-295563e37e34 
> 9bd03f63fa9d4beb8de31e6c2f2c8d12 56c3f5c047e74a78a71438c4412e6e13 -\
>   - -] Filter Aggre\
> gateInstanceExtraSpecsFilter returned 2 host(s) get_filtered_objects 
> /usr/lib/python2.7/site-packages/nova/filters.py:104
> 2018-05-29 11:12:56.377 19428 DEBUG 
> nova.scheduler.filters.aggregate_multitenancy_isolation 
> [req-45b8afd5-9683-40a6-8416-295563e37e34 9bd03f63fa9d4beb8de31e6c\
> 2f2c8d12 56c3f5c04\
> 7e74a78a71438c4412e6e13 - - -] (compute-01.cloud.pd.infn.it 
> <http://compute-01.cloud.pd.infn.it>, compute-01.cloud.pd.infn.it 
> <http://compute-01.cloud.pd.infn.it>) ram: 12797MB disk: 48128MB io_ops: 
> 0 instances: 0 fails tenant id on\
>   aggregate host_pa\
> sses 
> /usr/lib/python2.7/site-packages/nova/scheduler/filters/aggregate_multitenancy_isolation.py:50
> 2018-05-29 11:12:56.378 19428 DEBUG 
> nova.scheduler.filters.aggregate_multitenancy_isolation 
> [req-45b8afd5-9683-40a6-8416-295563e37e34 9bd03f63fa9d4beb8de31e6c\
> 2f2c8d12 56c3f5c04\
> 7e74a78a71438c4412e6e13 - - -] (compute-03.cloud.pd.infn.it 
> <http://compute-03.cloud.pd.infn.it>, compute-03.cloud.pd.infn.it 
> <http://compute-03.cloud.pd.infn.it>) ram: 8701MB disk: -4096MB io_ops: 
> 0 instances: 0 fails tenant id on \
> aggregate host_pas\
> ses 
> /usr/lib/python2.7/site-packages/nova/scheduler/filters/aggregate_multitenancy_isolation.py:50
> 2018-05-29 11:12:56.378 19428 INFO nova.filters 
> [req-45b8afd5-9683-40a6-8416-295563e37e34 
> 9bd03f63fa9d4beb8de31e6c2f2c8d12 56c3f5c047e74a78a71438c4412e6e13 - \
> - -] Filter Aggreg\
> ateMultiTenancyIsolation returned 0 hosts
> 
> 
> 
> I am confused ...
> Any hints ?
> 
> Thanks, Massimo
> 
> [*]
> 
> 
> 2018-05-29 11:09:54.328 19428 DEBUG nova.filters 
> [req-1a838e77-8042-4550-b157-4943445119a2 
> ab573ba3ea014b778193b6922ffffe6d ee1865a76440481cbcff08544c7d580a -\
>   - -] Filter AggregateInstanceExtraSpecsFilter returned 3 host(s) 
> get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
> 2018-05-29 11:09:54.330 19428 DEBUG nova.filters 
> [req-1a838e77-8042-4550-b157-4943445119a2 
> ab573ba3ea014b778193b6922ffffe6d ee1865a76440481cbcff08544c7d580a -\
>   - -] Filter AggregateMultiTenancyIsolation returned 2 host(s) 
> get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
> 2018-05-29 11:09:54.332 19428 DEBUG nova.filters 
> [req-1a838e77-8042-4550-b157-4943445119a2 
> ab573ba3ea014b778193b6922ffffe6d ee1865a76440481cbcff08544c7d580a -\
>   - -] Filter RetryFilter returned 2 host(s) get_filtered_objects 
> /usr/lib/python2.7/site-packages/nova/filters.py:104
> 2018-05-29 11:09:54.332 19428 DEBUG nova.filters 
> [req-1a838e77-8042-4550-b157-4943445119a2 
> ab573ba3ea014b778193b6922ffffe6d ee1865a76440481cbcff08544c7d580a -\
>   - -] Filter AvailabilityZoneFilter returned 2 host(s) 
> get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
> 2018-05-29 11:09:54.333 19428 DEBUG nova.filters 
> [req-1a838e77-8042-4550-b157-4943445119a2 
> ab573ba3ea014b778193b6922ffffe6d ee1865a76440481cbcff08544c7d580a -\
>   - -] Filter RamFilter returned 2 host(s) get_filtered_objects 
> /usr/lib/python2.7/site-packages/nova/filters.py:104
> 2018-05-29 11:09:54.334 19428 DEBUG nova.filters 
> [req-1a838e77-8042-4550-b157-4943445119a2 
> ab573ba3ea014b778193b6922ffffe6d ee1865a76440481cbcff08544c7d580a -\
>   - -] Filter CoreFilter returned 2 host(s) get_filtered_objects 
> /usr/lib/python2.7/site-packages/nova/filters.py:104
> 2018-05-29 11:09:54.334 19428 DEBUG nova.filters 
> [req-1a838e77-8042-4550-b157-4943445119a2 
> ab573ba3ea014b778193b6922ffffe6d ee1865a76440481cbcff08544c7d580a -\
>   - -] Filter AggregateRamFilter returned 2 host(s) get_filtered_objects 
> /usr/lib/python2.7/site-packages/nova/filters.py:104
> 2018-05-29 11:09:54.335 19428 DEBUG nova.filters 
> [req-1a838e77-8042-4550-b157-4943445119a2 
> ab573ba3ea014b778193b6922ffffe6d ee1865a76440481cbcff08544c7d580a -\
>   - -] Filter AggregateCoreFilter returned 2 host(s) 
> get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
> 2018-05-29 11:09:54.335 19428 DEBUG nova.filters 
> [req-1a838e77-8042-4550-b157-4943445119a2 
> ab573ba3ea014b778193b6922ffffe6d ee1865a76440481cbcff08544c7d580a -\
>   - -] Filter DiskFilter returned 2 host(s) get_filtered_objects 
> /usr/lib/python2.7/site-packages/nova/filters.py:104
> 2018-05-29 11:09:54.336 19428 DEBUG nova.filters 
> [req-1a838e77-8042-4550-b157-4943445119a2 
> ab573ba3ea014b778193b6922ffffe6d ee1865a76440481cbcff08544c7d580a -\
>   - -] Filter ComputeFilter returned 2 host(s) get_filtered_objects 
> /usr/lib/python2.7/site-packages/nova/filters.py:104
> 2018-05-29 11:09:54.337 19428 DEBUG nova.filters 
> [req-1a838e77-8042-4550-b157-4943445119a2 
> ab573ba3ea014b778193b6922ffffe6d ee1865a76440481cbcff08544c7d580a -\
>   - -] Filter ComputeCapabilitiesFilter returned 2 host(s) 
> get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
> 2018-05-29 11:09:54.338 19428 DEBUG nova.filters 
> [req-1a838e77-8042-4550-b157-4943445119a2 
> ab573ba3ea014b778193b6922ffffe6d ee1865a76440481cbcff08544c7d580a -\
>   - -] Filter ImagePropertiesFilter returned 2 host(s) 
> get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
> 2018-05-29 11:09:54.339 19428 DEBUG nova.filters 
> [req-1a838e77-8042-4550-b157-4943445119a2 
> ab573ba3ea014b778193b6922ffffe6d ee1865a76440481cbcff08544c7d580a -\
>   - -] Filter ServerGroupAntiAffinityFilter returned 2 host(s) 
> get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
> 2018-05-29 11:09:54.339 19428 DEBUG nova.filters 
> [req-1a838e77-8042-4550-b157-4943445119a2 
> ab573ba3ea014b778193b6922ffffe6d ee1865a76440481cbcff08544c7d580a -\
>   - -] Filter ServerGroupAffinityFilter returned 2 host(s) 
> get_filtered_objects /usr/lib/python2.7/site-packages/nova/filters.py:104
> 
> 
> 
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> 



More information about the OpenStack-operators mailing list