[ops][nova] Problems with AggregateMultiTenancyIsolation/AggregateInstanceExtraSpecsFilter after Train --> Yoga update
Mohammed Naser
mnaser at vexxhost.com
Mon May 30 07:50:54 UTC 2022
Hi there,
Are you sure you haven't done any other changes to your environment?
Both those filters haven't been changed for years (~2017):
https://github.com/openstack/nova/commits/master/nova/scheduler/filters/aggregate_multitenancy_isolation.py
https://github.com/openstack/nova/commits/master/nova/scheduler/filters/aggregate_instance_extra_specs.py
I think you've got something else here, I'd suggest enabling debug and
checking why these nodes are being filtered out.
Mohammed
On Mon, May 30, 2022 at 9:39 AM Massimo Sgaravatto
<massimo.sgaravatto at gmail.com> wrote:
>
> It looks like I need now to create a HostAggregate for "size=big" for each project, specifying as properties:
> filter_tenant_id=<projectid>, size='big'
>
> Is this the expected behaviour ?
>
> Till Train a single HostAggregate with the property: size='big' was enough
>
> Thanks, Massimo
>
>
>
> On Fri, May 27, 2022 at 10:08 AM Massimo Sgaravatto <massimo.sgaravatto at gmail.com> wrote:
>>
>> Dear all
>>
>> We have the following use case:
>>
>> - reserve 3 hypervisors for VMs with "big" flavors (whatever the users of these instances are)
>> - partition the rest of the hypervisors according to the project (so projects A1,A2..,An can use only subset S1 of hypervisors, project B1,B2,..,Bm can use only subset S2 of hypervisors)
>>
>> We implemented this:
>>
>> 1- by setting an aggregate_instance_extra_spec 'size' property (with value 'normal' or 'big') for each flavor [*]
>> 2- by creating a BigVMs HostAggregate for size=big [**]
>> 3- by creating an HostAggregate for size=normal for each project, such as this one [***]
>>
>>
>> This used to work.
>>
>> A few days ago we updated our infrastructure from Train to Yoga
>> This was an offline Fast Forward Update: we went through the intermediate releases just to do the dbsyncs.
>> Since this update the instantiation of VMs with flavors with the size=big property doesn't work anymore
>>
>>
>> This is what I see in nova-scheduler log:
>> 2022-05-27 08:38:02.058 5273 INFO nova.filters [req-f92c0e38-262a-4d22-a7fd-8874c4265401 e237e43716fb490db5bda4b777835669 32b5d42c02b0411b8ebf2c33079eeecf - default default] Filtering removed all hosts for the request with instance ID '2412e188-9d5f-4812-ad21-195769a3c220'. Filter results: ['AggregateMultiTenancyIsolation: (start: 59, end: 10)', 'AggregateInstanceExtraSpecsFilter: (start: 10, end: 0)']
>>
>>
>> Only modifying the property of the BigVMs HA using the filter_tenant_id adding the relevant project:
>>
>> [root at cld-ctrl-01 ~]# openstack aggregate show BigVMs | grep prop
>> | properties | filter_tenant_id='32b5d42c02b0411b8ebf2c33079eeecf', size='big' |
>>
>> the scheduling works
>>
>> Specifying each project and keeping the list up-to-date would be a problem. Moreover if I am not wrong there is a maximum length for the property field.
>>
>> Any hints ?
>> I didn't find anything related to this issue in the nova release notes for Openstack releases > Train
>>
>>
>> These are the filters that we enabled:
>>
>> [filter_scheduler]
>> enabled_filters = AggregateMultiTenancyIsolation,AggregateInstanceExtraSpecsFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGro\
>> upAffinityFilter,PciPassthroughFilter,NUMATopologyFilter
>>
>>
>> Thanks a lot, Massimo
>>
>>
>> [*]
>> E.g.
>> [root at cld-ctrl-01 ~]# openstack flavor show cldareapd.medium | grep prope
>> | properties | aggregate_instance_extra_specs:size='normal' |[root at cld-ctrl-01 ~]# openstack flavor show cloudvenetocloudveneto.40cores128GB25-bigunipd | grep prop
>> | properties | aggregate_instance_extra_specs:size='big' |
>>
>>
>> [**]
>>
>> [root at cld-ctrl-01 ~]# openstack aggregate show BigVMs
>> +-------------------+---------------------------------------------------------------------------------------+
>> | Field | Value |
>> +-------------------+---------------------------------------------------------------------------------------+
>> | availability_zone | nova |
>> | created_at | 2018-06-20T06:54:51.000000 |
>> | deleted_at | None |
>> | hosts | cld-blu-08.cloud.pd.infn.it, cld-blu-09.cloud.pd.infn.it, cld-blu-10.cloud.pd.infn.it |
>> | id | 135 |
>> | is_deleted | False |
>> | name | BigVMs |
>> | properties |size='big' |
>> | updated_at | None |
>> | uuid | 4b593395-1c76-441c-9022-d421f4ea2dfb |
>> +-------------------+---------------------------------------------------------------------------------------+
>>
>>
>> [***]
>> [root at cld-ctrl-01 ~]# openstack aggregate show Unipd-AdminTesting-Unipd
>> +-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
>> | Field | Value |
>> +-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
>> | availability_zone | nova |
>> | created_at | 2018-04-12T07:31:01.000000 |
>> | deleted_at | None |
>> | hosts | cld-blu-01.cloud.pd.infn.it, cld-blu-02.cloud.pd.infn.it, cld-blu-05.cloud.pd.infn.it, cld-blu-06.cloud.pd.infn.it, cld-blu-07.cloud.pd.infn.it, cld-blu-11.cloud.pd.infn.it, cld-blu-12.cloud.pd.infn.it, cld-blu-13.cloud.pd.infn.it, cld-blu-14.cloud.pd.infn.it, cld-blu-15.cloud.pd.infn.it, cld-blu-16.cloud.pd.infn.it |
>> | id | 126 |
>> | is_deleted | False |
>> | name | Unipd-AdminTesting-Unipd |
>> | properties | filter_tenant_id='32b5d42c02b0411b8ebf2c33079eeecf', size='normal' |
>> | updated_at | 2018-06-08T09:06:20.000000 |
>> | uuid | 38f6a0d4-77ab-42e0-abeb-57e06ba13cca |
>> +-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
>> [root at cld-ctrl-01 ~]#
>>
--
Mohammed Naser
VEXXHOST, Inc.
More information about the openstack-discuss
mailing list