Open Stack

Thanks Mohammed for your feedback

This [*] is what I see in the log file. The 3 relevant nodes are filtered
out by the AggregateMultiTenancyIsolation (i.e. they are not among the 10
returned hosts)

In the Train --> Yoga updated we also removed the AvailabilityZoneFilter
(which has been deprecated): I tried to re-add it but this didn't help
I can't remember other changes that could be relevant with this issue

Cheers, Massimo


[*]

2022-05-30 10:08:32.620 3168092 DEBUG nova.filters
[req-f698ce3b-18c9-4f3f-9b73-49496e19c237 e237e43716fb490db5bda4b777835669
32b5d42c02b0411b8ebf2c3\
3079eeecf - default default] Filtering removed all hosts for the request
with instance ID 'fd6d978a-3739-45c6-b1cc-28fc9e57d381'. Filter results:
[('\
AggregateMultiTenancyIsolation', [('cld-blu-12.cloud.pd.infn.it', '
cld-blu-12.cloud.pd.infn.it'), ('cld-blu-02.cloud.pd.infn.it',
'cld-blu-02.cloud.p\
d.infn.it'), ('cld-blu-15.cloud.pd.infn.it', 'cld-blu-15.cloud.pd.infn.it'),
('cld-blu-11.cloud.pd.infn.it', 'cld-blu-11.cloud.pd.infn.it'), ('cld-bl\
u-14.cloud.pd.infn.it', 'cld-blu-14.cloud.pd.infn.it'), ('
cld-blu-13.cloud.pd.infn.it', 'cld-blu-13.cloud.pd.infn.it'), ('
cld-blu-07.cloud.pd.infn.it\
', 'cld-blu-07.cloud.pd.infn.it'), ('cld-blu-06.cloud.pd.infn.it', '
cld-blu-06.cloud.pd.infn.it'), ('cld-blu-01.cloud.pd.infn.it',
'cld-blu-01.cloud.\
pd.infn.it'), ('cld-blu-16.cloud.pd.infn.it', 'cld-blu-16.cloud.pd.infn.it')]),
('AggregateInstanceExtraSpecsFilter', None)] get_filtered_objects /us\
r/lib/python3.6/site-packages/nova/filters.py:114
2022-05-30 10:08:32.620 3168092 INFO nova.filters
[req-f698ce3b-18c9-4f3f-9b73-49496e19c237 e237e43716fb490db5bda4b777835669
32b5d42c02b0411b8ebf2c33\
079eeecf - default default] Filtering removed all hosts for the request
with instance ID 'fd6d978a-3739-45c6-b1cc-28fc9e57d381'. Filter results:
['Ag\
gregateMultiTenancyIsolation: (start: 58, end: 10)',
'AggregateInstanceExtraSpecsFilter: (start: 10, end: 0)']

On Mon, May 30, 2022 at 9:51 AM Mohammed Naser <mnaser at vexxhost.com> wrote:

> Hi there,
>
> Are you sure you haven't done any other changes to your environment?
> Both those filters haven't been changed for years (~2017):
>
>
> https://github.com/openstack/nova/commits/master/nova/scheduler/filters/aggregate_multitenancy_isolation.py
>
> https://github.com/openstack/nova/commits/master/nova/scheduler/filters/aggregate_instance_extra_specs.py
>
> I think you've got something else here, I'd suggest enabling debug and
> checking why these nodes are being filtered out.
>
> Mohammed
>
> On Mon, May 30, 2022 at 9:39 AM Massimo Sgaravatto
> <massimo.sgaravatto at gmail.com> wrote:
> >
> > It looks like I need now to create a HostAggregate for "size=big" for
> each project, specifying as properties:
> >  filter_tenant_id=<projectid>, size='big'
> >
> > Is this the expected behaviour ?
> >
> > Till Train a single HostAggregate with the property: size='big' was
> enough
> >
> > Thanks, Massimo
> >
> >
> >
> > On Fri, May 27, 2022 at 10:08 AM Massimo Sgaravatto <
> massimo.sgaravatto at gmail.com> wrote:
> >>
> >> Dear all
> >>
> >> We have the following use case:
> >>
> >> - reserve 3 hypervisors for VMs with "big" flavors (whatever the users
> of these instances are)
> >> - partition the rest of the hypervisors according to the project (so
> projects A1,A2..,An can use only subset S1 of hypervisors, project
> B1,B2,..,Bm can use only subset S2 of hypervisors)
> >>
> >> We implemented this:
> >>
> >> 1- by setting an aggregate_instance_extra_spec  'size'  property (with
> value 'normal' or 'big') for each flavor  [*]
> >> 2- by creating a BigVMs  HostAggregate for size=big [**]
> >> 3- by creating an HostAggregate for size=normal for each project, such
> as this one [***]
> >>
> >>
> >> This used to work.
> >>
> >> A few days ago we updated our infrastructure from Train to Yoga
> >> This was an offline Fast Forward Update: we went through the
> intermediate releases just to do the dbsyncs.
> >> Since this update the instantiation of VMs with flavors with the
> size=big property doesn't work anymore
> >>
> >>
> >> This is what I see in nova-scheduler log:
> >> 2022-05-27 08:38:02.058 5273 INFO nova.filters
> [req-f92c0e38-262a-4d22-a7fd-8874c4265401 e237e43716fb490db5bda4b777835669
> 32b5d42c02b0411b8ebf2c33079eeecf - default default] Filtering removed all
> hosts for the request with instance ID
> '2412e188-9d5f-4812-ad21-195769a3c220'. Filter results:
> ['AggregateMultiTenancyIsolation: (start: 59, end: 10)',
> 'AggregateInstanceExtraSpecsFilter: (start: 10, end: 0)']
> >>
> >>
> >> Only modifying the  property of the BigVMs HA using the
> filter_tenant_id adding the relevant project:
> >>
> >> [root at cld-ctrl-01 ~]# openstack aggregate show BigVMs | grep prop
> >> | properties        |
> filter_tenant_id='32b5d42c02b0411b8ebf2c33079eeecf', size='big'
>            |
> >>
> >> the scheduling works
> >>
> >> Specifying each project and keeping the list up-to-date would be a
> problem. Moreover if I am not wrong there is a maximum length for the
> property field.
> >>
> >> Any hints ?
> >> I didn't find anything related to this issue in the nova release notes
> for Openstack releases > Train
> >>
> >>
> >> These are the filters that we enabled:
> >>
> >> [filter_scheduler]
> >> enabled_filters =
> AggregateMultiTenancyIsolation,AggregateInstanceExtraSpecsFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGro\
> >> upAffinityFilter,PciPassthroughFilter,NUMATopologyFilter
> >>
> >>
> >> Thanks a lot, Massimo
> >>
> >>
> >> [*]
> >> E.g.
> >> [root at cld-ctrl-01 ~]# openstack flavor show cldareapd.medium | grep
> prope
> >> | properties                 |
> aggregate_instance_extra_specs:size='normal' |[root at cld-ctrl-01 ~]#
> openstack flavor show cloudvenetocloudveneto.40cores128GB25-bigunipd | grep
> prop
> >> | properties                 |
> aggregate_instance_extra_specs:size='big'      |
> >>
> >>
> >> [**]
> >>
> >> [root at cld-ctrl-01 ~]# openstack aggregate show BigVMs
> >>
> +-------------------+---------------------------------------------------------------------------------------+
> >> | Field             | Value
>                                      |
> >>
> +-------------------+---------------------------------------------------------------------------------------+
> >> | availability_zone | nova
>                                     |
> >> | created_at        | 2018-06-20T06:54:51.000000
>                                     |
> >> | deleted_at        | None
>                                     |
> >> | hosts             | cld-blu-08.cloud.pd.infn.it,
> cld-blu-09.cloud.pd.infn.it, cld-blu-10.cloud.pd.infn.it |
> >> | id                | 135
>                                      |
> >> | is_deleted        | False
>                                      |
> >> | name              | BigVMs
>                                     |
> >> | properties        |size='big'                       |
> >> | updated_at        | None
>                                     |
> >> | uuid              | 4b593395-1c76-441c-9022-d421f4ea2dfb
>                                     |
> >>
> +-------------------+---------------------------------------------------------------------------------------+
> >>
> >>
> >> [***]
> >> [root at cld-ctrl-01 ~]# openstack aggregate show Unipd-AdminTesting-Unipd
> >>
> +-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
> >> | Field             | Value
>
>
>
>                                          |
> >>
> +-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
> >> | availability_zone | nova
>
>
>
>                                         |
> >> | created_at        | 2018-04-12T07:31:01.000000
>
>
>
>                                         |
> >> | deleted_at        | None
>
>
>
>                                         |
> >> | hosts             | cld-blu-01.cloud.pd.infn.it,
> cld-blu-02.cloud.pd.infn.it, cld-blu-05.cloud.pd.infn.it,
> cld-blu-06.cloud.pd.infn.it, cld-blu-07.cloud.pd.infn.it,
> cld-blu-11.cloud.pd.infn.it, cld-blu-12.cloud.pd.infn.it,
> cld-blu-13.cloud.pd.infn.it, cld-blu-14.cloud.pd.infn.it,
> cld-blu-15.cloud.pd.infn.it, cld-blu-16.cloud.pd.infn.it |
> >> | id                | 126
>
>
>
>                                          |
> >> | is_deleted        | False
>
>
>
>                                          |
> >> | name              | Unipd-AdminTesting-Unipd
>
>
>
>                                         |
> >> | properties        |
> filter_tenant_id='32b5d42c02b0411b8ebf2c33079eeecf', size='normal'
>
>
>
>               |
> >> | updated_at        | 2018-06-08T09:06:20.000000
>
>
>
>                                         |
> >> | uuid              | 38f6a0d4-77ab-42e0-abeb-57e06ba13cca
>
>
>
>                                         |
> >>
> +-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
> >> [root at cld-ctrl-01 ~]#
> >>
>
>
> --
> Mohammed Naser
> VEXXHOST, Inc.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20220530/18d75af8/attachment-0001.htm>