[openstack-dev] Scheduler hints, API and Objects

Nikola Đipanov ndipanov at redhat.com
Fri Sep 4 12:57:20 UTC 2015


On 06/25/2015 04:50 PM, Monty Taylor wrote:
> On 06/25/2015 10:22 AM, Andrew Laski wrote:
>> I have been growing concerned recently with some attempts to formalize
>> scheduler hints, both with API validation and Nova objects defining
>> them, and want to air those concerns and see if others agree or can help
>> me see why I shouldn't worry.
>>
>> Starting with the API I think the strict input validation that's being
>> done, as seen in
>> http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/compute/schemas/v3/scheduler_hints.py?id=53677ebba6c86bd02ae80867028ed5f21b1299da,
>> is unnecessary, and potentially problematic.
>>
>> One problem is that it doesn't indicate anything useful for a client. 
>> The schema indicates that there are hints available but can make no
>> claim about whether or not they're actually enabled.  So while a
>> microversion bump would typically indicate a new feature available to an
>> end user, in the case of a new scheduler hint a microversion bump really
>> indicates nothing at all.  It does ensure that if a scheduler hint is
>> used that it's spelled properly and the data type passed is correct, but
>> that's primarily useful because there is no feedback mechanism to
>> indicate an invalid or unused scheduler hint.  I think the API schema is
>> a poor proxy for that deficiency.
>>
>> Since the exposure of a hint means nothing as far as its usefulness, I
>> don't think we should be codifying them as part of our API schema at
>> this time.  At some point I imagine we'll evolve a more useful API for
>> passing information to the scheduler as part of a request, and when that
>> happens I don't think needing to support a myriad of meaningless hints
>> in older API versions is going to be desirable.
> 
> I totally agree.
> 
> If hints are to become an object, then need to be _real_ resources that
> can be listed, and that have structured metadata that has an API.
> Flavors are a great example of this. From an end user perspective, I can
> ask the cloud what flavors exist, those flavors tell me information that
> I can use to make a decision, and I can pass in a reference to those
> things. If I pass in an invalid flavor, I get a meaningful error message.
> 
>> Finally, at this time I'm not sure we should take the stance that only
>> in-tree scheduler hints are supported.  While I completely agree with
>> the desire to expose things in cross-cloud ways as we've done and are
>> looking to do with flavor and image properties I think scheduling is an
>> area where we want to allow some flexibility for deployers to write and
>> expose scheduling capabilities that meet their specific needs.  Over
>> time I hope we will get to a place where some standardization can
>> happen, but I don't think locking in the current scheduling hints is the
>> way forward for that.  I would love to hear from multi-cloud users here
>> and get some input on whether that's crazy and they are expecting
>> benefits from validation on the current scheduler hints.
> 
> As a multi-cloud user, I do not use scheduler hints because there is no
> API to discover that they exist, and also no shared sense of semantics.
> (I know a flavor that claims 8G of RAM will give me, you guessed it, 8G
> of ram) So I consider scheduler hints currently to be COMPLETE vendor
> lock-in and/or only things to be used by private cloud folks who are
> also admins of their clouds.
> 
> I would not touch them with a 10 foot pole until such a time as there is
> an actual API for listing, describing and selecting them.
> 
> I would suggest that if we make one of those, we should quickly
> formalize meanings of fields - so that cloud can have specific hints
> that seem like cloud content - but that the way I learn about them is
> the same, and if there are two hints that do the same thing I can expect
> them to look the same in two different clouds.
> 

So this kind of argumentation keeps confusing me TBH. Unless I am not
understanding some basic things about how Nova works, the above argument
cleanly applies to flavors as well. Flavor '42' is not going to be the
same thing across clouds, but that's not where this ends. Once you throw
in extra_specs, in particular related to PCI devices and NUMA/CPU
pinning features. There is really no discoverability there whatsoever (*).

What I am trying to get to is not whether this is right or wrong, but to
point out the fact that Flavors are simply not a good abstraction that
can have reasonable meaning "across cloud boundaries" (i.e. different
Nova deployments), at least the way they are implemented at the moment.
We should not pretend that they are, and try to demonize useful code
making use of them, but come up with a better abstraction that can have
reasonable meaning across different deployments.

I think this is what Andrew was hinting at when he said that scheduling
is an area that cannot reasonably be standardized in this way.

I recently spoke to John briefly about this and got the feeling that he
has similar views - but I'd encourage him to comment if he wishes to do so.

N.

(*) For PCI devices, we can list aliases per host, but that's clearly an
admin API so not suitable for end user consumption really, and the alias
is an opaque string that is defined in nova.conf - has no meaning
outside a particular deployment really.



More information about the OpenStack-dev mailing list