Open Stack

On 09/04/15 at 01:57pm, Nikola Đipanov wrote:
>On 06/25/2015 04:50 PM, Monty Taylor wrote:
>> On 06/25/2015 10:22 AM, Andrew Laski wrote:
>>> I have been growing concerned recently with some attempts to formalize
>>> scheduler hints, both with API validation and Nova objects defining
>>> them, and want to air those concerns and see if others agree or can help
>>> me see why I shouldn't worry.
>>>
>>> Starting with the API I think the strict input validation that's being
>>> done, as seen in
>>> http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/compute/schemas/v3/scheduler_hints.py?id=53677ebba6c86bd02ae80867028ed5f21b1299da,
>>> is unnecessary, and potentially problematic.
>>>
>>> One problem is that it doesn't indicate anything useful for a client.
>>> The schema indicates that there are hints available but can make no
>>> claim about whether or not they're actually enabled.  So while a
>>> microversion bump would typically indicate a new feature available to an
>>> end user, in the case of a new scheduler hint a microversion bump really
>>> indicates nothing at all.  It does ensure that if a scheduler hint is
>>> used that it's spelled properly and the data type passed is correct, but
>>> that's primarily useful because there is no feedback mechanism to
>>> indicate an invalid or unused scheduler hint.  I think the API schema is
>>> a poor proxy for that deficiency.
>>>
>>> Since the exposure of a hint means nothing as far as its usefulness, I
>>> don't think we should be codifying them as part of our API schema at
>>> this time.  At some point I imagine we'll evolve a more useful API for
>>> passing information to the scheduler as part of a request, and when that
>>> happens I don't think needing to support a myriad of meaningless hints
>>> in older API versions is going to be desirable.
>>
>> I totally agree.
>>
>> If hints are to become an object, then need to be _real_ resources that
>> can be listed, and that have structured metadata that has an API.
>> Flavors are a great example of this. From an end user perspective, I can
>> ask the cloud what flavors exist, those flavors tell me information that
>> I can use to make a decision, and I can pass in a reference to those
>> things. If I pass in an invalid flavor, I get a meaningful error message.
>>
>>> Finally, at this time I'm not sure we should take the stance that only
>>> in-tree scheduler hints are supported.  While I completely agree with
>>> the desire to expose things in cross-cloud ways as we've done and are
>>> looking to do with flavor and image properties I think scheduling is an
>>> area where we want to allow some flexibility for deployers to write and
>>> expose scheduling capabilities that meet their specific needs.  Over
>>> time I hope we will get to a place where some standardization can
>>> happen, but I don't think locking in the current scheduling hints is the
>>> way forward for that.  I would love to hear from multi-cloud users here
>>> and get some input on whether that's crazy and they are expecting
>>> benefits from validation on the current scheduler hints.
>>
>> As a multi-cloud user, I do not use scheduler hints because there is no
>> API to discover that they exist, and also no shared sense of semantics.
>> (I know a flavor that claims 8G of RAM will give me, you guessed it, 8G
>> of ram) So I consider scheduler hints currently to be COMPLETE vendor
>> lock-in and/or only things to be used by private cloud folks who are
>> also admins of their clouds.
>>
>> I would not touch them with a 10 foot pole until such a time as there is
>> an actual API for listing, describing and selecting them.
>>
>> I would suggest that if we make one of those, we should quickly
>> formalize meanings of fields - so that cloud can have specific hints
>> that seem like cloud content - but that the way I learn about them is
>> the same, and if there are two hints that do the same thing I can expect
>> them to look the same in two different clouds.
>>
>
>So this kind of argumentation keeps confusing me TBH. Unless I am not
>understanding some basic things about how Nova works, the above argument
>cleanly applies to flavors as well. Flavor '42' is not going to be the
>same thing across clouds, but that's not where this ends. Once you throw
>in extra_specs, in particular related to PCI devices and NUMA/CPU
>pinning features. There is really no discoverability there whatsoever (*).
>
>What I am trying to get to is not whether this is right or wrong, but to
>point out the fact that Flavors are simply not a good abstraction that
>can have reasonable meaning "across cloud boundaries" (i.e. different
>Nova deployments), at least the way they are implemented at the moment.
>We should not pretend that they are, and try to demonize useful code
>making use of them, but come up with a better abstraction that can have
>reasonable meaning across different deployments.
>
>I think this is what Andrew was hinting at when he said that scheduling
>is an area that cannot reasonably be standardized in this way.

Essentially.  Flavors work out because they can be queried in order to 
get at the important information.  It doesn't matter what the flavor is 
called in each cloud because you can programmatically find the flavor 
that gives you 8 GB of root disk.  I'm ignoring the other part of your 
point for now which is that there's no guarantee you can find a matching 
flavor across clouds or that you can even ascertain the exact meaning of 
a flavor when extra_specs are involved, but I agree.

But flavors still work well enough because something like root_gb is a 
concrete concept that doesn't change across clouds.  Scheduling concepts 
are more difficult to nail down in a consistent way.  There are some 
things like affinity that seem easy enough to express but still have 
some subjectivity to them.  How far away or how close should instances 
be in order to satisfy those constraints?  The in-tree hints require 
instances to be on the same host of different hosts.  But what if we 
wanted to provide affinity between a set of hosts, what's important to 
express there?  The number of hosts in that set, the fact that they're 
physically close to the same volume store, their IP space, etc...?  
There are so many potential things that deployers may want to provide 
that we almost need a full grammar to describe it all.  Maybe we'll get 
there, but for now I think we should be hesitant about locking this 
down.

>
>I recently spoke to John briefly about this and got the feeling that he
>has similar views - but I'd encourage him to comment if he wishes to do so.
>
>N.
>
>(*) For PCI devices, we can list aliases per host, but that's clearly an
>admin API so not suitable for end user consumption really, and the alias
>is an opaque string that is defined in nova.conf - has no meaning
>outside a particular deployment really.
>
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev