Open Stack

On 09/28/2018 12:19 PM, Chris Dent wrote:
> On Fri, 28 Sep 2018, Jay Pipes wrote:
> 
>> On 09/28/2018 09:25 AM, Eric Fried wrote:
>>> It's time somebody said this.
> 
> Yes, a useful topic, I think.
> 
>>> Every time we turn a corner or look under a rug, we find another use
>>> case for provider traits in placement. But every time we have to have
>>> the argument about whether that use case satisfies the original
>>> "intended purpose" of traits.
>>>
>>> That's only reason I've ever been able to glean: that it (whatever "it"
>>> is) wasn't what the architects had in mind when they came up with the
>>> idea of traits.
>>
>> Don't pussyfoot around things. It's me you're talking about, Eric. You
>> could just ask me instead of passive-aggressively posting to the list
>> like this.
> 
> It's not just you. Ed and I have also expressed some fairly strong
> statement about how traits are "supposed" to be used and I would
> guess that from Eric's perspective all three of us (amongst others)
> have some form of architectural influence. Since it takes a village
> and all that.

Correct. I certainly wasn't talking about Jay specifically. I also
wanted people other than placement cores/architects to participate in
the discussion (thanks Julia and Zane).

>> They aren't arbitrary. They are there for a reason: a trait is a
>> boolean capability. It describes something that either a provider is
>> capable of supporting or it isn't.
> 
> This is somewhat (maybe even only slightly) different from what I
> think the definition of a trait is, and that nuance may be relevant.
> 
> I describe a trait as a "quality that a resource provider has" (the
> car is blue). This contrasts with a resource class which is a
> "quantity that a resource provider has" (the car has 4 doors).

Yes, this. I don't want us to go off in the weeds about the reason or
relevance of the choice of name, but "trait" is a superset of
"capability" and easily encompasses "BLUE" or "PHYSNET_PUBLIC" or
"OWNED_BY_NEUTRON" or "XYZ_BITSTREAM" or "PCI_ADDRESS_01_AB_23_CD" or
"RAID5".

> Our implementation is pretty much exactly that ^. We allow
> clients to ask "give me things that have qualities x, y, z, not
> qualities a, b, c, and quanities of G of 5 and H of 7".
> 
> Add in aggregates and we have exactly what you say:
> 
>> * Does the provider have *capacity* for the requested resources?
>> * Does the provider have the required (or forbidden) *capabilities*?
>> * Does the provider belong to some group?
> 
> The nuance of difference is that your description of *capabilities*
> seems more narrow than my description of *qualities* (aka
> characteristics). You've got something fairly specific in mind, as a
> way of constraining the profusion of noise that has happened with
> how various kinds of information about resources of all sorts is
> managed in OpenStack, as you describe in your message.
> 
> I do not think it should be placement's job to control that noise.
> It should be placement's job to provide a very strict contract about
> what you can do with a trait:
> 
> * create it, if necessary
> * assign it to one or more resource providers
> * ask for providers that either have it
> * ... or do not have it
> 
> That's all. Placement _code_ should _never_ be aware of the value of
> a trait (except for the magical MISC_SHARES...). It should never
> become possible to regex on traits or do comparisons
> (required=<CUSTOM_TEMP_85). Just "yes" or "no" to presence of quality.
> 
>> If we want to add further constraints to the placement allocation
>> candidates request that ask things like:
>>
>> * Does the provider have version 1.22.61821 of BIOS firmware from
>> Marvell installed on it?
> 
> That's a quality of the provider in a moment.
> 
>> * Does the provider support an FPGA that has had an OVS program
>> flashed to it in the last 20 days?
> 
> If you squint, so is this.
> 
>> * Does the provider belong to physical network "corpnet" and also
>> support creation of virtual NICs of type either "DIRECT" or "NORMAL"?
> 
> And these.
> 
> But at least some of them are dynamic rather than some kind of
> platonic ideal associated with the resource provider.
> 
> I don't think placement should be concerned about temporal aspects
> of traits. If we can't write a web service that can handle setting
> lots of traits every second of every day, we should go home. If
> clients of placement want to set weird traits, more power to them.
> 
> However, if clients of placement (such as nova) which are being the
> orchestrator of resource providers manipulated by multiple systems
> (neutron, cinder, ironic, cyborg, etc) wish to set some constraints
> on how and what traits can do and mean, then that is up to them.
> 
> nova-scheduler is the thing that is doing `GET
> /allocation_candidates` for those multiple system. It presumably
> should have some say in what traits it is willing to express and
> use.

Right, this is where it's getting sticky. I feel like the push-back
comes from people wearing their placement hats saying "you can't (ab)use
placement like this, even though it would totally work" versus people
wearing their nova/ironic/whatever hats saying "we shouldn't favor this
implementation because there's something fundamentally wrong with it
and/or this other way would be better".

> But the placement service doesn't and shouldn't care.
> 
>> Then we should add a data model that allow providers to be decorated
>> with key/value (or more complex than key/value) information where we
>> can query for those kinds of constraints without needing to encode all
>> sorts of non-binary bits of information into a capability string.
> 
> Let's never do this, please. The three capabilities (ha!) of
> placement that you listed above ("Does the...") are very powerful as
> is and have a conceptual integrity that's really quite awesome. I
> think keeping it contained and constrained in very "simple" concepts
> like that was stroke of genius you (Jay) made and I'd hope we can
> keep it clean like that.

So here it is. Two of the top influencers in placement, one saying we
shouldn't overload traits, the other saying we shouldn't add a primitive
that would obviate the need for that. Historically, this kind of
disagreement seems to result in an impasse: neither thing happens and
those who would benefit are forced to find a workaround or punt.
Frankly, I don't particularly care which way we go; I just want to be
able to do the things.

> If we weren't a multiple-service oriented system, and instead had
> some kind of k8s-like etcd-like
> keeper-of-all-the-info-about-everything, then sure, having what we
> currently model as resource providers be a giant blob of metadata
> (with quantities, qualitiies, and key-values) that is an authority
> for the entire system might make some kind of sense.
> 
> But we don't. If we wanted to migrate to having something like that,
> using placement as the trojan horse for such a change, either with
> intent or by accident, would be unfortunate.
> 
>> Propose such a thing and I'll gladly support it. But I won't support
>> bastardizing the simple concept of a boolean capability just because
>> we don't want to change the API or database schema.
> 
> For me, it is not a matter of not wanting to change the API or the
> database schema. It's about not wanting to expand the concepts, and
> thus the purpose, of the system. It's about wanting to keep focus
> and functionality narrow so we can have a target which is "maturity"
> and know when we're there.
> 
> My summary: Traits are symbols that are 255 characters long that are
> associated with a resource provider. It's possible to query for
> resource providers that have or do not have a specific trait. This
> has the effect of making the meaning of a trait a descriptor of the
> resource provider. What the descriptor signifies is up to the thing
> creating and using the resource provider, not placement. We need to
> harden that contract and stick to it. Placement is like a common
> carrier, it doesn't care what's in the box.
> 
> /me cues brad pitt
> 
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>