Open Stack

On Tue, 16 Apr 2019, Eric Fried wrote:

> Interestingly, the above is closely approaching the space we're
> exploring for "subtree affinity". I'm wondering if there's a Unified
> Solution...

Some kind of unified solution is what I'm trying to find too, mostly
by noodling.

> I also said "feels". I can't really explain it any better than I could
> explain why "using group numbers as values" gave me the ooks. And given
> we're coming up ugly with all other proposals, convince me that this one
> is practical and not fraught with peril and I'll quickly get over my
> discomfort. Right now I'm pretty close to that point because it
> elegantly solves both classes of problem described above, and I can't
> think of a way to break it that isn't ridiculously contrived.

My message isn't a proposal, it's a question to see if there is a
proposal in there somewhere.

>> Similarly, aggregate membership would flow down as well, because a
>> child is always in its parent's aggregate too because it is inside its
>> parent.
>
> This one I'm not so convinced about. Can we defer making changes here
> until we have similarly concrete use cases?

I mentioned it because Tetsuro is suggesting that we likely do or
will have use cases, in thread

   http://lists.openstack.org/pipermail/openstack-discuss/2019-April/thread.html#4787

and if traits flow down and aggregate membership does not, then we
don't have a grand unified theory and a big point of my message is
gone.

>> A numeric requiredN or member_ofN span would be capped by the resource
>> provider that satisfied resourcesN.
>
> Eh? I was following you up to this point. Do you just mean that we don't
> have to worry about ascending the tree looking for requiredN because the
> trait is implicitly on the provider with resourceN by virtue of being on
> its ancestor?

No, I think I just got lost trying to explain myself and thinking
about making sure that distinct number groups still manage to stay
distinct. So let's go with "ignore that paragraph".

>> We need to work out a consistent and relatively easy to explain
>> mental model for this, because we need to be able to talk about it
>> with other people without them being required to re-experience all
>> the mental hurdles we are having to overcome.
>
> I think the hurdles are more around "why" and "are you sure you want to"
> - once we've made those decisions, IMO it can be understood fairly
> easily with one or both of "encapsulation" and "traits flow down" as
> you've explained them.

I think perhaps you've been in this too long to recognize how mind
boggling some of the concepts (and the inconsistencies thereof) can
be. If we are able to come up with not just a unified solution, but
a unified theory that supports that solution, it's a huge win for
those that follow us.

>> [4] A corollary could be "classes of inventory always flow up": If
>> you need a SRIOV_NET_VF, this root resource provider can provide it
>> because it has a great grandchild which has it.
>
> This one bakes my noodle pretty good. I have a harder time visualizing
> how the above use cases are satisfied by walking backwards up the tree
> accumulating resources (and you have to accumulate the traits as well,
> right?) until I hit a point where I've gathered everything I need.

Classes flowing up is already true for unnumbered resources. If you
don't specify which group you want it from, any child can provide
it. That's what I was describing. The associated question would be
if a numbered resources should do the same, which presumably it
should not because that would pretty much violate the point of
numbered, wouldn't it? Which begs the question: Why/How are traits
and maybe aggregates different? I think the answer is the
containership.

> So I'll come down in favor of making "traits flow down" happen. Question
> is, how? (And I know we've talked about this before - maybe Queens+Denver?)
>
> (A) In the database.

This feels like it misrepresents the reality and the relationships
and the concept of _nested_. But the way the data is represented in
the DB doesn't have to directly map to the meaning, so it might be a
goer. A worthwhile member of the list of alternatives.

> (B) In the algorithms.
>  (i) GET /rps and GET /a_cs queries need JOINs I can't even begin to
> comprehend.

If we wanted to we could figure this out (or do a D as another way
round). I think if we're going to commit to use a SQL db as the
datastore, we can do quite a bit of work to improve our SQL,
probably by writing some SQL, directly, in a manual test environment
to find out the right queries (or just get a dump out of Jay's
brain). Doing recursive or graph queries in tables is a thing people
do, it can be looked up on stackoverflow, etc.

That is, I think this is a surmountable problem and the organic
approach we've had up to now may make it look harder than it might
be.

Of course I could be completely wrong about that. I haven't tried
it yet, and I don't expect to have any chance before mid May.

>  (ii) Do we tweak the outputs (GET /rps response and GET /a_cs
> provider_summaries) to report the "inherited" traits as well?

That seems wrong to me. a) we don't list anything but uuid, name and
generation on /rps, b) in /a_cs the meaningful thing is the groups
of providers. That inheritance satisfied a trait isn't meaningful,
it's that something in the group did, isn't it? Plus, we don't want
to misrepresent the reality of traits back to, for example, the
weighers in nova-scheduler.

> Perhaps this suggests a hybrid approach:
>
> (C) Create a "ghost" table of inherited resource provider traits. If
> $old_microversion we ignore it; if $new_microversion we logically
> combine it with the existing rp traits table in all our queries.

This sounds a lot like a 'view', in which case if a view is
possible, then B is possible. Also, a ghost table sounds a whole lot
like a cache and makes me scared (as a ghost should I suppose) and:

     https://twitter.com/jaypipes/status/1113457517538021376

(D) Switch to a graph db. I think this is an important avenue of
exploration for greenfield deployments but pretty much no good for
existing deployments given the historical shyness about
compatibility, migrations, etc. (as well as fatigue that people seem
to have from being compelled to do a migration to get existing
placement).  However, the exploration could very well reveal some
techniques that are transferable.

Thanks for making a list of suggestions. Having ideas to compare is
awesome. I think the immediate next step is continue sharing and
comparing.


-- 
Chris Dent                       ٩◔̯◔۶           https://anticdent.org/
freenode: cdent                                         tw: @anticdent