Open Stack

Excerpts from Mike Spreitzer's message of 2014-04-03 11:05:10 -0700:
> Clint Byrum <clint at fewbar.com> wrote on 04/03/2014 01:10:30 PM:
> 
> > Things that affect the stack as a whole really belong in the stack
> > API. That would also put them in the OS::Heat::Stack resource, so the
> > template language already supports that.
> 
> The OS::Heat::Stack resource is one of several that create nested stacks;
> we should be able to apply holistic scheduling to all stacks, regardless
> of whether they are nest or which kind of nested stack they are.
> Yes, if holistic scheduling were a feature in the Heat API then all kinds 
> of
> resources that create nested stacks "should" expose that feature
> (shout out to Trove, autoscaling groups, ...).
> 
> > As for policies which might affect a holistic scheduler, those can just
> > be resources as well. Just like deployments relate to servers, resources
> > can relate to any policies they need.
> 
> A holistic scheduler needs input that describes all the resources to be 
> scheduled as well as all the policies that apply.  Should a template 
> contain a resource whose input includes a copy of the rest of the 
> template?
> 

Are you suggesting that a scheduler should try and parse the template
itself? Or that the scheduler would take over scheduling which
heat-engine takes the work? The whole question raises many more
questions, and I wonder if there's just something you haven't told us
about this use case. :-P

A holistic scheduler is meant to relate things. Templates relate things
naturally, so doing it with resources seems obvious to me.

Something like OS::Scheduler::ResourceGroup which would inform the
scheduler that a grouping is needed. And then the resources all are
part of that group via their properties, something like 'resource_group:
{get_resource: group1}'. If there's a policy for that group that I want
applied, that is a policy that would also refer to the group and inform
the scheduler that this group gets this policy.

For instances where the whole stack needs to be considered in a group,
this is where I suggest that all resources should just be added to the
group. When we have proof that this approach works, we can talk about
introducing shorthand for it.

What I don't want to see is a special template section which introduces
unnecessary complexity before we have concrete evidence that the approach
is viable.

> > I would prefer that we focus on making HOT composable rather than
> > extensible. If there is actually something missing from the root of the
> > language, then it should be in the language. Use cases should almost
> > always try to find a way to work as resources first, and then if that
> > is unwieldy, look into language enhancements to factor things out.
> 
> Yeah, I would too.  Like I said, I have no satisfactory solution yet. Here 
> is more of the problem.  I would like to follow an evolutionary path 
> starting from the instance groups that are in Nova today.  I think I can 
> outline such an evolution.  I am sure there will be debate about it.  I am 
> even more sure that it will take time to accomplish that evolution.  OTOH, 
> locally we have a holistic scheduler already working.  We want to be able 
> to start using it today.  What can we do in this interim, and how can we 
> arrange things to do progressive convergence so that the interim solution 
> evolves as Nova & scheduling evolve, so that there is no big switch at the 
> end to the end solution?

Big switches are fine as long as they're simplifications. So if we make
you be explicit about the groupings today, but then you find that you
have many stacks where everything is in fact in one group, then you can
make a clear argument for a root level item to set a property on all
resources which support it, which I think is the right way to go.