Open Stack

On Wed, Jun 19, 2013 at 04:01:24AM +0000, Adrian Otto wrote:
> Angus,
> 
> I'm glad you are asking good questions. I have additional input for you to consider below.
> 
> On Jun 18, 2013, at 6:09 PM, Angus Salkeld <asalkeld at redhat.com> wrote:
> 
> > On 18/06/13 23:32 +0000, Adrian Otto wrote:
> >> Yes. I think having a POST method in the API makes perfect sense. Assuming we reach agreement on that, the next question that comes up is:
> > 
> > Err, I am not convinced.
> > 
> > Before that, I think it's worth highlighting the different proposals/requirements here:
> > 
> > 1) heat needs a way to setup an autoscaling group
> > 2) autoscaling needs a way of telling heat to scale up/down
> > 3) People might want to integrate other orchestration engines with Heat

FWIW, I thought it was more that people might want to integrate
autoscaling with other orchestration engines, which IMO will become
possible when ceilometer alarms are completed - autoscaling ==
alarms+orchestration IMO, but that's somewhat OT for this thread ;)

> > So one by one:
> > 1) autoscaling has a post api that the Heat autoscaling group resource
> >   posts to, Heat will provide a webhook in the event of a scaling action.
> > 2) when autoscaling determines that there should be a scaling action
> >   it calls the webhook. The reason I suggest a webhook is in-instance
> >   applications might want to scale their applications (like
> >   openshift-gears) - so don't assume a heat endpoint.
> >   Also we can have this as an action not a resource-create.
> >   PUT /$tenant/stacks/$stack/resources/<autoscale-group>
> >   (with an action of scale-up/down)
> > 
> > I think we should discourage (make it impossible) for users from modifing
> > stacks outside of stack-update.
> > 
> > To me one of the most powerful and apealing things of Heat is the
> > ability to reproducibly re-create a stack from a template. This
> > new public API is going to make this difficult.
> 
> Adding an API only makes it difficult if you decide to bypass templates and use the API. You can still be disciplined and keep your templates updated to achieve the reproducible goal. Yes, and API of this sort is a sharp instrument, but it can be useful if applied properly.

So just use templates, which avoids the potential for abuse, the complexity
of two interfaces to the same functionality, and ensures your deployments
are repeatable.

> The ability to quickly and easily add or modify a resource in a stack/assembly is very useful, particularly in cases where an orchestration was partially successful, but gets stuck. Imagine if it's something that takes 4 hours to conduct from beginning to end, and you got 3.8 hours into it when something jammed. It would be *really* handy to be able to un-jam it without starting over from scratch. You'd still need to fix your template if you wanted to tweak to persist for the next time you use the template.

This is already possible via stack update, and the partially successful
case has already been discussed and is captured in this bug:

https://bugs.launchpad.net/heat/+bug/1160052

> It's also useful for situations like rolling updates, for when a complete redeployment is not practical, affordable, or desirable. Think of this as the point of intersection between a configuration management system and the orchestration system. Yes, you can accomplish this with stack-update, but that could be rather awkward or impractical depending on the size of the stack/assembly. Imagine something with 2000 nodes in it… perhaps a large scale triple-o use case. Sometimes groupings are sufficient, and sometimes they are not.

I think that for this use case, stack update will still work fine, *but*
we'll need to allow recursive updates of composed stacks, ie updating
nested stacks (we currently just replace them if the nested stack
definition changes).

I've raised the following BP, I don't think this will be that hard to
implement as it's effectively reusing our existing stack update mechanism.

https://blueprints.launchpad.net/heat/+spec/nested-stack-updates

In your example above, I think enforcing deployment via version controlled
templates is even more important - having a huge infrastructure deployment
which is asynchonously updated, not version controlled and essentially in
an unknown state sounds like a maintenance nightmare to me.

> > Also creating a resource is not a trivial issue, the user would have
> > create the resources in the correct order (with correct inter-resource
> > references etc..) mostly throwing away the point of orchestation in
> > the first place. If you are doing this you may as well talk directly
> > to nova/cinder/networking directly.
> 
> Using the low level API's would bypass the workflow features of Heat, which we plan to contribute to considerably. We don't want to do that. We want to leverage those workflow features. There is value in pulling strings through Heat, even if all you are doing is adding a node. There is value to having an expression of the complete deployment, and all of the related resources that's consistent with what's happened with the lower level APIs (such as nova/cinder/networking). That value does not diminish when you have a need to surgically alter a stack.

So we just use stack update - because it exists now, and we may as well
improve that existing mechanism for modifying stack definitions, rather
than inventing a new interface to functionality we already have (IMHO) :)

Thanks,

Steve