Open Stack

On 15/08/13 17:50 -0500, Christopher Armstrong wrote:
>*Introduction and Requirements*
>
>So there's kind of a perfect storm happening around autoscaling in Heat
>right now. It's making it really hard to figure out how I should compose
>this email. There are a lot of different requirements, a lot of different
>cool ideas, and a lot of projects that want to take advantage of
>autoscaling in one way or another: Trove, OpenShift, TripleO, just to name
>a few...
>
>I'll try to list the requirements from various people/projects that may be
>relevant to autoscaling or scaling in general.
>
>1. Some users want a service like Amazon's Auto Scaling or Rackspace's
>Otter -- a simple API that doesn't really involve orchestration.
>2. If such a API exists, it makes sense for Heat to take advantage of its
>functionality instead of reimplementing it.
>3. If Heat integrates with that separate API, however, that API will need
>two ways to do its work:
>   1. native instance-launching functionality, for the "simple" use
>   2. a way to talk back to Heat to perform orchestration-aware scaling
>operations.
>4. There may be things that are different than AWS::EC2::Instance that we
>would want to scale (I have personally been playing around with the concept
>of a ResourceGroup, which would maintain a nested stack of resources based
>on an arbitrary template snippet).
>5. Some people would like to be able to perform manual operations on an
>instance group -- such as Clint Byrum's recent example of "remove instance
>4 from resource group A".
>
>Please chime in with your additional requirements if you have any! Trove
>and TripleO people, I'm looking at you :-)
>
>
>*TL;DR*
>
>Point 3.2. above is the main point of this email: exactly how should the
>autoscaling API talk back to Heat to tell it to add more instances? I
>included the other points so that we keep them in mind while considering a
>solution.
>
>*Possible Solutions*
>
>I have heard at least three possibilities so far:
>
>1. the autoscaling API should maintain a full template of all the nodes in
>the autoscaled nested stack, manipulate it locally when it wants to add or
>remove instances, and post an update-stack to the nested-stack associated
>with the InstanceGroup.
>
>Pros: It doesn't require any changes to Heat.
>
>Cons: It puts a lot of burden of state management on the autoscale API, and
>it arguably spreads out the responsibility of "orchestration" to the
>autoscale API. Also arguable is that automated agents outside of Heat
>shouldn't be managing an "internal" template, which are typically developed
>by devops people and kept in version control.
>
>2. There should be a new custom-built API for doing exactly what the
>autoscaling service needs on an InstanceGroup, named something unashamedly
>specific -- like "instance-group-adjust".
>
>Pros: It'll do exactly what it needs to do for this use case; very little
>state management in autoscale API; it lets Heat do all the orchestration
>and only give very specific delegation to the external autoscale API.
>
>Cons: The API grows an additional method for a specific use case.

I like this one above:
adjust(new_size, victim_list=['i1','i7'])

So if you are reducing the new_size we look in the victim_list to
choose those first. This should cover Clint's use case as well.

-Angus

>
>3. the autoscaling API should update the "Size" Property of the
>InstanceGroup resource in the stack that it is placed in. This would
>require the ability to PATCH a specific piece of a template (an operation
>isomorphic to update-stack).
>
>Pros: The API modification is generic, simply a more optimized version of
>update-stack; very little state management required in autoscale API.
>
>Cons: This would essentially require manipulating the user-provided
>template.  (unless we have a concept of "private properties", which perhaps
>wouldn't appear in the template as provided by the user, but can be
>manipulated with such an update stack operation?)
>
>
>*Addenda*
>
>Keep in mind that there are use cases which require other types of
>manipulation of the InstanceGroup -- not just the autoscaling API. For
>example, see Clint's #5 above.
>
>
>Also, about implementation: Andrew Plunk and I have begun work on Heat
>resources for Rackspace's Otter, which I think will be a really good proof
>of concept for how this stuff should work in the Heat-native autoscale API.
>I am trying to gradually work the design into the native Heat autoscaling
>design, and we will need to solve the autoscale-controlling-InstanceGroup
>issue soon.
>
>-- 
>IRC: radix
>Christopher Armstrong
>Rackspace

>_______________________________________________
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev