Open Stack

On Thu, Jun 13, 2013 at 12:08:00PM +0100, Mark McLoughlin wrote:
> On Thu, 2013-06-13 at 11:56 +0100, Steven Hardy wrote:
> > On Thu, Jun 13, 2013 at 11:13:02AM +0100, Mark McLoughlin wrote:
> > > On Thu, 2013-06-13 at 10:45 +0100, Steven Hardy wrote:
> > > > On Wed, Jun 12, 2013 at 05:06:31PM -0500, Christopher Armstrong wrote:
> > > 
> > > > > I believe the consensus is that the work on the new HOT format and
> > > > > underlying design changes is a prerequisite for the autoscaling work that
> > > > > Thomas and I are to do, so we'll definitely help with that. I'll let
> > > > > someone with a better understanding chime in about that.
> > > > 
> > > > As others have mentioned, I'm not clear on this coupling between template
> > > > and Autoscaling, it seems like this superset of AS functionality will just
> > > > require a new resource/resources to provide the interface, and as
> > > > previously discussed, probably an API which allows AS functionality to be
> > > > consumed without directly creating a Heat template (we may do that behind
> > > > the scenes though..)
> > > 
> > > Is there any reason for the non-template based autoscaling API to be
> > > part of Heat at all? Why would it not be a separate standalone
> > > autoscaling service?
> > 
> > Long term, probably not, short term yes, because the heat-engine provides
> > the actual AS logic behind the API, and it's not yet abtracted in a way
> > which can easily be separated.
> > 
> > The potential roadmap we discussed during sessions at summit was something
> > like:
> > 
> > 1. Add an AS API to heat, to provide more granular control of AS
> > functionality (and access to features not supported by our current CFN
> > resource abstractions)
> > 
> > 2. Allow AS API to provide AS functionality without specifying a heat
> > template, e.g by passing a list of Instance uuids (we'd probably initially
> > create a heat template internally to do this)
> > 
> > 3. Work on separating out the AS logic such that it is consumable without
> > the other heat core services (main heat API and heat-engine)
> > 
> > 4. Split off into a new incubated standalone AS project
> > 
> > 
> > This is essentially the roadmap I proposed back in May, which nobody
> > objected to:
> > 
> > http://lists.openstack.org/pipermail/openstack-dev/2013-May/008349.html
> > 
> > Of course the alternative is to start a separate Autoscaling standalone
> > project from scratch now, but IMHO the incremental add-features then split
> > approach results in better collaboration and less duplicated effort
> > (considering we already have a working Autoscaling solution, which is part
> > of Openstack, in Heat)
> > 
> > The other thing worth mentioning is that there's work happening right now
> > to integrate with the new ceilometer alarms functionality - when this is
> > done I expect autoscaling to be even simpler than it is now, basically
> > alarms in ceilometer will trigger orchestration actions in Heat, where we
> > just provide the orchestration/workflow of the scale up/down
> 
> Yep, I love the dependency diagram of the blueprints:
> 
> https://blueprints.launchpad.net/ceilometer/+spec/alarming
> 
> Exciting stuff!
> 
> > I think it's too early to say for sure if a separate project providing
> > Autoscaling is justified - my current feeling is that when integration with
> > ceilometer is complete it probably won't be but we'll have to get that work
> > done then evaluate.
> 
> Ok. For me, I just can't get my head around the idea that heat (the
> project, as opposed to the heat api service) would be responsible for
> anything that isn't specific to templates/stacks. Autoscaling is
> definitely something I could imagine people caring about even if they
> don't care about the template/stack orchestration domain.

Well I'd be interested to fully understand the use-cases which require
autoscaling but don't require any orchestration.  It seems to me that the
two are closely related, since when you autoscale stuff you have to manage
orchestration of provisioning, updating dependent resources, co-ordinating
updates to instances in the group etc etc, which is obviously what Heat is
all about.

> Or another way of thinking about it is if autoscaling was a standalone
> service provided by the heat project, anyone running that service would
> have to look very carefully at any heat security advisories to check
> whether it applies to the autoscaling service.
>
> But I'm fine with the "let's build it as a standalone service in Heat
> and then see if it makes sense to split it out" approach.

Well I'm actually proposing lets add functionality to what we've got and
think about how it could potentially be made consumable as a standalone
service.  If we get to the point of actually providing a standalone
service, then we can revisit the separate project discussion, which would
address the concerns above.

It would be different if there was an Autoscaling service project either in
Openstack, or being proposed for incubation, ie something Heat could
actually use to replace our existing implementation, but there isn't -
hence my build-on-what-we-have opinion.

Cheers,

Steve