Open Stack

Dear all,

Auto-Scaling is an important feature supported by Heat and needed by
many users we talked to.  There are two flavors of AutoScalingGroup
resources in Heat today: the AWS-based one and the Heat native one.  As
more requests coming in, the team has proposed to separate auto-scaling
support into a separate service so that people who are interested in it
can jump onto it.  At the same time, Heat engine (especially the resource
type code) will be drastically simplified.  The separated AS service
could move forward more rapidly and efficiently.

This work was proposed a while ago with the following wiki and
blueprints (mostly approved during Havana cycle), but the progress is
slow.  A group of developers now volunteer to take over this work and
move it forward.

wiki: https://wiki.openstack.org/wiki/Heat/AutoScaling
BPs:
 - https://blueprints.launchpad.net/heat/+spec/as-lib-db
 - https://blueprints.launchpad.net/heat/+spec/as-lib
 - https://blueprints.launchpad.net/heat/+spec/as-engine-db
 - https://blueprints.launchpad.net/heat/+spec/as-engine
 - https://blueprints.launchpad.net/heat/+spec/autoscaling-api
 - https://blueprints.launchpad.net/heat/+spec/autoscaling-api-client
 - https://blueprints.launchpad.net/heat/+spec/as-api-group-resource
 - https://blueprints.launchpad.net/heat/+spec/as-api-policy-resource
 - https://blueprints.launchpad.net/heat/+spec/as-api-webhook-trigger-resource
 - https://blueprints.launchpad.net/heat/+spec/autoscaling-api-resources

Once this whole thing lands, Heat engine will talk to the AS engine in
terms of ResourceGroup, ScalingPolicy, Webhooks.  Heat engine won't care
how auto-scaling is implemented although the AS engine may in turn ask
Heat to create/update stacks for scaling's purpose.  In theory, AS
engine can create/destroy resources by directly invoking other OpenStack
services.  This new AutoScaling service may eventually have its own DB,
engine, API, api-client.  We can definitely aim high while work hard on
real code.

After reviewing the BPs/Wiki and some communication, we get two options
to push forward this.  I'm writing this to solicit ideas and comments
from the community.

Option A: Top-Down Quick Split
------------------------------

This means we will follow a roadmap shown below, which is not 100% 
accurate yet and very rough:

  1) Get the separated REST service in place and working
  2) Switch Heat resources to use the new REST service

Pros:
  - Separate code base means faster review/commit cycle
  - Less code churn in Heat
Cons:
  - A new service need to be installed/configured/launched
  - Need commitments from dedicated, experienced developers from very
    beginning

Option B: Bottom-Up Slow Growth
-------------------------------

The roadmap is more conservative, with many (yes, many) incremental
patches to migrate things carefully.

  1) Separate some of the autoscaling logic into libraries in Heat
  2) Augment heat-engine with new AS RPCs
  3) Switch AS related resource types to use the new RPCs
  4) Add new REST service that also talks to the same RPC
     (create new GIT repo, API endpoint and client lib...) 

Pros:
  - Less risk breaking user lands with each revision well tested
  - More smooth transition for users in terms of upgrades

Cons:
  - A lot of churn within Heat code base, which means long review cycles
  - Still need commitments from cores to supervise the whole process

There could be option C, D... but the two above are what we came up with
during the discussion.

Another important thing we talked about is about the open discussion on
this.  OpenStack Wiki seems a good place to document settled designs but
not for interactive discussions.  Probably we should leverage etherpad
and the mailinglist when moving forward.  Suggestions on this are also
welcomed.

Thanks.

Regards,
 Qiming