Open Stack

On Tue, Dec 2, 2014 at 8:15 AM, Zane Bitter <zbitter at redhat.com> wrote:

> On 28/11/14 02:33, Qiming Teng wrote:
>
>> Dear all,
>>
>> Auto-Scaling is an important feature supported by Heat and needed by
>> many users we talked to.  There are two flavors of AutoScalingGroup
>> resources in Heat today: the AWS-based one and the Heat native one.  As
>> more requests coming in, the team has proposed to separate auto-scaling
>> support into a separate service so that people who are interested in it
>> can jump onto it.  At the same time, Heat engine (especially the resource
>> type code) will be drastically simplified.  The separated AS service
>> could move forward more rapidly and efficiently.
>>
>> This work was proposed a while ago with the following wiki and
>> blueprints (mostly approved during Havana cycle), but the progress is
>> slow.  A group of developers now volunteer to take over this work and
>> move it forward.
>>
>
> Thank you!
>
>
>  wiki: https://wiki.openstack.org/wiki/Heat/AutoScaling
>> BPs:
>>   - https://blueprints.launchpad.net/heat/+spec/as-lib-db
>>   - https://blueprints.launchpad.net/heat/+spec/as-lib
>>   - https://blueprints.launchpad.net/heat/+spec/as-engine-db
>>   - https://blueprints.launchpad.net/heat/+spec/as-engine
>>   - https://blueprints.launchpad.net/heat/+spec/autoscaling-api
>>   - https://blueprints.launchpad.net/heat/+spec/autoscaling-api-client
>>   - https://blueprints.launchpad.net/heat/+spec/as-api-group-resource
>>   - https://blueprints.launchpad.net/heat/+spec/as-api-policy-resource
>>   - https://blueprints.launchpad.net/heat/+spec/as-api-webhook-
>> trigger-resource
>>   - https://blueprints.launchpad.net/heat/+spec/autoscaling-api-resources
>>
>> Once this whole thing lands, Heat engine will talk to the AS engine in
>> terms of ResourceGroup, ScalingPolicy, Webhooks.  Heat engine won't care
>> how auto-scaling is implemented although the AS engine may in turn ask
>> Heat to create/update stacks for scaling's purpose.  In theory, AS
>> engine can create/destroy resources by directly invoking other OpenStack
>> services.  This new AutoScaling service may eventually have its own DB,
>> engine, API, api-client.  We can definitely aim high while work hard on
>> real code.
>>
>> After reviewing the BPs/Wiki and some communication, we get two options
>> to push forward this.  I'm writing this to solicit ideas and comments
>> from the community.
>>
>> Option A: Top-Down Quick Split
>> ------------------------------
>>
>> This means we will follow a roadmap shown below, which is not 100%
>> accurate yet and very rough:
>>
>>    1) Get the separated REST service in place and working
>>    2) Switch Heat resources to use the new REST service
>>
>> Pros:
>>    - Separate code base means faster review/commit cycle
>>    - Less code churn in Heat
>> Cons:
>>    - A new service need to be installed/configured/launched
>>    - Need commitments from dedicated, experienced developers from very
>>      beginning
>>
>
> Anything that involves a kind of flag-day switchover like this
> (maintaining the implementation in two different places) will be very hard
> to land, and if by some miracle it does will likely cause a lot of
> user-breaking bugs.
>

Well we can use the environment to provide the two options for a cycle
(like the cloud watch lite) and the operator can switch when they feel
comfortable.
The reason I'd like to keep the door somewhat open to this is the huge
burden of work we will put on Qiming and his
team for option B (and the load on the core team). As you know this has
been thought of before and fizzled out, I don't want that to happen again.
If we can make this more manageable for the team doing this, then I think
that is a good thing. We could implement the guts of the AS in
a library and import it from both places (to prevent duplicate
implementations).


>
>  Option B: Bottom-Up Slow Growth
>> -------------------------------
>>
>> The roadmap is more conservative, with many (yes, many) incremental
>> patches to migrate things carefully.
>>
>>    1) Separate some of the autoscaling logic into libraries in Heat
>>    2) Augment heat-engine with new AS RPCs
>>    3) Switch AS related resource types to use the new RPCs
>>    4) Add new REST service that also talks to the same RPC
>>       (create new GIT repo, API endpoint and client lib...)
>>
>> Pros:
>>    - Less risk breaking user lands with each revision well tested
>>    - More smooth transition for users in terms of upgrades
>>
>>
I think this is only true up until "4)", at that point it's the same pain
as option A
(the operator needs a new REST endpoint, daemons to run, etc) - so delayed
pain.


> Cons:
>>    - A lot of churn within Heat code base, which means long review cycles
>>    - Still need commitments from cores to supervise the whole process
>>
>
> I vote for option B (surprise!), and I will sign up right now to as many
> nagging emails as you care to send when you need reviews if you will take
> on this work :)
>
>  There could be option C, D... but the two above are what we came up with
>> during the discussion.
>>
>
I'd suggest a combination between A and B.

1) Separate some of the autoscaling logic into libraries in Heat
2) Get the separated REST service in place and working (using the above
heat library)
3) Add an environment option to be able to switch Heat resources to use the
new REST service
4) after a cycle remove the internal support within Heat

(open to other suggestions tho') - I am not convinced of the usefulness of
the RPC step when that can be bypassed.

-Angus


>> Another important thing we talked about is about the open discussion on
>> this.  OpenStack Wiki seems a good place to document settled designs but
>> not for interactive discussions.  Probably we should leverage etherpad
>> and the mailinglist when moving forward.  Suggestions on this are also
>> welcomed.
>>
>
> +1
>
> cheers,
> Zane.
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141202/52132b58/attachment.html>