Open Stack

On 26/09/18 10:27 PM, Qiming Teng wrote:
> Hi,
> 
> Due to many reasons, I cannot join you on this event, but I do like to
> leave some comments here for references.
> 
> On Tue, Sep 18, 2018 at 11:27:29AM +0800, Rico Lin wrote:
>> *TL;DR*
>> *How about a forum in Berlin for discussing autoscaling integration (as a
>> long-term goal) in OpenStack?*
> 
> First of all, there is nothing called "auto-scaling" in my mind and
> "auto" is most of the time a scary word to users. It means the service
> or tool is hiding some details from the users when it is doing something
> without human intervention. There are cases where this can be useful,
> there are also many other cases the service or tool is messing up things
> to a state difficult to recover from. What matters most is the usage
> scenarios we support. I don't think users care that much how project
> teams are organized.

Yeah, I mostly agree with you, and in fact I often use the term 'scaling 
group' to encompass all of the different types of groups in Heat. Our 
job is to provide an API that is legible to external tools to increase 
and decrease the size of the group. The 'auto' part is created by 
connecting it with other services, whether they be OpenStack services 
like Aodh or Monasca, monitoring services provided by the user 
themselves, or just manual invocation.

(BTW people from the HA-clustering world have a _very_ negative reaction 
to Senlin's use of the term 'cluster'... there is no perfect terminology.)

>> Hi all, as we start to discuss how can we join develop from Heat and Senlin
>> as we originally planned when we decided to fork Senlin from Heat long time
>> ago.
>>
>> IMO the biggest issues we got now are we got users using autoscaling in
>> both services, appears there is a lot of duplicated effort, and some great
>> enhancement didn't exist in another service.
>> As a long-term goal (from the beginning), we should try to join development
>> to sync functionality, and move users to use Senlin for autoscaling. So we
>> should start to review this goal, or at least we should try to discuss how
>> can we help users without break or enforce anything.
> 
> The original plan, iirc, was to make sure Senlin resources are supported
> in Heat,

This happened.

> and we will gradually fade out the existing 'AutoScalingGroup'
> and related resource types in Heat.

That's almost impossible to do without breaking existing users.

> I have no clue since when Heat is
> interested in "auto-scaling" again.

It's something that Rico and I have been discussing - it turns out that 
Heat still has a *lot* of users running very important stuff on Heat 
scaling group code which, as you know, is burdened by a lot of technical 
debt.

>> What will be great if we can build common library cross projects, and use
>> that common library in both projects, make sure we have all improvement
>> implemented in that library, finally to use Senlin from that from that
>> library call in Heat autoscaling group. And in long-term, we gonna let all
>> user use more general way instead of multiple ways but generate huge
>> confusing for users.
> 
> The so called "auto-scaling" is always a solution, built by
> orchestrating many moving parts across the infrastructure. In some
> cases, you may have to install agents into VMs for workload metering.

Totally agree, but...

> I
> am not convinced this can be done using a library approach.

Clearly there are _some_ parts that could in principle be shared. (I 
added some comments to the etherpad to clarify what I think Rico was 
referring to.)

It seems to me that there's value in discussing it together rather than 
just working completely independently, even if the outcome of that 
discussion is that

>> *As an action, I propose we have a forum in Berlin and sync up all effort
>> from both teams to plan for idea scenario design. The forum submission [1]
>> ended at 9/26.*
>> Also would benefit from both teams to start to think about how they can
>> modulize those functionalities for easier integration in the future.
>>
>>  From some Heat PTG sessions, we keep bring out ideas on how can we improve
>> current solutions for Autoscaling. We should start to talk about will it
>> make sense if we combine all group resources into one, and inherit from it
>> for other resources (ideally gonna deprecate rest resource types). Like we
>> can do Batch create/delete in Resource Group, but not in ASG. We definitely
>> got some unsynchronized works inner Heat, and cross Heat and Senlin.
> 
> Totally agree with you on this. We should strive to minimize the
> technologies users have to master when they have a need.

+1 - to expand on Rico's example, we have at least 3 completely separate 
implementations of batching, each supporting different actions:

Heat AutoscalingGroup: updates only
Heat ResourceGroup: create or update
Senlin Batch Policy: updates only

and users are asking for batch delete as well. This is clearly an area 
where technical debt from duplicate implementations is making it hard to 
deliver value to users.

cheers,
Zane.