[openstack-dev] Fwd: [Senlin]Support more complicated scalingscenario

Jun Xu jun.xu.sz at 139.com
Mon Nov 23 09:12:28 UTC 2015


Thanks HU yanyan, I still has some questions about your answers, and I 
respond inline.

On 2015/11/23 10:43, Yanyan Hu wrote:
> Hi, Xu Jun, please find some answers inline. Thanks.
>
>
> 2015-11-20 16:06 GMT+08:00 xujun at cmss.chinamobile.com 
> <mailto:xujun at cmss.chinamobile.com> <xujun at cmss.chinamobile.com 
> <mailto:xujun at cmss.chinamobile.com>>:
>
>         Thanks yanyan!
>
>         Xu Jun is a contributor from CMCC. He asked a very interesting
>         question about cluster scaling support in Senlin. To make the
>         discussion more thorough, I just post the question and my
>         answer here.
>
>         The question from Jun is as following:
>
>         For an action, senlin will check all according polices, like
>         if a cluster attach two scaling-in polices,
>         the two scaling-in polices will be checked when doing a
>         scaling-action on this cluster. This is not same as
>         OS::Heat::ScalingPolicy in heat?
>         How should I use senlin for following cases?
>         1.  15% < cpu_util  < 30%, scaling_in 1 instance
>         2.   cpu_util < 15%, scaling_in 2 instances
>
>         This is a very interesting question and you're right about the
>         difference between Senlin ScalingPolicy and
>         OS::Heat::ScalingPolicy. In Heat, OS::Heat::ScalingPolicy is
>         actually not just a policy. It is a combination of a webhook
>         and a rule about how ASG respond to the webhook
>         triggering(resource signal). So you can define two different
>         OS::Heat::ScalingPolicy instances to make them deal with two
>         cases you described respectively.
>
>         But in Senlin, ScalingPolicy is a REAL policy, it only
>         describes how a Senlin cluster react to an action triggered by
>         Senlin webhook which is defined separately. The problem is
>         when an cluster action e.g. CLUSTER_SCALE_IN is triggered, all
>         policies attached to it will be checked in sequence based on
>         policies priority definition._So if you create two Senlin
>         ScalingPolicy and attach them to the same cluster, only one of
>         them will take effect actually._
>         _
>         _
>         # 1.  But in policy_check function, all the policies will be
>         checked in priority-based order for a CLUSTER_SCALING_IN
>         action if the cluster attached with SCALING multiple policies.
>                is this a bug?  or  what  is the significanceof prority).
>
> /Sorry, I didn't describe it clearly. I mean although both scaling 
> policies will be checked before CLUSTER_SCALING_IN action is executed, 
> count result from one ScalingPolicy will actually be overridden by the 
> result from another ScalingPolicy which has higher priority./

After debugging,  I found that former result is not overridden by 
another policy.
http://git.openstack.org/cgit/openstack/senlin/tree/senlin/engine/actions/base.py#n441

>
>             2. if  a cluster attached a scaling policy with event =
>         CLUSTER_SCALE_IN,  when aCLUSTER_SCALING_OUT action is
>         triggered,  the policy also will be checked,  is this reasonable?
>
> /         When a ScalingPolicy is defined, you can use 'event' 
> property to specify the action type you want the policy to take effect 
> on, like:
> http://git.openstack.org/cgit/openstack/senlin/tree/examples/policies/scaling_policy.yaml#n5
>
>          Although a ScalingPolicy will be checked for both 
> CLUSTER_SCALE_IN and CLUSTER_SCALE_OUT actions, the check routine will 
> return immediately if the action type is not what it is expecting.
> http://git.openstack.org/cgit/openstack/senlin/tree/senlin/policies/scaling_policy.py#n133/

Yes  it's not checked in pre_op,  but all ScalingPolicies still will be 
checked whether in cooldown.
http://git.openstack.org/cgit/openstack/senlin/tree/senlin/engine/actions/base.py#n431*
*
>
>
>
>         Currently, you can use the following way to support your use
>         case in Senlin:
>         1. Define two Senlin webhooks which target on the
>         CLUSTER_SCALE_OUT action of the cluster and specify the
>         'param' as {'count': 1} for webhook1 and {'count': 2 } for
>         webhook2;
>         1. Define two ceilometer/aodh alarms with the first one
>         matching case1 and second one matching case2. Then define
>         webhook1 url as alarm1's alarm-action and webhook2 url as
>         alarm2's alarm-action.
>
>         #
>         Your suggestion has a problem when I want different cooldown
>         for each ceilometer/aodh alarms, for following cases, how
>         should I do?
>         1.  15% < cpu_util  < 30%,  scaling_in 1 instance with 300s
>         cooldown time
>         2.   cpu_util < 15%, scaling_in 2 instances with 600s
>          cooldown time
>
> /     You can define the cooldown by specifying it when creating the 
> policy or attaching it to a cluster. The cooldown check logic will 
> prevent a policy taking effect if cooldown is still in progress.
> http://git.openstack.org/cgit/openstack/senlin/tree/senlin/engine/actions/base.py#n431
> /
>
>
>         For a senlin webhook, could we assign a policy which will be
>         checked ?
>
> /   User is not allowed to specify the policy when defining a webhook. 
> The webhook target is decided by target object(cluster or node) and 
> target action type./

     Yes we can define cooldown for each policy, but my meaning is that 
each cluster_scaling_in action is only checked by specified 
scaling_policy like OS::Heat::ScalingPolicy in heat.
     1) In heat, we could define two scaling_in actions(via define two 
OS::Heat::ScalingPolicy polices ), each scaling_in action is checked by 
one OS::Heat::ScalingPolicy, so each scaling_in action's cooldown is 
only checked in one OS::Heat::ScalingPolicy.
      2)But in senlin, each scaling_in action will be checked by all 
attached scaling_policies, so all scaling_polices' cooldown will be 
checked.How does senlin support different cooldown time for each 
scaling_in action?

>
>         Then each time alarm1 is triggered, cluster will be scaled out
>         with count 1 which means one new node will be created and
>         added to cluster. When alarm2 is triggered, cluster will be
>         scaled out with count 2 that two new nodes will be created and
>         added to cluster.
>
>         The question you asked is really interesting and we did
>         consider to support this kind of requirement using a 'complex'
>         ScalingPolicy which defined both trigger(alarm), webhook and
>         some rules for scaling. But after some discussion, we felt
>         that maybe we should let some high level service/enduser to
>         define this kind of 'POLICY' since it's more like a workflow
>         definition rather than a description of the rule cluster
>         scaling. So currently, we only provide atomic operation(e.g.
>         webhook, 'simple' ScalingPolicy) in Senlin while leaving the
>         work of combining these operations to support a use case to
>         enduser/high-level service.
>
>         Thanks a lot for throwing this interesting question and I do
>         agree that we should make more discussion about it to think
>         whether we need to adjust our policy design to support this
>         kind of scenario more smoothly.
>
>         --
>         Best regards,
>
>         Yanyan
>
>
>
>
> -- 
> Best regards,
>
> Yanyan
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 
Best regards,

Jun Xu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151123/cb147d4e/attachment.html>


More information about the OpenStack-dev mailing list