[openstack-dev] Fwd: [Senlin]Support more complicated scalingscenario
Jun Xu
jun.xu.sz at 139.com
Mon Nov 23 09:12:28 UTC 2015
Thanks HU yanyan, I still has some questions about your answers, and I
respond inline.
On 2015/11/23 10:43, Yanyan Hu wrote:
> Hi, Xu Jun, please find some answers inline. Thanks.
>
>
> 2015-11-20 16:06 GMT+08:00 xujun at cmss.chinamobile.com
> <mailto:xujun at cmss.chinamobile.com> <xujun at cmss.chinamobile.com
> <mailto:xujun at cmss.chinamobile.com>>:
>
> Thanks yanyan!
>
> Xu Jun is a contributor from CMCC. He asked a very interesting
> question about cluster scaling support in Senlin. To make the
> discussion more thorough, I just post the question and my
> answer here.
>
> The question from Jun is as following:
>
> For an action, senlin will check all according polices, like
> if a cluster attach two scaling-in polices,
> the two scaling-in polices will be checked when doing a
> scaling-action on this cluster. This is not same as
> OS::Heat::ScalingPolicy in heat?
> How should I use senlin for following cases?
> 1. 15% < cpu_util < 30%, scaling_in 1 instance
> 2. cpu_util < 15%, scaling_in 2 instances
>
> This is a very interesting question and you're right about the
> difference between Senlin ScalingPolicy and
> OS::Heat::ScalingPolicy. In Heat, OS::Heat::ScalingPolicy is
> actually not just a policy. It is a combination of a webhook
> and a rule about how ASG respond to the webhook
> triggering(resource signal). So you can define two different
> OS::Heat::ScalingPolicy instances to make them deal with two
> cases you described respectively.
>
> But in Senlin, ScalingPolicy is a REAL policy, it only
> describes how a Senlin cluster react to an action triggered by
> Senlin webhook which is defined separately. The problem is
> when an cluster action e.g. CLUSTER_SCALE_IN is triggered, all
> policies attached to it will be checked in sequence based on
> policies priority definition._So if you create two Senlin
> ScalingPolicy and attach them to the same cluster, only one of
> them will take effect actually._
> _
> _
> # 1. But in policy_check function, all the policies will be
> checked in priority-based order for a CLUSTER_SCALING_IN
> action if the cluster attached with SCALING multiple policies.
> is this a bug? or what is the significanceof prority).
>
> /Sorry, I didn't describe it clearly. I mean although both scaling
> policies will be checked before CLUSTER_SCALING_IN action is executed,
> count result from one ScalingPolicy will actually be overridden by the
> result from another ScalingPolicy which has higher priority./
After debugging, I found that former result is not overridden by
another policy.
http://git.openstack.org/cgit/openstack/senlin/tree/senlin/engine/actions/base.py#n441
>
> 2. if a cluster attached a scaling policy with event =
> CLUSTER_SCALE_IN, when aCLUSTER_SCALING_OUT action is
> triggered, the policy also will be checked, is this reasonable?
>
> / When a ScalingPolicy is defined, you can use 'event'
> property to specify the action type you want the policy to take effect
> on, like:
> http://git.openstack.org/cgit/openstack/senlin/tree/examples/policies/scaling_policy.yaml#n5
>
> Although a ScalingPolicy will be checked for both
> CLUSTER_SCALE_IN and CLUSTER_SCALE_OUT actions, the check routine will
> return immediately if the action type is not what it is expecting.
> http://git.openstack.org/cgit/openstack/senlin/tree/senlin/policies/scaling_policy.py#n133/
Yes it's not checked in pre_op, but all ScalingPolicies still will be
checked whether in cooldown.
http://git.openstack.org/cgit/openstack/senlin/tree/senlin/engine/actions/base.py#n431*
*
>
>
>
> Currently, you can use the following way to support your use
> case in Senlin:
> 1. Define two Senlin webhooks which target on the
> CLUSTER_SCALE_OUT action of the cluster and specify the
> 'param' as {'count': 1} for webhook1 and {'count': 2 } for
> webhook2;
> 1. Define two ceilometer/aodh alarms with the first one
> matching case1 and second one matching case2. Then define
> webhook1 url as alarm1's alarm-action and webhook2 url as
> alarm2's alarm-action.
>
> #
> Your suggestion has a problem when I want different cooldown
> for each ceilometer/aodh alarms, for following cases, how
> should I do?
> 1. 15% < cpu_util < 30%, scaling_in 1 instance with 300s
> cooldown time
> 2. cpu_util < 15%, scaling_in 2 instances with 600s
> cooldown time
>
> / You can define the cooldown by specifying it when creating the
> policy or attaching it to a cluster. The cooldown check logic will
> prevent a policy taking effect if cooldown is still in progress.
> http://git.openstack.org/cgit/openstack/senlin/tree/senlin/engine/actions/base.py#n431
> /
>
>
> For a senlin webhook, could we assign a policy which will be
> checked ?
>
> / User is not allowed to specify the policy when defining a webhook.
> The webhook target is decided by target object(cluster or node) and
> target action type./
Yes we can define cooldown for each policy, but my meaning is that
each cluster_scaling_in action is only checked by specified
scaling_policy like OS::Heat::ScalingPolicy in heat.
1) In heat, we could define two scaling_in actions(via define two
OS::Heat::ScalingPolicy polices ), each scaling_in action is checked by
one OS::Heat::ScalingPolicy, so each scaling_in action's cooldown is
only checked in one OS::Heat::ScalingPolicy.
2)But in senlin, each scaling_in action will be checked by all
attached scaling_policies, so all scaling_polices' cooldown will be
checked.How does senlin support different cooldown time for each
scaling_in action?
>
> Then each time alarm1 is triggered, cluster will be scaled out
> with count 1 which means one new node will be created and
> added to cluster. When alarm2 is triggered, cluster will be
> scaled out with count 2 that two new nodes will be created and
> added to cluster.
>
> The question you asked is really interesting and we did
> consider to support this kind of requirement using a 'complex'
> ScalingPolicy which defined both trigger(alarm), webhook and
> some rules for scaling. But after some discussion, we felt
> that maybe we should let some high level service/enduser to
> define this kind of 'POLICY' since it's more like a workflow
> definition rather than a description of the rule cluster
> scaling. So currently, we only provide atomic operation(e.g.
> webhook, 'simple' ScalingPolicy) in Senlin while leaving the
> work of combining these operations to support a use case to
> enduser/high-level service.
>
> Thanks a lot for throwing this interesting question and I do
> agree that we should make more discussion about it to think
> whether we need to adjust our policy design to support this
> kind of scenario more smoothly.
>
> --
> Best regards,
>
> Yanyan
>
>
>
>
> --
> Best regards,
>
> Yanyan
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Best regards,
Jun Xu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151123/cb147d4e/attachment.html>
More information about the OpenStack-dev
mailing list