<html><head><meta http-equiv="content-type" content="text/html; charset=UTF-8"><style>body { line-height: 1.5; }blockquote { margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em; }div.foxdiv20151123140459841049 { }body { font-size: 10.5pt; font-family: 'Microsoft YaHei UI'; color: rgb(0, 0, 0); line-height: 1.5; }</style></head><body>
<div><span></span></div><blockquote style="margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em;"><div class="FoxDiv20151123140459841049"><div dir="ltr"><div class="gmail_quote"><div dir="ltr">Thanks HU yanyan, I still has some question about your answers, and I respond inline.<br><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="">2015-11-20 16:06 GMT+08:00 <a href="mailto:xujun@cmss.chinamobile.com" target="_blank">xujun@cmss.chinamobile.com</a> <span dir="ltr"><<a href="mailto:xujun@cmss.chinamobile.com" target="_blank">xujun@cmss.chinamobile.com</a>></span>:<br><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div>
<div><span></span></div><blockquote style="margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em;"><div><div dir="ltr"><div><div><div><font color="#ff0000">Thanks yanyan!</font></div><span><div><br></div>Xu Jun is a contributor from CMCC. He asked a very interesting question about cluster scaling support in Senlin. To make the discussion more thorough, I just post the question and my answer here.<br><br></span></div><span>The question from Jun is as following:<br><br>For an action, senlin will check all according polices, like if a cluster attach two scaling-in polices, <br>the two scaling-in polices will be checked when doing a scaling-action on this cluster. This is not same as OS::Heat::ScalingPolicy in heat？<br>How should I use senlin for following cases？<br>1. 15% < cpu_util < 30%, scaling_in 1 instance<br>2. cpu_util < 15%, scaling_in 2 instances<br><br>This is a very interesting question and you're right about the difference between Senlin ScalingPolicy and OS::Heat::ScalingPolicy. In Heat, OS::Heat::ScalingPolicy is actually not just a policy. It is a combination of a webhook and a rule about how ASG respond to the webhook triggering(resource signal). So you can define two different OS::Heat::ScalingPolicy instances to make them deal with two cases you described respectively.<br><br>But in Senlin, ScalingPolicy is a REAL policy, it only describes how a Senlin cluster react to an action triggered by Senlin webhook which is defined separately. The problem is when an cluster action e.g. CLUSTER_SCALE_IN is triggered, all policies attached to it will be checked in sequence based on policies priority definition.<strike> </strike><u><font color="#ff9900">So if you create two Senlin ScalingPolicy and attach them to the same cluster, only one of them will take effect actually.</font></u></span></div><div><u><font color="#ff9900"><br></font></u></div><div><font color="#0000ff"># 1. But in policy_check function, all the policies will be checked in priority-based order for a CLUSTER_SCALING_IN action if the cluster attached with SCALING multiple policies. </font></div><div><font color="#0000ff"> is this a bug? or what <span style="font-size:10.5pt;line-height:1.5;background-color:window"> is the </span><span style="font-family:"";font-size:10.5pt;line-height:1.5;background-color:window">significance</span><font style="font-size:10.5pt;line-height:1.5;background-color:window"><span style="font-size:10.5pt;line-height:1.5;background-color:window"> of prority</span></font></font><span style="font-size:10.5pt;line-height:1.5;background-color:window;color:rgb(0,0,255)">). </span></div></div></div></blockquote></div></blockquote></span><div> <span style="color:rgb(255,0,0)"><span style="background-color:rgb(255,255,255)"> <i>Sorry, I didn't describe it clearly. I mean although both scaling policies will be checked before CLUSTER_SCALING_IN action is executed, count result from one ScalingPolicy will actually be overridden by the result from another ScalingPolicy which has higher priority. </i></span></span></div><div><font color="#0000ff" face="\Microsoft YaHei UI\"> </font></div><div><font face="\Microsoft YaHei UI\" color="#008000"><b>After debug it , I found thart former result is not overridden by another policy.</b></font></div><div><font face="\Microsoft YaHei UI\" color="#008000"><span style="line-height: 1.5;"><b>http://git.openstack.org/cgit/openstack/senlin/tree/senlin/engine/actions/base.py#n441</b></span></font></div><div><font color="#ff0000"><i> <br></i></font> <br></div><span class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><blockquote style="margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em;"><div><div dir="ltr"><div><font color="#0000ff"> 2. if a cluster attached a scaling policy with event = <span style="font-family:"";font-size:10.5pt;line-height:1.5;background-color:window">CLUSTER_SCALE_IN, when a</span><font style="font-size:10.5pt;line-height:1.5;background-color:window"> </font><span style="font-size:10.5pt;line-height:1.5;background-color:window">CLUSTER_SCALING_OUT </span><span style="font-size:10.5pt;line-height:1.5;background-color:window">action is triggered, the policy also will be checked, is this reasonable?</span></font></div></div></div></blockquote></div></blockquote></span><div><span style="color:rgb(255,0,0)"><i> When a ScalingPolicy is defined, you can use 'event' property to specify the action type you want the policy to take effect on, like:<br> <a href="http://git.openstack.org/cgit/openstack/senlin/tree/examples/policies/scaling_policy.yaml#n5" target="_blank" class="">http://git.openstack.org/cgit/openstack/senlin/tree/examples/policies/scaling_policy.yaml#n5</a></i></span></div><div><span style="color:rgb(255,0,0)"><i><br> Although a ScalingPolicy will be checked for both CLUSTER_SCALE_IN and CLUSTER_SCALE_OUT actions, the check routine will return immediately if the action type is not what it is expecting. <br> <a href="http://git.openstack.org/cgit/openstack/senlin/tree/senlin/policies/scaling_policy.py#n133" target="_blank" class="">http://git.openstack.org/cgit/openstack/senlin/tree/senlin/policies/scaling_policy.py#n133</a></i></span><br></div><div><span style="color:rgb(255,0,0)"><br></span></div><div><b style="color: rgb(0, 128, 0);">Yes in pre_op， it‘s not checked, but all ScalingPolicies still will check whether in cooldown.</b></div><div><div><font color="#008000" face="\Microsoft YaHei UI\"><b><span microsoft="" yahei="" ui'";="" font-size:="" 14px;="" color:="" rgb(0,="" 128,="" 0);="" background-color:="" rgba(0,="" 0,="" font-weight:="" bold;="" font-style:="" normal;text-decoration:="" none;'="">http://git.openstack.org/cgit/openstack/senlin/tree/senlin/engine/actions/base.py#n431</span><span style="line-height: 1.5;"> </span></b></font></div></div><span class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><blockquote style="margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em;"><div><div dir="ltr"><span><div><br><br>Currently, you can use the following way to support your use case in Senlin:<br>1. Define two Senlin webhooks which target on the CLUSTER_SCALE_OUT action of the cluster and specify the 'param' as {'count': 1} for webhook1 and {'count': 2 } for webhook2;<br>1. Define two ceilometer/aodh alarms with the first one matching case1 and second one matching case2. Then define webhook1 url as alarm1's alarm-action and webhook2 url as alarm2's alarm-action.</div><div><br></div></span><div><font color="#0000ff"># </font></div><div><font color="#0000ff">Your suggestion has a problem when I want different cooldown for each <span style="font-size:10.5pt;line-height:1.5;background-color:window">ceilometer/aodh </span><span style="font-size:10.5pt;line-height:1.5;background-color:window">alarms, for following cases, how should I do?</span></font></div><div><font color="#0000ff">1. 15% < cpu_util < 30%, scaling_in 1 instance with 300s cooldown time<br>2. cpu_util < 15%, scaling_in 2 instances with 600s cooldown time</font></div></div></div></blockquote></div></blockquote></span><div><span style="color:rgb(255,0,0)"><i> You can define the cooldown by specifying it when creating the policy or attaching it to a cluster. The cooldown check logic will prevent a policy taking effect if cooldown is still in progress.<br> <a href="http://git.openstack.org/cgit/openstack/senlin/tree/senlin/engine/actions/base.py#n431" target="_blank">http://git.openstack.org/cgit/openstack/senlin/tree/senlin/engine/actions/base.py#n431</a><br></i></span> </div><div><span style="color:rgb(255,0,0)"><br></span></div><div><font color="#008000" face="\Microsoft YaHei UI\"><b>Yes we can define cooldown for each policy, but I want each cluster_scaling_in action is only checked by one scaling_policy like </b></font><span style="line-height: 1.5;"><font color="#008000" face="\Microsoft YaHei UI\"><b>OS::Heat::ScalingPolicy?</b></font></span></div><div><font color="#008000"><b>In heat, we could define two scaling_in actions(via define two </b></font><font color="#008000" style="font-size: 10.5pt; line-height: 1.5; background-color: window;"><b> </b></font><b style="font-size: 10.5pt; line-height: 1.5; background-color: window; color: rgb(0, 128, 0);">OS::Heat::ScalingPolicy polices</b><font color="#008000" style="font-size: 10.5pt; line-height: 1.5; background-color: window;"><b> ), each scaling_in action is checked by one </b></font><b style="font-size: 10.5pt; line-height: 1.5; background-color: window; color: rgb(0, 128, 0);">OS::Heat::ScalingPolicy, so each scaling_in action's cooldown is only checked in one </b><b style="font-size: 10.5pt; line-height: 1.5; background-color: window; color: rgb(0, 128, 0);">OS::Heat::ScalingPolicy.</b></div><div><font color="#008000"><b>but in senlin, each scaling_in action will be checked by all attached scaling_policies, so all scaling_polices' cooldown will be checked.</b></font></div><div><font color="#008000"><b><br></b></font></div><div><span style="line-height: 1.5;"><font color="#008000" face="\Microsoft YaHei UI\"><b><br></b></font></span></div><span class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><blockquote style="margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em;"><div><div dir="ltr"><div><font color="#0000ff"><br></font></div><div><font color="#0000ff">For a senlin webhook, could we assign a policy which will be checked ?</font></div></div></div></blockquote></div></blockquote></span><div> <i><span style="color:rgb(255,0,0)"> User is not allowed to specify the policy when defining a webhook. The webhook target is decided by target object(cluster or node) and target action type.</span></i><br></div><span class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div><blockquote style="margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em;"><div><div dir="ltr"><span><div><br>Then each time alarm1 is triggered, cluster will be scaled out with count 1 which means one new node will be created and added to cluster. When alarm2 is triggered, cluster will be scaled out with count 2 that two new nodes will be created and added to cluster.<br><br>The question you asked is really interesting and we did consider to support this kind of requirement using a 'complex' ScalingPolicy which defined both trigger(alarm), webhook and some rules for scaling. But after some discussion, we felt that maybe we should let some high level service/enduser to define this kind of 'POLICY' since it's more like a workflow definition rather than a description of the rule cluster scaling. So currently, we only provide atomic operation(e.g. webhook, 'simple' ScalingPolicy) in Senlin while leaving the work of combining these operations to support a use case to enduser/high-level service.<br><br>Thanks a lot for throwing this interesting question and I do agree that we should make more discussion about it to think whether we need to adjust our policy design to support this kind of scenario more smoothly.<br><br>--</div><div><div><div><div><div><div><div>Best regards,<div><br></div><div>Yanyan</div></div>
</div></div></div></div></div></div></span></div>
</div></blockquote>
</div></blockquote></span></div><span class="HOEnZb"><font color="#888888"><br><br clear="all"></font></span><br></div></div></div>-- <br><div class="gmail_signature">Best regards,<div><br></div><div>Yanyan</div></div>
</div>
</div></blockquote>
</body></html>