<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Aug 2, 2015, at 7:40 PM, 王华 <<a href="mailto:wanghua.humble@gmail.com" class="">wanghua.humble@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">Hi all,
<div class=""><br class="">
</div>
<div class="">
<p id="yui_3_10_3_1_1438566042119_450" style="margin:0px 0px 1.2em;padding:0px;width:auto;max-width:60em;color:rgb(51,51,51);font-family:sans-serif;font-size:12px;line-height:18px" class="">
As discussed in the Vancouver Summit, we are going to drop the bay lock implementation. Instead, each conductor will call Heat concurrently and rely on heat for concurrency control. However, I think we need an approach for state convergence from heat to magnum.
Either periodic task [1] or heat notification [2] looks like a candidate.</p>
<p id="yui_3_10_3_1_1438566042119_457" style="margin:0px 0px 1.2em;padding:0px;width:auto;max-width:60em;color:rgb(51,51,51);font-family:sans-serif;font-size:12px;line-height:18px" class="">
[1] <a rel="nofollow" href="https://blueprints.launchpad.net/magnum/+spec/add-periodic-task" style="max-width:60em;color:rgb(0,51,170);text-decoration:none" class="">https://blueprints.launchpad.net/magnum/+spec/add-periodic-task</a><br style="max-width:60em" class="">
[2] <a rel="nofollow" href="http://lists.openstack.org/pipermail/openstack-dev/2015-March/058898.html" style="max-width:60em;color:rgb(0,51,170);text-decoration:none" class="">http://lists.openstack.org/pipermail/openstack-dev/2015-March/058898.html</a><br style="max-width:60em" class="">
--hongbin</p>
<p id="yui_3_10_3_1_1438566042119_457" style="margin:0px 0px 1.2em;padding:0px;width:auto;max-width:60em;color:rgb(51,51,51);font-family:sans-serif;font-size:12px;line-height:18px" class="">
If we use periodic task to sync state from heat to magnum, I think we should make periodic task a independent process and magnum-conductor only operates heat stack.</p>
<p id="yui_3_10_3_1_1438566042119_457" style="margin:0px 0px 1.2em;padding:0px;width:auto;max-width:60em;color:rgb(51,51,51);font-family:sans-serif;font-size:12px;line-height:18px" class="">
How to make periodic task high available？</p>
<p id="yui_3_10_3_1_1438566042119_457" style="margin:0px 0px 1.2em;padding:0px;width:auto;max-width:60em;color:rgb(51,51,51);font-family:sans-serif;font-size:12px;line-height:18px" class="">
1.We can run several periodic tasks.</p>
<p id="yui_3_10_3_1_1438566042119_457" style="margin:0px 0px 1.2em;padding:0px;width:auto;max-width:60em;color:rgb(51,51,51);font-family:sans-serif;font-size:12px;line-height:18px" class="">
2.Or we can use a leader selection mechanism to make only a periodic task running</p>
<p id="yui_3_10_3_1_1438566042119_457" style="margin:0px 0px 1.2em;padding:0px;width:auto;max-width:60em;color:rgb(51,51,51);font-family:sans-serif;font-size:12px;line-height:18px" class="">
and other periodic tasks waiting.</p>
<p id="yui_3_10_3_1_1438566042119_457" style="margin:0px 0px 1.2em;padding:0px;width:auto;max-width:60em;color:rgb(51,51,51);font-family:sans-serif;font-size:12px;line-height:18px" class="">
Shall we make periodic task a independent process? How to make periodic task high available? </p>
</div>
</div>
</div>
</blockquote>
</div>
<div>Good question. The traditional solution for handling this in a distributed system is to hold a leader election. The elected leader is responsible for dispatching the job to a queue that one available worker will pick up and run. However, that may not actually
be needed in our case. Consider the question: </div>
<div><br class="">
</div>
<div>What harm will come if all master nodes in the cluster perform the same periodic task?</div>
<div><br class="">
</div>
<div>One drawback is that more resources will be consumed than necessary (efficiency). As long as updates to the Magnum database are transactional, having concurrent updates to the same bay is actually not something we expect would result in data corruption.
Worst case the same update would be processed multiple times. The advantage of using this approach is that we would not need to implement any form of leader selection. This would keep our implementation simpler, and less error prone.</div>
<div><br class="">
</div>
<div>We could still supervise each periodic task process so if they end up crashing on each node, they would be restarted. This is simple to do from a patent process that calls os.wait() on the child task.</div>
<div><br class="">
</div>
<div>Thoughts?</div>
<div><br class="">
</div>
<div>Adrian</div>
</body>
</html>