<div dir="ltr">Hi,<div><br></div><div>><span style="font-family:Calibri,sans-serif;font-size:14.399999618530273px">I guess then heat is mainly doing top-level orchestration, and then mistral does the workflow middle-level, and >taskflow is (hopefully) at the lowest-level??</span></div>
<div><span style="font-family:Calibri,sans-serif;font-size:14.399999618530273px"><br></span></div><div><font face="Calibri, sans-serif"><span style="font-size:14.399999618530273px">You drove the right picture. I can not say who is top-level and who is low-level orchestration. This is all gear wheels which should work all together well to achieve the result while Het is probably the driving wheel among them who makes sure that everything is working.</span></font></div>
<div><font face="Calibri, sans-serif"><span style="font-size:14.399999618530273px"><br></span></font></div><div><font face="Calibri, sans-serif"><span style="font-size:14.399999618530273px">Thanks</span></font></div><div>
<font face="Calibri, sans-serif"><span style="font-size:14.399999618530273px">Georgy</span></font></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Oct 22, 2013 at 5:14 PM, Joshua Harlow <span dir="ltr"><<a href="mailto:harlowja@yahoo-inc.com" target="_blank">harlowja@yahoo-inc.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div>Ah,</div>
<div><br>
</div>
<div>Seems like a reasonable approach then :-)</div>
<div><br>
</div>
<div>I guess then heat is mainly doing top-level orchestration, and then mistral does the workflow middle-level, and taskflow is (hopefully) at the lowest-level??</div>
<div><br>
</div>
<div>Thanks Georgy!</div>
<div><br>
</div>
<span>
<div style="border-right:medium none;padding-right:0in;padding-left:0in;padding-top:3pt;text-align:left;font-size:11pt;border-bottom:medium none;font-family:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-left:medium none">
<span style="font-weight:bold">From: </span>Georgy Okrokvertskhov <<a href="mailto:gokrokvertskhov@mirantis.com" target="_blank">gokrokvertskhov@mirantis.com</a>><br>
<span style="font-weight:bold">Reply-To: </span>OpenStack Development Mailing List <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Date: </span>Tuesday, October 22, 2013 4:53 PM<div class="im"><br>
<span style="font-weight:bold">To: </span>OpenStack Development Mailing List <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
</div><div class="im"><span style="font-weight:bold">Subject: </span>Re: [openstack-dev] [Heat] HOT Software configuration proposal<br>
</div></div><div><div class="h5">
<div><br>
</div>
<div>
<div>
<div dir="ltr">Hi Joshua,
<div><br>
</div>
<div>><span style="font-size:12.800000190734863px;font-family:arial,sans-serif">Sounds like taskflow could be that program (+1 from me, ha)?</span></div>
<br style="font-family:arial,sans-serif;font-size:12.800000190734863px">
<span style="font-size:12.800000190734863px;font-family:arial,sans-serif">>Mistral to me is a nice authenticated REST api + other goodies ontop of</span><br style="font-family:arial,sans-serif;font-size:12.800000190734863px">
<span style="font-size:12.800000190734863px;font-family:arial,sans-serif">>something that reliably executes workflows.</span>
<div><span style="font-size:12.800000190734863px;font-family:arial,sans-serif"><br>
</span></div>
<div><span style="font-size:12.800000190734863px;font-family:arial,sans-serif">I would say that Mistral is a way to do this. My arguments are the following:</span></div>
<div><span style="font-size:12.800000190734863px;font-family:arial,sans-serif">1. Mistral decouples code. Heat can use API calls to invoke a workflows execution instead of linking with taskflow library in the code. This is standard SOA approach which
OpenStack uses a lot.</span></div>
<div><span style="font-size:12.800000190734863px;font-family:arial,sans-serif">2. Mistral will expose DSL to define tasks while taskflow will require python code for task definition.</span></div>
<div><span style="font-size:12.800000190734863px;font-family:arial,sans-serif"><br>
</span></div>
<div><span style="font-size:12.800000190734863px;font-family:arial,sans-serif">Mistral itself uses taskflow library to execute workflows but Mistral in addition does parsing and translation from DSL task definition to actual python code.</span></div>
<div><span style="font-size:12.800000190734863px;font-family:arial,sans-serif"><br>
</span></div>
<div><span style="font-size:12.800000190734863px;font-family:arial,sans-serif">Heat can use taskflow for other purposes but workflows execution is not a good reason for that. Just because of nature of workflows for deployment, there is no knowledge about
workflow until end user uploads it, so you can not use taskflow itself and code this workflow in python without preliminary knowledge about workflow</span><span style="font-size:12.800000190734863px;font-family:arial,sans-serif">. </span></div>
<div><span style="font-size:12.800000190734863px;font-family:arial,sans-serif"><br>
</span></div>
<div><span style="font-size:12.800000190734863px;font-family:arial,sans-serif">If Heat uses just taskflow it should do all the work on workflow parsing and translation to a code that Heat team wants to avoid. At least this is my understanding.</span></div>
<div><span style="font-size:12.800000190734863px;font-family:arial,sans-serif"><br>
</span></div>
<div><span style="font-size:12.800000190734863px;font-family:arial,sans-serif">Thanks</span></div>
<div><span style="font-size:12.800000190734863px;font-family:arial,sans-serif">Georgy</span></div>
<div><span style="font-size:12.800000190734863px;font-family:arial,sans-serif"><br>
</span></div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Tue, Oct 22, 2013 at 4:34 PM, Joshua Harlow <span dir="ltr">
<<a href="mailto:harlowja@yahoo-inc.com" target="_blank">harlowja@yahoo-inc.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Sounds like taskflow could be that program (+1 from me, ha)?<br>
<br>
Mistral to me is a nice authenticated REST api + other goodies ontop of<br>
something that reliably executes workflows.<br>
<br>
But then what I described is also the majority of what openstack does<br>
(authenticated REST api + other goodies ontop of VM/volume/network/...<br>
workflows).<br>
<div>
<div><br>
On 10/22/13 3:28 PM, "Clint Byrum" <<a href="mailto:clint@fewbar.com" target="_blank">clint@fewbar.com</a>> wrote:<br>
<br>
>Excerpts from Georgy Okrokvertskhov's message of 2013-10-22 13:32:40<br>
>-0700:<br>
>> Hi Thomas,<br>
>><br>
>> I agree with you on semantics part. At the same time I see a potential<br>
>> question which might appear - if semantics is limited by few states<br>
>>visible<br>
>> for Heat engine, then who actually does software orchestration?<br>
>> Will it be reasonable then to have software orchestration as separate<br>
>> subproject for Heat as a part of "Orchestration" OpenStack program? Heat<br>
>> engine will then do dependency tracking and will use components as a<br>
>> reference for software orchestration engine which will perform actual<br>
>> deployment and high level software components coordination.<br>
>><br>
>> This separated software orchestration engine may address all specific<br>
>> requirements proposed by different teams in this thread without<br>
>>affecting<br>
>> existing Heat engine.<br>
>><br>
><br>
>I'm not sure I know what "software orchestration" is, but I will take a<br>
>stab at a succinct definition:<br>
><br>
>Coordination of software configuration across multiple hosts.<br>
><br>
>If that is what you mean, then I believe what you actually want is<br>
>workflow. And for that, we have the Mistral project which was recently<br>
>announced [1].<br>
><br>
>Use that and you will simply need to define your desired workflow and<br>
>feed it into Mistral using a Mistral Heat resource. We can create a<br>
>nice bootstrapping resource for Heat instances that shims the mistral<br>
>workflow execution agent into machines (or lets us use one already there<br>
>via custom images).<br>
><br>
>I can imagine it working something like this:<br>
><br>
>resources:<br>
> mistral_workflow_handle:<br>
> type: OS::Mistral::WorkflowHandle<br>
> web_server:<br>
> type: OS::Nova::Server<br>
> components:<br>
> mistral_agent:<br>
> component_type: mistral<br>
> params:<br>
> workflow_: {ref: mistral_workflow_handle}<br>
> mysql_server:<br>
> type: OS::Nova::Server<br>
> components:<br>
> mistral_agent:<br>
> component_type: mistral<br>
> params:<br>
> workflow_handle: {ref: mistral_workflow_handle}<br>
> mistral_workflow:<br>
> type: OS::Mistral::Workflow<br>
> properties:<br>
> handle: {ref: mistral_workflow_handle}<br>
> workflow_reference: mysql_webapp_workflow<br>
> params:<br>
> mysql_server: {ref: mysql_server}<br>
> webserver: {ref: web_server}<br>
><br>
><br>
>And then the workflow is just defined outside of the Heat template (ok<br>
>I'm sure somebody will want to embed it, but I prefer stronger<br>
>separation). Something like this gets uploaded as<br>
>"mysql_webapp_workflow":<br>
><br>
>[ 'step1': 'install_stuff',<br>
> 'step2': 'wait(step1)',<br>
> 'step3': 'allocate_sql_user(server=%mysql_server%)'<br>
> 'step4': 'credentials=wait_and_read(step3)'<br>
> 'step5': 'write_config_file(server=%webserver%)' ]<br>
><br>
>Or maybe it is declared as a graph, or whatever, but it is not Heat's<br>
>problem how to do workflows, it just feeds the necessary data from<br>
>orchestration into the workflow engine. This also means you can use a<br>
>non OpenStack workflow engine without any problems.<br>
><br>
>I think after having talked about this, we should have workflow live in<br>
>its own program.. we can always combine them if we want to, but having a<br>
>clear line would mean keeping the interfaces clean.<br>
><br>
>[1]<br>
><a href="http://lists.openstack.org/pipermail/openstack-dev/2013-October/016605.htm" target="_blank">http://lists.openstack.org/pipermail/openstack-dev/2013-October/016605.htm</a><br>
>l<br>
><br>
>_______________________________________________<br>
>OpenStack-dev mailing list<br>
><a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
Georgy Okrokvertskhov<br>
Technical Program Manager,<br>
Cloud and Infrastructure Services,<br>
Mirantis<br>
<a href="http://www.mirantis.com/" target="_blank">http://www.mirantis.com</a><br>
Tel. <a href="tel:%2B1%20650%20963%209828" value="+16509639828" target="_blank">+1 650 963 9828</a><br>
Mob. <a href="tel:%2B1%20650%20996%203284" value="+16509963284" target="_blank">+1 650 996 3284</a><br>
</div>
</div>
</div>
</div></div></span>
</div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>Georgy Okrokvertskhov<br>
Technical Program Manager,<br>Cloud and Infrastructure Services,<br>
Mirantis<br>
<a href="http://www.mirantis.com/" target="_blank">http://www.mirantis.com</a><br>
Tel. +1 650 963 9828<br>
Mob. +1 650 996 3284<br>
</div>