Open Stack

Ah,

Seems like a reasonable approach then :-)

I guess then heat is mainly doing top-level orchestration, and then mistral does the workflow middle-level, and taskflow is (hopefully) at the lowest-level??

Thanks Georgy!

From: Georgy Okrokvertskhov <gokrokvertskhov at mirantis.com<mailto:gokrokvertskhov at mirantis.com>>
Reply-To: OpenStack Development Mailing List <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Tuesday, October 22, 2013 4:53 PM
To: OpenStack Development Mailing List <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal

Hi Joshua,

>Sounds like taskflow could be that program (+1 from me, ha)?

>Mistral to me is a nice authenticated REST api + other goodies ontop of
>something that reliably executes workflows.

I would say that Mistral is a way to do this. My arguments are the following:
1. Mistral decouples code. Heat can use API calls to invoke a workflows execution instead of linking with taskflow library in the code. This is standard SOA approach which OpenStack uses a lot.
2. Mistral will expose DSL to define tasks while taskflow will require python code for task definition.

Mistral itself uses taskflow library to execute workflows but Mistral in addition does parsing and translation from DSL task definition to actual python code.

Heat can use taskflow for other purposes but workflows execution is not a good reason for that. Just because of nature of workflows for deployment, there is no knowledge about workflow until end user uploads it, so you can not use taskflow itself and code this workflow in python without preliminary knowledge about workflow.

If Heat uses just taskflow it should do all the work on workflow parsing and translation to a code that Heat team wants to avoid. At least this is my understanding.

Thanks
Georgy



On Tue, Oct 22, 2013 at 4:34 PM, Joshua Harlow <harlowja at yahoo-inc.com<mailto:harlowja at yahoo-inc.com>> wrote:
Sounds like taskflow could be that program (+1 from me, ha)?

Mistral to me is a nice authenticated REST api + other goodies ontop of
something that reliably executes workflows.

But then what I described is also the majority of what openstack does
(authenticated REST api + other goodies ontop of VM/volume/network/...
workflows).

On 10/22/13 3:28 PM, "Clint Byrum" <clint at fewbar.com<mailto:clint at fewbar.com>> wrote:

>Excerpts from Georgy Okrokvertskhov's message of 2013-10-22 13:32:40
>-0700:
>> Hi Thomas,
>>
>> I agree with you on semantics part. At the same time I see a potential
>> question which might appear - if semantics is limited by few states
>>visible
>> for Heat engine, then who actually does software orchestration?
>> Will it be reasonable then to have software orchestration as separate
>> subproject for Heat as a part of "Orchestration" OpenStack program? Heat
>> engine will then do dependency tracking and will use components as a
>> reference for software orchestration engine which will perform actual
>> deployment and high level software components coordination.
>>
>> This separated software orchestration engine may address all specific
>> requirements proposed by different teams in this thread without
>>affecting
>> existing Heat engine.
>>
>
>I'm not sure I know what "software orchestration" is, but I will take a
>stab at a succinct definition:
>
>Coordination of software configuration across multiple hosts.
>
>If that is what you mean, then I believe what you actually want is
>workflow. And for that, we have the Mistral project which was recently
>announced [1].
>
>Use that and you will simply need to define your desired workflow and
>feed it into Mistral using a Mistral Heat resource. We can create a
>nice bootstrapping resource for Heat instances that shims the mistral
>workflow execution agent into machines (or lets us use one already there
>via custom images).
>
>I can imagine it working something like this:
>
>resources:
>  mistral_workflow_handle:
>    type: OS::Mistral::WorkflowHandle
>  web_server:
>    type: OS::Nova::Server
>    components:
>      mistral_agent:
>        component_type: mistral
>        params:
>          workflow_: {ref: mistral_workflow_handle}
>  mysql_server:
>    type: OS::Nova::Server
>    components:
>      mistral_agent:
>        component_type: mistral
>        params:
>          workflow_handle: {ref: mistral_workflow_handle}
>  mistral_workflow:
>    type: OS::Mistral::Workflow
>    properties:
>      handle: {ref: mistral_workflow_handle}
>      workflow_reference: mysql_webapp_workflow
>      params:
>        mysql_server: {ref: mysql_server}
>        webserver: {ref: web_server}
>
>
>And then the workflow is just defined outside of the Heat template (ok
>I'm sure somebody will want to embed it, but I prefer stronger
>separation). Something like this gets uploaded as
>"mysql_webapp_workflow":
>
>[ 'step1': 'install_stuff',
>  'step2': 'wait(step1)',
>  'step3': 'allocate_sql_user(server=%mysql_server%)'
>  'step4': 'credentials=wait_and_read(step3)'
>  'step5': 'write_config_file(server=%webserver%)' ]
>
>Or maybe it is declared as a graph, or whatever, but it is not Heat's
>problem how to do workflows, it just feeds the necessary data from
>orchestration into the workflow engine. This also means you can use a
>non OpenStack workflow engine without any problems.
>
>I think after having talked about this, we should have workflow live in
>its own program.. we can always combine them if we want to, but having a
>clear line would mean keeping the interfaces clean.
>
>[1]
>http://lists.openstack.org/pipermail/openstack-dev/2013-October/016605.htm
>l
>
>_______________________________________________
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Georgy Okrokvertskhov
Technical Program Manager,
Cloud and Infrastructure Services,
Mirantis
http://www.mirantis.com<http://www.mirantis.com/>
Tel. +1 650 963 9828
Mob. +1 650 996 3284
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131023/f9c6b045/attachment.html>