[openstack-dev] [Mistral] really simple workflow for Heat configuration tasks

Renat Akhmerov rakhmerov at mirantis.com
Tue Nov 12 06:51:45 UTC 2013


Hey, thanks for bringing up these good points. At this point I may not clearly understand all your concerns so we need to keep discussing these things. See my comments below.


On 12 нояб. 2013 г., at 8:15, Angus Salkeld <asalkeld at redhat.com> wrote:

> Hi all
> 
> I think some of you were at the Software Config session at summit,
> but I'll link the ideas that were discussed:
> https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP
> https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config
> 
> To me the basics of it are:
> 1. we need an entity/resource to place the configuration (in Heat)
> 2. we need a resource to install the configuration
>   (basically a task in Mistral)
> 
> 
> A big issue to me is the conflict between heat's taskflow and the new
> external one. What I mean by conflict is that it will become tricky
> to manage two parallel taskflow instances in one stack.
> 
> This could be solved by:
> 1: totally using mistral (only use mistral workflow)

It will be possible only if we keep collaborating on this and figure out all the requirements for Mistral so that your use case is supported naturally.

> 2: use a very simple model of just asking mistral to run tasks (no
>   workflow) this allows us to use heat's workflow
>   but mistral's task runner.
> 

Yes, with the current Mistral design it is possible to create a workflow with just one task, I think it’s the same as you mean here. I believe we can even add some sort of shortcut api methods for working with individual tasks (just upload a single task and it will create a workflow if needed transparently, but it’s not really important for a user in this case)

> Given that mistral has no real implementation yet, "2" would seem
> reasonable to me. (I think Heat developers are open to "1" when
> Mistral is more mature.)
> 

Yes, however, the real Mistral power is going to be in task dependency resolution and figuring out the right execution order.

> How could we use Mistral for config installation?
> -------------------------------------------------
> 1. We have a resource type in Heat that creates tasks in a Mistral
>   workflow (manual workflow).
> 2. Heat pre-configures the server to have a Mistral worker
>   installed.
> 3. the Mistral worker pulls tasks from the workflow and passes them
>   to an agent that can run it. (the normal security issues jump up
>   here - giving access to the taskflow from a guest).
> 
> To do this we need an api that can add tasks to a workflow dynamically.
> like this:
> - create a simple workflow
> - create and run task A [run on server X]
> - create and run task B [run on server Y]
> - create and run task C [run on server X]
> 
> (note: the task is run and completes before the next is added if there
>       is a dependancy, if tasks can be run in parallel then we add multiple
>       tasks)
> 

We’re also planning to have dynamic workflows where new tasks can be spawned while a workflow is being executed. It may be helpful here. Some tricky questions arise here though like “definition of done” for a workflow, but we have some ideas on that. It’s not described and published anywhere at this point. I think it will be a part of the specifications coming out soon. If you have any suggestions here, please share your thoughts.

> The api could be something like:
> CRUD <mistral>/workflows/
> CRUD <mistral>/workflows/<wf>/tasks
> 
> 

Looks good.

> One thing that I am not sure of is how a server(worker) would know if a task
> was for it or not.
> - perhaps we have a capability property of the task that we can use
>  (capablitiy[server] = <server-id>) or actually specify the worker we
>  want.
> 

We’re planning to have tagging mechanism for tasks. So that every task has a set of tags associated with it and when a worker polls for available tasks it can specify desired tags to filter unsuitable tasks out. This feature is also related with the capability of dynamic task spawning since it will be possible to specify a dependency without having to know a task name itself but with just specifying tags. For example, I can configure that taskA requires all tasks tagged as “Deployment” and “PostDeployment” to be completed and it doesn’t matter what the tasks are.

By the way, it may be really helpful for the problem you addressed before. I mean switching to Mistral workflows completely (option 1) because tagging gives much more opportunities for building workflows, particularly in a half-static way I would say.

> I think this would be a good starting point for Mistral as it is a
> very simple but concrete starting point. Also if this is not done in
> Mistral we will have to add this in Heat (lets rather have it where
> it should be). This will also give us a chance to have confidence
> with Mistral before trying to do more complex workflows.
> 

Absolutely correct. We’re planning to implement Mistral basic capabilities asap (~ 1 month) and show them to everyone. It’s not easy to talk about something sometimes when we don’t see the actual code and functionality :) Looking at a working system and playing with it I’m sure will give us much more ideas on what we really want from this system, how to change the initial requirements and improve the design. Lots of people wouldn’t agree with me, but I have a strong opinion here, been there, seen it.

> If you (Heat and Mistral developers) are open to this we can discuss
> what needs to be done. I am willing to help with implementation.
> 

I think we agreed that we’re open. Any help would be appreciated.

> Thanks
> -Angus
> 
> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




More information about the OpenStack-dev mailing list