Open Stack

On 12/11/13 13:04 +0100, Thomas Spatzier wrote:
>Hi Angus,
>
>that is an interesting idea. Since you mentioned the software config
>proposal in the beginning as a related item, I guess you are trying to
>solve some software config related issues with Mistral. So a few questions,
>looking at this purely from a software config perspect:
>
>Are you thinking about doing the infrastructure orchestration (VMs,
>volumes, network etc) with Heat's current capabilities and then let the
>complete software orchestration be handled by Mistral tasks? I.e. bootstrap
>the workers on each VM and have the definition of when which agent does
>something defined in a flow?

Well we either add an api to heat to do "install_config" or we use
a service that is designed to do tasks. Clint convienced me quite
easily that "install/apply_config" is just a task.

>
>If yes, is there a way for passing data around - e.g. output produced by
>one software config step is input for another software config step?
>
>Again, if my above assumption is true, couldn't there be problems when we
>having two ways of doing orchestration, when the software layer thing would
>take the Heat engine out of some processing and take away some control? Or
>are you thinking about using Mistral as a general mechanism for task
>execution in Heat, which would then probably resolve the conflict?
>
At this point we really do not need a "flow", just a task concept
from Mistral. Prehaps ways of grouping them and targeting them
for a particular server.

I'd see the config_deployer resource posting a task to Mistral
and we have an agent in the server that can consume tasks and
pass them to sub-agents that understand the particular format.

If we do this then Heat is in charge of the orchestration and
there are not two workflows fighting for control. I do agree
that there should just be one.

I think once Mistral is more mature we can decide whether to pass
full workflow control over to it, but for now the task functionality
is all we need. (and a time based one would be neat too btw).

-Angus

>Regards,
>Thomas
>
>Angus Salkeld <asalkeld at redhat.com> wrote on 12.11.2013 02:15:15:
>> From: Angus Salkeld <asalkeld at redhat.com>
>> To: openstack-dev at lists.openstack.org,
>> Date: 12.11.2013 02:18
>> Subject: [openstack-dev] [Mistral] really simple workflow for Heat
>> configuration tasks
>>
>> Hi all
>>
>> I think some of you were at the Software Config session at summit,
>> but I'll link the ideas that were discussed:
>> https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP
>> https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config
>>
>> To me the basics of it are:
>> 1. we need an entity/resource to place the configuration (in Heat)
>> 2. we need a resource to install the configuration
>>     (basically a task in Mistral)
>>
>>
>> A big issue to me is the conflict between heat's taskflow and the new
>> external one. What I mean by conflict is that it will become tricky
>> to manage two parallel taskflow instances in one stack.
>>
>> This could be solved by:
>> 1: totally using mistral (only use mistral workflow)
>> 2: use a very simple model of just asking mistral to run tasks (no
>>     workflow) this allows us to use heat's workflow
>>     but mistral's task runner.
>>
>> Given that mistral has no real implementation yet, "2" would seem
>> reasonable to me. (I think Heat developers are open to "1" when
>> Mistral is more mature.)
>>
>> How could we use Mistral for config installation?
>> -------------------------------------------------
>> 1. We have a resource type in Heat that creates tasks in a Mistral
>>     workflow (manual workflow).
>> 2. Heat pre-configures the server to have a Mistral worker
>>     installed.
>> 3. the Mistral worker pulls tasks from the workflow and passes them
>>     to an agent that can run it. (the normal security issues jump up
>>     here - giving access to the taskflow from a guest).
>>
>> To do this we need an api that can add tasks to a workflow dynamically.
>> like this:
>> - create a simple workflow
>> - create and run task A [run on server X]
>> - create and run task B [run on server Y]
>> - create and run task C [run on server X]
>>
>> (note: the task is run and completes before the next is added if there
>>         is a dependancy, if tasks can be run in parallel then we add
>multiple
>>         tasks)
>>
>> The api could be something like:
>> CRUD <mistral>/workflows/
>> CRUD <mistral>/workflows/<wf>/tasks
>>
>>
>> One thing that I am not sure of is how a server(worker) would know if a
>task
>> was for it or not.
>> - perhaps we have a capability property of the task that we can use
>>    (capablitiy[server] = <server-id>) or actually specify the worker we
>>    want.
>>
>> I think this would be a good starting point for Mistral as it is a
>> very simple but concrete starting point. Also if this is not done in
>> Mistral we will have to add this in Heat (lets rather have it where
>> it should be). This will also give us a chance to have confidence
>> with Mistral before trying to do more complex workflows.
>>
>> If you (Heat and Mistral developers) are open to this we can discuss
>> what needs to be done. I am willing to help with implementation.
>>
>> Thanks
>> -Angus
>>
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>_______________________________________________
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev