Open Stack

Angus Salkeld <asalkeld at redhat.com> wrote on 12.11.2013 23:05:57:
> From: Angus Salkeld <asalkeld at redhat.com>
> To: openstack-dev at lists.openstack.org,
> Date: 12.11.2013 23:09
> Subject: Re: [openstack-dev] [Mistral] really simple workflow for
> Heat configuration tasks
>
> On 12/11/13 13:04 +0100, Thomas Spatzier wrote:
> >Hi Angus,
> >
> >that is an interesting idea. Since you mentioned the software config
> >proposal in the beginning as a related item, I guess you are trying to
> >solve some software config related issues with Mistral. So a few
questions,
> >looking at this purely from a software config perspect:
> >
> >Are you thinking about doing the infrastructure orchestration (VMs,
> >volumes, network etc) with Heat's current capabilities and then let the
> >complete software orchestration be handled by Mistral tasks? I.e.
bootstrap
> >the workers on each VM and have the definition of when which agent does
> >something defined in a flow?
>
> Well we either add an api to heat to do "install_config" or we use
> a service that is designed to do tasks. Clint convienced me quite
> easily that "install/apply_config" is just a task.
>
> >
> >If yes, is there a way for passing data around - e.g. output produced by
> >one software config step is input for another software config step?
> >
> >Again, if my above assumption is true, couldn't there be problems when
we
> >having two ways of doing orchestration, when the software layer thing
would
> >take the Heat engine out of some processing and take away some control?
Or
> >are you thinking about using Mistral as a general mechanism for task
> >execution in Heat, which would then probably resolve the conflict?
> >
> At this point we really do not need a "flow", just a task concept
> from Mistral. Prehaps ways of grouping them and targeting them
> for a particular server.
>
> I'd see the config_deployer resource posting a task to Mistral
> and we have an agent in the server that can consume tasks and
> pass them to sub-agents that understand the particular format.

Ok, makes sense to me. And I don't see a conflict with the software config
proposal, but this is one of the "implementation details" we said need to
be figured out :-)

>
> If we do this then Heat is in charge of the orchestration and
> there are not two workflows fighting for control. I do agree
> that there should just be one.
>
> I think once Mistral is more mature we can decide whether to pass
> full workflow control over to it, but for now the task functionality
> is all we need. (and a time based one would be neat too btw).
>
> -Angus
>
> >Regards,
> >Thomas
> >
> >Angus Salkeld <asalkeld at redhat.com> wrote on 12.11.2013 02:15:15:
> >> From: Angus Salkeld <asalkeld at redhat.com>
> >> To: openstack-dev at lists.openstack.org,
> >> Date: 12.11.2013 02:18
> >> Subject: [openstack-dev] [Mistral] really simple workflow for Heat
> >> configuration tasks
> >>
> >> Hi all
> >>
> >> I think some of you were at the Software Config session at summit,
> >> but I'll link the ideas that were discussed:
> >>
https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config-WIP
> >> https://wiki.openstack.org/wiki/Heat/Blueprints/hot-software-config
> >>
> >> To me the basics of it are:
> >> 1. we need an entity/resource to place the configuration (in Heat)
> >> 2. we need a resource to install the configuration
> >>     (basically a task in Mistral)
> >>
> >>
> >> A big issue to me is the conflict between heat's taskflow and the new
> >> external one. What I mean by conflict is that it will become tricky
> >> to manage two parallel taskflow instances in one stack.
> >>
> >> This could be solved by:
> >> 1: totally using mistral (only use mistral workflow)
> >> 2: use a very simple model of just asking mistral to run tasks (no
> >>     workflow) this allows us to use heat's workflow
> >>     but mistral's task runner.
> >>
> >> Given that mistral has no real implementation yet, "2" would seem
> >> reasonable to me. (I think Heat developers are open to "1" when
> >> Mistral is more mature.)
> >>
> >> How could we use Mistral for config installation?
> >> -------------------------------------------------
> >> 1. We have a resource type in Heat that creates tasks in a Mistral
> >>     workflow (manual workflow).
> >> 2. Heat pre-configures the server to have a Mistral worker
> >>     installed.
> >> 3. the Mistral worker pulls tasks from the workflow and passes them
> >>     to an agent that can run it. (the normal security issues jump up
> >>     here - giving access to the taskflow from a guest).
> >>
> >> To do this we need an api that can add tasks to a workflow
dynamically.
> >> like this:
> >> - create a simple workflow
> >> - create and run task A [run on server X]
> >> - create and run task B [run on server Y]
> >> - create and run task C [run on server X]
> >>
> >> (note: the task is run and completes before the next is added if there
> >>         is a dependancy, if tasks can be run in parallel then we add
> >multiple
> >>         tasks)
> >>
> >> The api could be something like:
> >> CRUD <mistral>/workflows/
> >> CRUD <mistral>/workflows/<wf>/tasks
> >>
> >>
> >> One thing that I am not sure of is how a server(worker) would know if
a
> >task
> >> was for it or not.
> >> - perhaps we have a capability property of the task that we can use
> >>    (capablitiy[server] = <server-id>) or actually specify the worker
we
> >>    want.
> >>
> >> I think this would be a good starting point for Mistral as it is a
> >> very simple but concrete starting point. Also if this is not done in
> >> Mistral we will have to add this in Heat (lets rather have it where
> >> it should be). This will also give us a chance to have confidence
> >> with Mistral before trying to do more complex workflows.
> >>
> >> If you (Heat and Mistral developers) are open to this we can discuss
> >> what needs to be done. I am willing to help with implementation.
> >>
> >> Thanks
> >> -Angus
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> >_______________________________________________
> >OpenStack-dev mailing list
> >OpenStack-dev at lists.openstack.org
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>