Open Stack

Clint Byrum <clint at fewbar.com> wrote on 23.10.2013 00:28:17:
> From: Clint Byrum <clint at fewbar.com>
> To: openstack-dev <openstack-dev at lists.openstack.org>,
> Date: 23.10.2013 00:30
> Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal
>
> Excerpts from Georgy Okrokvertskhov's message of 2013-10-22 13:32:40
-0700:
> > Hi Thomas,
> >
> > I agree with you on semantics part. At the same time I see a potential
> > question which might appear - if semantics is limited by few states
visible
> > for Heat engine, then who actually does software orchestration?
> > Will it be reasonable then to have software orchestration as separate
> > subproject for Heat as a part of "Orchestration" OpenStack program?
Heat
> > engine will then do dependency tracking and will use components as a
> > reference for software orchestration engine which will perform actual
> > deployment and high level software components coordination.
> >
> > This separated software orchestration engine may address all specific
> > requirements proposed by different teams in this thread without
affecting
> > existing Heat engine.
> >
>
> I'm not sure I know what "software orchestration" is, but I will take a
> stab at a succinct definition:
>
> Coordination of software configuration across multiple hosts.
>
> If that is what you mean, then I believe what you actually want is
> workflow. And for that, we have the Mistral project which was recently
> announced [1].

My view of "software orchestration" in a sense of what Heat should be able
to do is being able to bring up software installations (e.g. web server, a
DBMS, a custom application) on-top of a bare compute resource by invoking a
software config tool (e.g. Chef, Puppet ...) at the right point in time and
let that tool do the actual work.
Invoke does not necessarily mean to call an API of such a tool, but rather
make sure it is bootstrapped and maybe gets a "go" signal to start.
"software orchestration" could then further mean to give CM tools across
hosts the "go" signal when the config on one host has completed. This be
enabled by the signaling enhancements Steve Baker mentioned in one of his
recent mails.

For such kind of stuff, I think we could live without workflows but do it
purely declaratively. Of course, a workflow could be the underlying
mechanism, but I would not want to express this in a template. If users
have very complex problems to solve and cannot live with the simple
software orchestration I outlined, then still a workflow could be used for
everying on-top of the OS.

Anyway, that was just my view and others most probably have different views
again, so seems like we really have to sort out terminology :-)

>
> Use that and you will simply need to define your desired workflow and
> feed it into Mistral using a Mistral Heat resource. We can create a
> nice bootstrapping resource for Heat instances that shims the mistral
> workflow execution agent into machines (or lets us use one already there
> via custom images).
>
> I can imagine it working something like this:
>
> resources:
>   mistral_workflow_handle:
>     type: OS::Mistral::WorkflowHandle
>   web_server:
>     type: OS::Nova::Server
>     components:
>       mistral_agent:
>         component_type: mistral
>         params:
>           workflow_: {ref: mistral_workflow_handle}
>   mysql_server:
>     type: OS::Nova::Server
>     components:
>       mistral_agent:
>         component_type: mistral
>         params:
>           workflow_handle: {ref: mistral_workflow_handle}
>   mistral_workflow:
>     type: OS::Mistral::Workflow
>     properties:
>       handle: {ref: mistral_workflow_handle}
>       workflow_reference: mysql_webapp_workflow
>       params:
>         mysql_server: {ref: mysql_server}
>         webserver: {ref: web_server}
>

While I can imagine that this works, I think for a big percentage of use
cases I would be nice to avoid this inter-weaving of workflow constructs
with a HOT template. I think we could do a purely declarative approach (if
we scope software orchestration in context of Heat right), and not define
such handles and references.
We are trying to shield this from the users in other cases in HOT
(WaitConditionHandle and references), so why introduce it here ...

>
> And then the workflow is just defined outside of the Heat template (ok
> I'm sure somebody will want to embed it, but I prefer stronger
> separation). Something like this gets uploaded as
> "mysql_webapp_workflow":
>
> [ 'step1': 'install_stuff',
>   'step2': 'wait(step1)',
>   'step3': 'allocate_sql_user(server=%mysql_server%)'
>   'step4': 'credentials=wait_and_read(step3)'
>   'step5': 'write_config_file(server=%webserver%)' ]
>
> Or maybe it is declared as a graph, or whatever, but it is not Heat's
> problem how to do workflows, it just feeds the necessary data from
> orchestration into the workflow engine. This also means you can use a
> non OpenStack workflow engine without any problems.
>
> I think after having talked about this, we should have workflow live in
> its own program.. we can always combine them if we want to, but having a
> clear line would mean keeping the interfaces clean.
>
> [1] http://lists.openstack.org/pipermail/openstack-dev/2013-October/
> 016605.html
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>