Open Stack

Excerpts from Thomas Spatzier's message of 2013-10-23 00:08:23 -0700:
> Clint Byrum <clint at fewbar.com> wrote on 23.10.2013 00:28:17:
> > From: Clint Byrum <clint at fewbar.com>
> > To: openstack-dev <openstack-dev at lists.openstack.org>,
> > Date: 23.10.2013 00:30
> > Subject: Re: [openstack-dev] [Heat] HOT Software configuration proposal
> >
> > Excerpts from Georgy Okrokvertskhov's message of 2013-10-22 13:32:40
> -0700:
> > > Hi Thomas,
> > >
> > > I agree with you on semantics part. At the same time I see a potential
> > > question which might appear - if semantics is limited by few states
> visible
> > > for Heat engine, then who actually does software orchestration?
> > > Will it be reasonable then to have software orchestration as separate
> > > subproject for Heat as a part of "Orchestration" OpenStack program?
> Heat
> > > engine will then do dependency tracking and will use components as a
> > > reference for software orchestration engine which will perform actual
> > > deployment and high level software components coordination.
> > >
> > > This separated software orchestration engine may address all specific
> > > requirements proposed by different teams in this thread without
> affecting
> > > existing Heat engine.
> > >
> >
> > I'm not sure I know what "software orchestration" is, but I will take a
> > stab at a succinct definition:
> >
> > Coordination of software configuration across multiple hosts.
> >
> > If that is what you mean, then I believe what you actually want is
> > workflow. And for that, we have the Mistral project which was recently
> > announced [1].
> 
> My view of "software orchestration" in a sense of what Heat should be able
> to do is being able to bring up software installations (e.g. web server, a
> DBMS, a custom application) on-top of a bare compute resource by invoking a
> software config tool (e.g. Chef, Puppet ...) at the right point in time and
> let that tool do the actual work.
> Invoke does not necessarily mean to call an API of such a tool, but rather
> make sure it is bootstrapped and maybe gets a "go" signal to start.
> "software orchestration" could then further mean to give CM tools across
> hosts the "go" signal when the config on one host has completed. This be
> enabled by the signaling enhancements Steve Baker mentioned in one of his
> recent mails.
> 
> For such kind of stuff, I think we could live without workflows but do it
> purely declaratively. Of course, a workflow could be the underlying
> mechanism, but I would not want to express this in a template. If users
> have very complex problems to solve and cannot live with the simple
> software orchestration I outlined, then still a workflow could be used for
> everying on-top of the OS.
> 
> Anyway, that was just my view and others most probably have different views
> again, so seems like we really have to sort out terminology :-)
> 

Right, we can do a lot without any external workflow help. We may be
able to do basically everything without external workflow help, however
I want to make sure that users have a well defined method for solving
their problems before Heat has a way to declaratively solve them.

> >
> > Use that and you will simply need to define your desired workflow and
> > feed it into Mistral using a Mistral Heat resource. We can create a
> > nice bootstrapping resource for Heat instances that shims the mistral
> > workflow execution agent into machines (or lets us use one already there
> > via custom images).
> >
> > I can imagine it working something like this:
> >
> > resources:
> >   mistral_workflow_handle:
> >     type: OS::Mistral::WorkflowHandle
> >   web_server:
> >     type: OS::Nova::Server
> >     components:
> >       mistral_agent:
> >         component_type: mistral
> >         params:
> >           workflow_: {ref: mistral_workflow_handle}
> >   mysql_server:
> >     type: OS::Nova::Server
> >     components:
> >       mistral_agent:
> >         component_type: mistral
> >         params:
> >           workflow_handle: {ref: mistral_workflow_handle}
> >   mistral_workflow:
> >     type: OS::Mistral::Workflow
> >     properties:
> >       handle: {ref: mistral_workflow_handle}
> >       workflow_reference: mysql_webapp_workflow
> >       params:
> >         mysql_server: {ref: mysql_server}
> >         webserver: {ref: web_server}
> >
> 
> While I can imagine that this works, I think for a big percentage of use
> cases I would be nice to avoid this inter-weaving of workflow constructs
> with a HOT template. I think we could do a purely declarative approach (if
> we scope software orchestration in context of Heat right), and not define
> such handles and references.
> We are trying to shield this from the users in other cases in HOT
> (WaitConditionHandle and references), so why introduce it here ...
> 

I totally agree. Once the bits above work and are available as individual
pieces for users to construct things with, the lowest common denominator
parts can become components/provider/etc. or be optimized into resource
plugins.

My point is to focus on getting the interface between multi-host
workflow and Heat well defined and serving at least one use case. In
this context, Puppet and Chef, and 'cfn-init' are all single-instance
workflow engines btw.