Open Stack

On 09/10/13 19:31 +0100, Steven Hardy wrote:
>On Wed, Oct 09, 2013 at 06:59:22PM +0200, Alex Rudenko wrote:
>> Hi everyone,
>>
>> I've read this thread and I'd like to share some thoughts. In my opinion,
>> workflows (which run on VMs) can be integrated with heat templates as
>> follows:
>>
>>    1. workflow definitions should be defined separately and processed by
>>    stand-alone workflow engines (chef, puppet etc).
>
>I agree, and I think this is the direction we're headed with the
>software-config blueprints - essentially we should end up with some new
>Heat *resources* which encapsulate software configuration.

Exactly.

I think we need a software-configuration-aas sub-project that knows
how to take puppet/chef/salt/... config and deploy it. Then Heat just
has Resources for these (OS::SoftwareConfig::Puppet). 

We should even move our WaitConditions and Metadata over to that 
yet-to-be-made service so that Heat is totally clean of software config.

How would this solve ordering issues:

resources:
  config1:
    type: OS::SoftwareConfig::Puppet
    hosted_on: server1
    ...
  config2:
    type: OS::SoftwareConfig::Puppet
    hosted_on: server1
    depends_on: config3
    ...
  config3:
    type: OS::SoftwareConfig::Puppet
    hosted_on: server2
    depends_on: config1
    ...
  server1:
    type: OS::Nova::Server
    ...
  server2:
    type: OS::Nova::Server
    ...


Heat knows all about ordering:
It starts the resources:
server1, server2
config1
config3
config2

There is the normal contract in the client:
we post the config to software-config-service
and we wait for the state == ACTIVE (when the config is applied)
before progressing to a resource that is dependant on it.

-Angus

>
>IMO there is some confusion around the scope of HOT, we should not be
>adding functionality to it which already exists in established config
>management tools IMO, instead we should focus on better integration with
>exisitng tools at the resource level, and identifying template interfaces
>which require more flexibility (for example serialization primitives)
>
>>    2. the HOT resources should reference workflows which they require,
>>    specifying a type of workflow and the way to access a workflow definition.
>>    The workflow definition might be provided along with HOT.
>
>So again, I think this acatually has very little to do with HOT.  The
>*Heat* resources may define software configuration, or possibly some sort
>of workflow, which is acted upon by $thing which is not Heat.
>
>So in the example provided by the OP, maybe you'd have a Murano resource,
>which knows how to define the input to the Murano API, which might trigger
>workflow type actions to happen in the Murano service.
>
>>    3. Heat should treat the orchestration templates as transactions (i.e.
>>    Heat should be able to rollback in two cases: 1) if something goes wrong
>>    during processing of an orchestration workflow 2) when a stand-alone
>>    workflow engine reports an error during processing of a workflow associated
>>    with a resource)
>
>So we already have the capability for resources to recieve signals, which
>would allow (2) in the asynchronous case.  But it seems to me that this is
>still a serialization problem, ie a synchronous case, therefore (2) is just
>part of (1).
>
>E.g
>
>- Heat stack create starts
>- Murano resource created (CREATE IN_PROGRESS state)
>- Murano workdlow stuff happens, signals Heat with success/failure
>- Murano resource transitions to either COMPLETE or FAILED state
>- If a FAILED state happened, e.g on update, we can roll back to the
>  previous stack definition (this is already possible in Heat)
>
>>    4. Heat should expose an API which enables basic communication between
>>    running workflows. Additionally, Heat should provide an API to workflows
>>    that allows workflows to specify whether they completed successfully or
>>    not. The reference to these APIs should be passed to the workflow engine
>>    that is responsible for executing workflows on VMs.
>
>I personally don't think this is in scope for Heat.  We already have an API
>which exposes the status of stacks and resources.  Exposing some different
>API which describes a workflow implemented by a specific subset of resource
>types makes no sense to me.
>
>>
>> Pros of each point:
>> 1 & 2 - keeps Heat simple and gives a possibility to choose the best
>> workflows and engines among available ones.
>> 3 - adds some kind of all-or-nothing semantics improving the control and
>> awareness of what's going on inside VMs.
>> 4 - allows workflow synchronization and communication through Heat API.
>> Provides the error reporting mechanism for workflows. If a workflow does
>> not need this functionality, it can ignore it.
>
>IMHO (4) is very much a step too far, and is not well aligned with the
>current interfaces provided by Heat.
>
>I'm really keen to further discuss the use-cases here, but if possible, it
>would be helpful if folks can describe their requirements in less abstract
>terms, and with reference to our existing interfaces and template model.
>
>> These thoughts might show some gaps in my understanding of how Heat works,
>> but I would like to share them anyway.
>
>You've raised some good points, thanks.
>
>I'm really keen to further discuss the use-cases here, but if possible, it
>would be helpful if folks can describe their requirements in less abstract
>terms, and with reference to our existing interfaces and template model.
>That way we can start defining what is actually missing to support specific
>use-cases.
>
>So far I see the following emerging as definite requirements:
>- Better/more flexible native serialization interfaces (possbly HOT
>  additions)
>- Better/more flexible *resources* which encapsulate software configuration
>  on instances, probably with the flexibility of applying more than one
>  config to an instance (not necessarily related to any HOT changes at all)
>
>Thanks,
>
>Steve
>
>_______________________________________________
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev