Open Stack

On Wed, Oct 09, 2013 at 12:53:45AM +0400, Stan Lagun wrote:
> Hello,
> 
> I’m one of the engineer working on Murano project. Recently we started a
> discussion about Murano and Heat Software orchestration and I want to
> continue this discussion with more technical details.

Thanks, we're certainly interested in Murano, and are keen to discuss your
roadmap, and where requirements and integration opportunities exist.

> In our project we do deployment of complex multi-instance Windows services.
> Those services usually require specific multi-VM orchestration that is
> currently impossible or at least not that easy to achieve with Heat. As you
> are currently doing HOT software orchestration design we would like to
> participate in HOT Software orchestration design and contribute into it, so
> that the Heat could address use-cases which we believe are very common.
> 
> For example here is how deployment of a SQL Server cluster goes:
> 
>    1.
> 
>    Allocate Windows VMs for SQL Server cluster

Heat can already do this, you just define either OS::Nova::Server or
AWS::EC2::Instance resource in your template, or possibly a group of
instances via OS::Heat::InstanceGroup if the configuration is the same for
all VMs

>    2.
>    Enable secondary IP address from user input on all SQL Windows instances

So this again is already possible, via several resource types,
AWS::EC2::NetworkInterface, AWS::EC2::EIP, OS::Neutron::Port etc..

I suggest using the Neutron resources where possible, if you don't care
about Clouformation portability.

>    3.
>    Install SQL Server prerequisites on each node

So Heat is already able to do this, via a couple of methods, for Linux VMs,
so we just need the in-instance agent support for windows (cloud-init,
optionally combined with agents like cfn-init from heat-cfntools)

Can you clarify what you're using for in-instance agents currently,
cloudbase-init, and/or some bespoke tools?

>    4.
>    Choose a master node and install Failover Cluster on it
>    5.
>    Configure all nodes so that they know which one of them is the master

I'm not sure what's involved in these steps, but it seems like there are
serialization requirements, which can be handled via WaitConditions.

One thing I think we do need to look at is ways to enable expression of
serialization requirements via HOT, which don't require use of the
AWS-compatible WaitCondition resources.

So I think we already have the required functionality, we just need to
build out better native interfaces to it.

>    Configure all nodes so that they know which one of them is the master
>    6.
> 
>    Install SQL Server on all nodes
>    7.
> 
>    Initialize AlwaysOn on all nodes except for the master
>    8.
> 
>    Initialize Primary replica
>    9.
> 
>    Initialize secondary replicas
> 
> 
> All of the steps must take place in appropriate order depending on the
> state of other nodes. Some steps require an output from previous steps and
> all of them require some input parameters. SQL Server requires an Active
> Directory service in order to use Failover mechanism and installation of
> Active Directory with primary and secondary controllers is a complex
> workflow of its own.

So all of this seems possible right now using WaitConditions, but as
mentioned above we should look at ways to provide a better and more
flexible native interface to similar functionality.

> That is why it is necessary to have some central coordination service which
> would handle deployment workflow and perform specific actions (create VMs
> and other OpenStack resources, do something on that VM) on each stage
> according to that workflow. We think that Heat is the best place for such
> service.

Yep, we already do coordination of VM deployment and other openstack
resources, by managing implicit and explicit dependencies between those
resources.

> Our idea is to extend HOT DSL by adding  workflow definition capabilities
> as an explicit list of resources, components’ states and actions. States
> may depend on each other so that you can reach state X only after you’ve
> reached states Y and Z that the X depends on. The goal is from initial
> state to reach some final state “Deployed”.

IMHO there isn't a real need to provide explicit control of the workflow
implied by the resource dependencies for the sort of use-case you describe.

What I think is needed is simply a better native interface to serialization
primitives/resources.

> There is such state graph for each of our deployment entities (service,
> VMs, other things). There is also an action that must be performed on each
> state.
> For example states graph from example above would look like this:
> 
> The goal is to reach Service_Done state which depends on VM1_Done and
> VM2_Done states and so on from initial Service_Start state.
> 
> We propose to extend HOT DSL with workflow definition capabilities where
> you can describe step by step instruction to install service and properly
> handle errors on each step.

So as has already been mentioned, Heat defines an internal workflow, based
on the declarative model defined in the template.

The model should define dependencies, and Heat should convert those
dependencies into a workflow internally.  IMO if the user also needs to
describe a workflow explicitly in the template, then we've probably failed
to provide the right template interfaces for describing depenendencies.

> We already have an experience in implementation of the DSL, workflow
> description and processing mechanism for complex deployments and believe
> we’ll all benefit by re-using this experience and existing code, having
> properly discussed and agreed on abstraction layers and distribution of
> responsibilities between OS components. There is an idea of implementing
> part of workflow processing mechanism as a part of Convection proposal,
> which would allow other OS projects to benefit by using this.
> 
> We would like to discuss if such design could become a part of future Heat
> version as well as other possible contributions from Murano team.

So as others have mentioned, you may also want to look at the TaskFlow
project, as if you really want to define workflow directly, that seems like
it may be the most appropriate place to do it.

However I'd very much like to continue the discussion, so we can understand
what is missing in Heat to support your use-case.

Can you evaluate the existing Heat functionality, and provide feedback
definining specific issues, so we can define what the functionality gaps
are, and discuss the best way to solve them?

Thanks!

Steve