Open Stack

It seems to me that something is missing in our discussion.

If something depends on something else there must be a definition of that
something. It is clear that it is not the case that one instance depends on
another but one application depends on another application. But there is no
such thing as application (service, whatever) in HOT templates. Only
low-level resources. And even resources cannot be grouped to some
application scope because typical HOT template has resources that are
shared between several applications (network, security groups etc.). It
also possible to have several applications sharing a single VM instance.
That brings us to a conclusion that applications and resources cannot be
mixed in the same template on the same level of abstraction.

Now suppose we did somehow established the dependency between two
applications. But this dependency is out of scope of particular HOT
template. Thats because HOT template says what user whishes to install. But
a dependency between applications is an attribute of applications
themselves, not the particular deployment. For example WordPress requires
database. It always does. Not that it requires it within this particular
template but a a universal rule. In Murano we call it data vs. metadata
separation. If there is a metadata that says "WordPress requires DB" then
you not just only don't have to repeat it in each template but you cannot
even ask a system to deploy WordPress without DB.

So the question is maybe we need to think about applications/services and
their metadata before going into workflow orchestration? Otherwise the
whole orchestration would be reinvented time and time again with each new
HOT template.

What are your thoughts on this?


On Wed, Oct 9, 2013 at 11:37 PM, Georgy Okrokvertskhov <
gokrokvertskhov at mirantis.com> wrote:

> Hi Lakshminaraya,
>
> Thank you for bringing your use case and your thought here. That is
> exactly tried to achieve in Murano project.
> There are important  aspects you highlighted. Sometime resource model is
> two high level to describe deployment process. If you start to use more
> granular approach to have defined steps of deployment you will finish with
> workflow approach where you have fine control of deployment process but
> description will be quite complex.
>
> I think the HOT approach is to provide a simple way do describe you
> deployment which consists of solid bricks (resources). If you are using
> standard resources you can easily create a simple HOT template for your
> deployment. If you need some custom resource you basically have two options
> - create new resource class and hide all complexity inside the code or use
> some workflows language to describe all steps required. The first approach
> is currently supported by Heat. We have an experience of creating new
> custom resources for orchestration deployment to specific IT infrastructure
> with specific hardware and software.
>
> Right now we are trying to figure out the possibility of adding workflows
> to HOT. It looks like adding workflows language directly might harm HOT
> simplicity by overloaded DSL syntax and structures.
>
> I actually see the value in Steve's idea to have specific resource or
> resource set to call workflows execution on external engine. In this case
> HOT template will be still pretty simple as all workflow details will be
> hidden, but still manageable without code writing.
>
> Thanks
> Gosha
>
>
> On Wed, Oct 9, 2013 at 11:31 AM, Lakshminaraya Renganarayana <
> lrengan at us.ibm.com> wrote:
>
>> Steven Hardy <shardy at redhat.com> wrote on 10/09/2013 05:24:38 AM:
>>
>>
>> >
>> > So as has already been mentioned, Heat defines an internal workflow,
>> based
>> > on the declarative model defined in the template.
>> >
>> > The model should define dependencies, and Heat should convert those
>> > dependencies into a workflow internally.  IMO if the user also needs to
>> > describe a workflow explicitly in the template, then we've probably
>> failed
>> > to provide the right template interfaces for describing depenendencies.
>>
>> I agree with Steven here, models should define the dependencies and Heat
>> should realize/enforce them. An important design issue is granularity at
>> which dependencies are defined and enforced. I am aware of the
>> wait-condition
>> and signal constructs in Heat, but I find them a bit low-level as they
>> are prone
>> to the classic dead-lock and race condition problems.  I would like to
>> have
>> higher level constructs that support finer-granularity dependences which
>> are needed for software orchestration. Reading through the various
>> disucssion
>> on this topic in this mailing list, I see that many would like to have
>> such
>> higher level constructs for coordination.
>>
>> In our experience with software orchestration using our own DSL and also
>> with
>> some extensions to Heat, we found that the granularity of VMs or
>> Resources to be
>> too coarse for defining dependencies for software orchestration. For
>> example, consider
>> a two VM app, with VMs vmA, vmB, and a set of software components (ai's
>> and bi's)
>> to be installed on them:
>>
>> vmA = base-vmA + a1 + a2 + a3
>> vmB = base-vmB + b1 + b2 + b3
>>
>> let us say that software component b1 of vmB, requires a config value
>> produced by
>> software component a1 of vmA. How to declaratively model this dependence?
>> Clearly,
>> modeling a dependence between just base-vmA and base-vmB is not enough.
>> However,
>> defining a dependence between the whole of vmA and vmB is too coarse. It
>> would be ideal
>> to be able to define a dependence at the granularity of software
>> components, i.e.,
>> vmB.b1 depends on vmA.a1. Of course, it would also be good to capture
>> what value
>> is passed between vmB.b1 and vmA.a1, so that the communication can be
>> facilitated
>> by the orchestration engine.
>>
>> We found that such finer granular modeling of the dependencies provides
>> two valuable benefits:
>>
>> 1. Faster total (resources + software setup) deployment time. For the
>> example described
>> above, a coarse-granularity dependence enforcer would start the
>> deployment of base-vmB after
>> vmA + a1 + a2 + a3 is completed, but a fine-granularity dependence
>> enforcer would start base-vmA
>> and base-vmB concurrently, and then suspend the execution of vmB.b1 until
>> vmA.a1 is complete and then
>> let the rest of deployment proceed concurrently, resulting in a faster
>> completion.
>>
>> 2. More flexible dependencies. For example, mutual dependencies between
>> resources,
>> which can be satisfied when orchestrated at a finer granularity. Using
>> the example described
>> above, fine-granularity would allow vmB.b1 depends_on vmA.a1 and also
>> vmA.a3 depends_on vmB.b2,
>> but coarse-granularity model would flag this as a cyclic dependence.
>>
>> There are two aspects that needs support:
>>
>> 1. Heat/HOT template level constructs to support declarative expression
>> of such fine-granularity
>> dependencies and the values communicated / passed for the dependence.
>> 2. Support from Heat engine / analyzer in supporting the runtime
>> ordering, coordination between
>> resources, and also the communication of the values.
>>
>> What are your thoughts?
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Georgy Okrokvertskhov
> Technical Program Manager,
> Cloud and Infrastructure Services,
> Mirantis
> http://www.mirantis.com
> Tel. +1 650 963 9828
> Mob. +1 650 996 3284
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
slagun at mirantis.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131010/e8185c6b/attachment.html>