Open Stack

Stan Lagun <slagun at mirantis.com> wrote on 10/09/2013 04:07:33 PM:

It seems to me that something is missing in our discussion.

> If something depends on something else there must be a definition of
> that something. It is clear that it is not the case that one
> instance depends on another but one application depends on another
> application. But there is no such thing as application (service,
> whatever) in HOT templates. Only low-level resources. And even
> resources cannot be grouped to some application scope because
> typical HOT template has resources that are shared between several
> applications (network, security groups etc.). It also possible to
> have several applications sharing a single VM instance. That brings
> us to a conclusion that applications and resources cannot be mixed
> in the same template on the same level of abstraction.

Good point on the levels of abstraction.

> Now suppose we did somehow established the dependency between two
> applications. But this dependency is out of scope of particular HOT
> template. Thats because HOT template says what user whishes to
> install. But a dependency between applications is an attribute of
> applications themselves, not the particular deployment. For example
> WordPress requires database. It always does. Not that it requires it
> within this particular template but a a universal rule. In Murano we
> call it data vs. metadata separation. If there is a metadata that
> says "WordPress requires DB" then you not just only don't have to
> repeat it in each template but you cannot even ask a system to
> deploy WordPress without DB.

I think the kind of dependency you have outlined above is more of
software component requirements of an application. These kind of
semantic dependencies are important and are probably outside the
scope of Heat.  The kind of dependencies I referred to are of the
nature of data-flow between software components: for example, a
tomcat application server needs (and hence, depends on)  the
DB's username/password to set up its configuration. How do we
model such a data-flow dependence and how to we facilitate the
communication of such values from the DB to the tomcat component?
IMHO, such questions are related to Heat.

> So the question is maybe we need to think about applications/
> services and their metadata before going into workflow
> orchestration? Otherwise the whole orchestration would be reinvented
> time and time again with each new HOT template.

> What are your thoughts on this?

I find your separation of metadata vs. data useful. In my opinion,
the kind of metadata you are trying to capture would be best
modeled by a DSL that sits on top of HOT/Heat.

Thanks,
LN


>

> On Wed, Oct 9, 2013 at 11:37 PM, Georgy Okrokvertskhov <
> gokrokvertskhov at mirantis.com> wrote:
> Hi Lakshminaraya,
>
> Thank you for bringing your use case and your thought here. That is
> exactly tried to achieve in Murano project.
> There are important  aspects you highlighted. Sometime resource
> model is two high level to describe deployment process. If you start
> to use more granular approach to have defined steps of deployment
> you will finish with workflow approach where you have fine control
> of deployment process but description will be quite complex.
>
> I think the HOT approach is to provide a simple way do describe you
> deployment which consists of solid bricks (resources). If you are
> using standard resources you can easily create a simple HOT template
> for your deployment. If you need some custom resource you basically
> have two options - create new resource class and hide all complexity
> inside the code or use some workflows language to describe all steps
> required. The first approach is currently supported by Heat. We have
> an experience of creating new custom resources for orchestration
> deployment to specific IT infrastructure with specific hardware and
software.
>
> Right now we are trying to figure out the possibility of adding
> workflows to HOT. It looks like adding workflows language directly
> might harm HOT simplicity by overloaded DSL syntax and structures.
>
> I actually see the value in Steve's idea to have specific resource
> or resource set to call workflows execution on external engine. In
> this case HOT template will be still pretty simple as all workflow
> details will be hidden, but still manageable without code writing.
>
> Thanks
> Gosha
>

> On Wed, Oct 9, 2013 at 11:31 AM, Lakshminaraya Renganarayana <
> lrengan at us.ibm.com> wrote:
> Steven Hardy <shardy at redhat.com> wrote on 10/09/2013 05:24:38 AM:
>
>
> >
> > So as has already been mentioned, Heat defines an internal workflow,
based
> > on the declarative model defined in the template.
> >
> > The model should define dependencies, and Heat should convert those
> > dependencies into a workflow internally.  IMO if the user also needs to
> > describe a workflow explicitly in the template, then we've probably
failed
> > to provide the right template interfaces for describing depenendencies.
>
> I agree with Steven here, models should define the dependencies and Heat
> should realize/enforce them. An important design issue is granularity at
> which dependencies are defined and enforced. I am aware of the
wait-condition
> and signal constructs in Heat, but I find them a bit low-level as
> they are prone
> to the classic dead-lock and race condition problems.  I would like to
have
> higher level constructs that support finer-granularity dependences which
> are needed for software orchestration. Reading through the various
disucssion
> on this topic in this mailing list, I see that many would like to have
such
> higher level constructs for coordination.
>
> In our experience with software orchestration using our own DSL and also
with
> some extensions to Heat, we found that the granularity of VMs or
> Resources to be
> too coarse for defining dependencies for software orchestration. For
> example, consider
> a two VM app, with VMs vmA, vmB, and a set of software components
> (ai's and bi's)
> to be installed on them:
>
> vmA = base-vmA + a1 + a2 + a3
> vmB = base-vmB + b1 + b2 + b3
>
> let us say that software component b1 of vmB, requires a config
> value produced by
> software component a1 of vmA. How to declaratively model this
> dependence? Clearly,
> modeling a dependence between just base-vmA and base-vmB is not
> enough. However,
> defining a dependence between the whole of vmA and vmB is too
> coarse. It would be ideal
> to be able to define a dependence at the granularity of software
> components, i.e.,
> vmB.b1 depends on vmA.a1. Of course, it would also be good to
> capture what value
> is passed between vmB.b1 and vmA.a1, so that the communication can
> be facilitated
> by the orchestration engine.
>
> We found that such finer granular modeling of the dependencies
> provides two valuable benefits:
>
> 1. Faster total (resources + software setup) deployment time. For
> the example described
> above, a coarse-granularity dependence enforcer would start the
> deployment of base-vmB after
> vmA + a1 + a2 + a3 is completed, but a fine-granularity dependence
> enforcer would start base-vmA
> and base-vmB concurrently, and then suspend the execution of vmB.b1
> until vmA.a1 is complete and then
> let the rest of deployment proceed concurrently, resulting in a
> faster completion.
>
> 2. More flexible dependencies. For example, mutual dependencies
> between resources,
> which can be satisfied when orchestrated at a finer granularity.
> Using the example described
> above, fine-granularity would allow vmB.b1 depends_on vmA.a1 and
> also vmA.a3 depends_on vmB.b2,
> but coarse-granularity model would flag this as a cyclic dependence.
>
> There are two aspects that needs support:
>
> 1. Heat/HOT template level constructs to support declarative
> expression of such fine-granularity
> dependencies and the values communicated / passed for the dependence.
> 2. Support from Heat engine / analyzer in supporting the runtime
> ordering, coordination between
> resources, and also the communication of the values.
>
> What are your thoughts?
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

>

>
> --
> Georgy Okrokvertskhov
> Technical Program Manager,
> Cloud and Infrastructure Services,
> Mirantis
> http://www.mirantis.com
> Tel. +1 650 963 9828
> Mob. +1 650 996 3284
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

>
>
>
> --
> Sincerely yours
> Stanislav (Stan) Lagun
> Senior Developer
> Mirantis
> 35b/3, Vorontsovskaya St.
> Moscow, Russia
> Skype: stanlagun
> www.mirantis.com
> slagun at mirantis.com_______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131009/f7589b46/attachment.html>