<html><body>
<p><tt><font size="2">Stan Lagun <slagun@mirantis.com> wrote on 10/09/2013 04:07:33 PM:<br>
<br>
It seems to me that something is missing in our discussion.<br>
</font></tt><br>
<tt><font size="2">> If something depends on something else there must be a definition of<br>
> that something. It is clear that it is not the case that one <br>
> instance depends on another but one application depends on another <br>
> application. But there is no such thing as application (service, <br>
> whatever) in HOT templates. Only low-level resources. And even <br>
> resources cannot be grouped to some application scope because <br>
> typical HOT template has resources that are shared between several <br>
> applications (network, security groups etc.). It also possible to <br>
> have several applications sharing a single VM instance. That brings <br>
> us to a conclusion that applications and resources cannot be mixed <br>
> in the same template on the same level of abstraction.</font></tt><br>
<br>
<tt><font size="2">Good point on the levels of abstraction.<br>
</font></tt><br>
<tt><font size="2">> Now suppose we did somehow established the dependency between two <br>
> applications. But this dependency is out of scope of particular HOT <br>
> template. Thats because HOT template says what user whishes to <br>
> install. But a dependency between applications is an attribute of <br>
> applications themselves, not the particular deployment. For example <br>
> WordPress requires database. It always does. Not that it requires it<br>
> within this particular template but a a universal rule. In Murano we<br>
> call it data vs. metadata separation. If there is a metadata that <br>
> says "WordPress requires DB" then you not just only don't have to <br>
> repeat it in each template but you cannot even ask a system to <br>
> deploy WordPress without DB.</font></tt><br>
<br>
<tt><font size="2">I think the kind of dependency you have outlined above is more of</font></tt><br>
<tt><font size="2">software component requirements of an application. These kind of</font></tt><br>
<tt><font size="2">semantic dependencies are important and are probably outside the</font></tt><br>
<tt><font size="2">scope of Heat. The kind of dependencies I referred to are of the </font></tt><br>
<tt><font size="2">nature of data-flow between software components: for example, a</font></tt><br>
<tt><font size="2">tomcat application server needs (and hence, depends on) the </font></tt><br>
<tt><font size="2">DB's username/password to set up its configuration. How do we </font></tt><br>
<tt><font size="2">model such a data-flow dependence and how to we facilitate the </font></tt><br>
<tt><font size="2">communication of such values from the DB to the tomcat component? </font></tt><br>
<tt><font size="2">IMHO, such questions are related to Heat.<br>
</font></tt><br>
<tt><font size="2">> So the question is maybe we need to think about applications/<br>
> services and their metadata before going into workflow <br>
> orchestration? Otherwise the whole orchestration would be reinvented<br>
> time and time again with each new HOT template.<br>
</font></tt><br>
<tt><font size="2">> What are your thoughts on this?</font></tt><br>
<br>
<tt><font size="2">I find your separation of metadata vs. data useful. In my opinion,</font></tt><br>
<tt><font size="2">the kind of metadata you are trying to capture would be best </font></tt><br>
<tt><font size="2">modeled by a DSL that sits on top of HOT/Heat.</font></tt><br>
<br>
<tt><font size="2">Thanks,</font></tt><br>
<tt><font size="2">LN</font></tt><br>
<br>
<br>
<tt><font size="2">> <br>
</font></tt><br>
<tt><font size="2">> On Wed, Oct 9, 2013 at 11:37 PM, Georgy Okrokvertskhov <<br>
> gokrokvertskhov@mirantis.com> wrote:</font></tt><br>
<tt><font size="2">> Hi Lakshminaraya,</font></tt><br>
<tt><font size="2">> <br>
> Thank you for bringing your use case and your thought here. That is <br>
> exactly tried to achieve in Murano project. </font></tt><br>
<tt><font size="2">> There are important aspects you highlighted. Sometime resource <br>
> model is two high level to describe deployment process. If you start<br>
> to use more granular approach to have defined steps of deployment <br>
> you will finish with workflow approach where you have fine control <br>
> of deployment process but description will be quite complex. </font></tt><br>
<tt><font size="2">> <br>
> I think the HOT approach is to provide a simple way do describe you <br>
> deployment which consists of solid bricks (resources). If you are <br>
> using standard resources you can easily create a simple HOT template<br>
> for your deployment. If you need some custom resource you basically <br>
> have two options - create new resource class and hide all complexity<br>
> inside the code or use some workflows language to describe all steps<br>
> required. The first approach is currently supported by Heat. We have<br>
> an experience of creating new custom resources for orchestration <br>
> deployment to specific IT infrastructure with specific hardware and software.</font></tt><br>
<tt><font size="2">> <br>
> Right now we are trying to figure out the possibility of adding <br>
> workflows to HOT. It looks like adding workflows language directly <br>
> might harm HOT simplicity by overloaded DSL syntax and structures. </font></tt><br>
<tt><font size="2">> <br>
> I actually see the value in Steve's idea to have specific resource <br>
> or resource set to call workflows execution on external engine. In <br>
> this case HOT template will be still pretty simple as all workflow <br>
> details will be hidden, but still manageable without code writing. </font></tt><br>
<tt><font size="2">> <br>
> Thanks</font></tt><br>
<tt><font size="2">> Gosha</font></tt><br>
<tt><font size="2">> <br>
</font></tt><br>
<tt><font size="2">> On Wed, Oct 9, 2013 at 11:31 AM, Lakshminaraya Renganarayana <<br>
> lrengan@us.ibm.com> wrote:</font></tt><br>
<tt><font size="2">> Steven Hardy <shardy@redhat.com> wrote on 10/09/2013 05:24:38 AM:</font></tt><br>
<tt><font size="2">> <br>
> <br>
> > <br>
> > So as has already been mentioned, Heat defines an internal workflow, based<br>
> > on the declarative model defined in the template.<br>
> > <br>
> > The model should define dependencies, and Heat should convert those<br>
> > dependencies into a workflow internally. IMO if the user also needs to<br>
> > describe a workflow explicitly in the template, then we've probably failed<br>
> > to provide the right template interfaces for describing depenendencies.</font></tt><br>
<tt><font size="2">> <br>
> I agree with Steven here, models should define the dependencies and Heat<br>
> should realize/enforce them. An important design issue is granularity at<br>
> which dependencies are defined and enforced. I am aware of the wait-condition<br>
> and signal constructs in Heat, but I find them a bit low-level as <br>
> they are prone<br>
> to the classic dead-lock and race condition problems. I would like to have <br>
> higher level constructs that support finer-granularity dependences which<br>
> are needed for software orchestration. Reading through the various disucssion<br>
> on this topic in this mailing list, I see that many would like to have such<br>
> higher level constructs for coordination.<br>
> <br>
> In our experience with software orchestration using our own DSL and also with <br>
> some extensions to Heat, we found that the granularity of VMs or <br>
> Resources to be <br>
> too coarse for defining dependencies for software orchestration. For<br>
> example, consider <br>
> a two VM app, with VMs vmA, vmB, and a set of software components <br>
> (ai's and bi's) <br>
> to be installed on them:<br>
> <br>
> vmA = base-vmA + a1 + a2 + a3<br>
> vmB = base-vmB + b1 + b2 + b3<br>
> <br>
> let us say that software component b1 of vmB, requires a config <br>
> value produced by<br>
> software component a1 of vmA. How to declaratively model this <br>
> dependence? Clearly,<br>
> modeling a dependence between just base-vmA and base-vmB is not <br>
> enough. However,<br>
> defining a dependence between the whole of vmA and vmB is too <br>
> coarse. It would be ideal<br>
> to be able to define a dependence at the granularity of software <br>
> components, i.e., <br>
> vmB.b1 depends on vmA.a1. Of course, it would also be good to <br>
> capture what value <br>
> is passed between vmB.b1 and vmA.a1, so that the communication can <br>
> be facilitated<br>
> by the orchestration engine. <br>
> <br>
> We found that such finer granular modeling of the dependencies <br>
> provides two valuable benefits:<br>
> <br>
> 1. Faster total (resources + software setup) deployment time. For <br>
> the example described<br>
> above, a coarse-granularity dependence enforcer would start the <br>
> deployment of base-vmB after<br>
> vmA + a1 + a2 + a3 is completed, but a fine-granularity dependence <br>
> enforcer would start base-vmA<br>
> and base-vmB concurrently, and then suspend the execution of vmB.b1 <br>
> until vmA.a1 is complete and then <br>
> let the rest of deployment proceed concurrently, resulting in a <br>
> faster completion.<br>
> <br>
> 2. More flexible dependencies. For example, mutual dependencies <br>
> between resources, <br>
> which can be satisfied when orchestrated at a finer granularity. <br>
> Using the example described<br>
> above, fine-granularity would allow vmB.b1 depends_on vmA.a1 and <br>
> also vmA.a3 depends_on vmB.b2,<br>
> but coarse-granularity model would flag this as a cyclic dependence.<br>
> <br>
> There are two aspects that needs support:<br>
> <br>
> 1. Heat/HOT template level constructs to support declarative <br>
> expression of such fine-granularity<br>
> dependencies and the values communicated / passed for the dependence.<br>
> 2. Support from Heat engine / analyzer in supporting the runtime <br>
> ordering, coordination between<br>
> resources, and also the communication of the values. <br>
> <br>
> What are your thoughts? </font></tt><br>
<tt><font size="2">> <br>
> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> OpenStack-dev@lists.openstack.org<br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</font></tt><br>
<tt><font size="2">> <br>
</font></tt><br>
<tt><font size="2">> <br>
> -- <br>
> Georgy Okrokvertskhov<br>
> Technical Program Manager,<br>
> Cloud and Infrastructure Services,<br>
> Mirantis<br>
> <a href="http://www.mirantis.com">http://www.mirantis.com</a><br>
> Tel. +1 650 963 9828<br>
> Mob. +1 650 996 3284</font></tt><br>
<tt><font size="2">> <br>
> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> OpenStack-dev@lists.openstack.org<br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</font></tt><br>
<tt><font size="2">> <br>
> <br>
> <br>
> -- </font></tt><br>
<tt><font size="2">> Sincerely yours<br>
> Stanislav (Stan) Lagun<br>
> Senior Developer<br>
> Mirantis<br>
> 35b/3, Vorontsovskaya St.<br>
> Moscow, Russia<br>
> Skype: stanlagun<br>
> www.mirantis.com<br>
> slagun@mirantis.com_______________________________________________<br>
> OpenStack-dev mailing list<br>
> OpenStack-dev@lists.openstack.org<br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</font></tt></body></html>