<html><body>
<p><font size="2" face="sans-serif">Excellent discussion on various issues around orchestration and coordination -- thanks to you all, in particular to Clint, Angus, Stan, Thomas, Joshua, Zane, Steve ...</font><br>
<br>
<font size="2" face="sans-serif">After reading the discussions, I am finding the following themes emerging (please feel free to correct/add):</font><br>
<br>
<font size="2" face="sans-serif">1. Most of the building blocks needed for effective coordination and orchestration are already in Heat/HOT.</font><br>
<font size="2" face="sans-serif">2. Heat would like to view software configuration as a resource (type) with related providers + plugins</font><br>
<font size="2" face="sans-serif">3. There is scope for communication/synchronization mechanisms that would complement the wait-conditions and signals</font><br>
<br>
<font size="2" face="sans-serif">I would like to propose a simple abstraction that would complement the current wait-conditions and signals. My proposal is based our experience with supporting such an abstraction on our DSL and also on an extension of Heat. In a nut-shell, this abstraction is a global data space (visible across resources, stacks) from which resources can read and write their inputs / outputs PLUS the semantics that reads will block until the read values are available and writes are non-blocking. We used ZooKeeper to implement this global data space and the blocking-read/non-blocking-writes semantics. But, these could be implemented using several other mechanisms and I believe the techniques currently used by Heat for meta-data service can be used here. </font><br>
<br>
<font size="2" face="sans-serif">I would like to make clear that I am not proposing a replacement for wait-conditions and signals. I am hoping that wait-conditions and signals would be used by power-users (concurrent/distributed programming experts) and the proposed abstraction would be used by folks (like me) who do not want to reason about concurrency and related problems. Also, the proposed global data-space with blocking reads and non-blocking writes is not a new idea (google tuple-spaces, linda) and it has been proven in other domains such as coordination languages to improve the level of abstraction and productivity.</font><br>
<br>
<font size="2" face="sans-serif">The benefits of the proposed abstraction are:</font><br>
<font size="2" face="sans-serif">G1. Support finer granularity of dependences</font><br>
<font size="2" face="sans-serif">G2. Allow Heat to reason/analyze about these dependences so that it can order resource creations/management</font><br>
<font size="2" face="sans-serif">G3. Avoid classic synchronization problems such as dead-locks and race conditions</font><br>
<font size="2" face="sans-serif">G4 *Conjecture* : Capture most of the coordination use cases (including those required for software configuration / orchestration).</font><br>
<br>
<font size="2" face="sans-serif">Here is more detailed description: Let us say that we can use either pre-defined or custom resource types to define resources at arbitrary levels of granularity. This can be easily supported and I guess is already possible in current version of Heat/HOT. Given this, the proposed abstraction has two parts: (1) an interface style specification a resource's inputs and outputs and (2) a global name/data space. The interface specification which would capture </font><br>
<br>
<font size="2" face="sans-serif">- INPUTS: all the attributes that are consumed/used/read by that resource (currently, we have Ref, GetAttrs that can give this implicitly)</font><br>
<br>
<font size="2" face="sans-serif">- OUTPUTS: all the attributes that are produced/written by that resource (I do not know if this write-set is currently well-defined for a resource. I think some of them are implicitly defined by Heat on particular resource types.)</font><br>
<br>
<font size="2" face="sans-serif">- Global name-space and data-space : all the values produced and consumed (INPUTS/OUTPUTS) are described using a names that are fully qualified (XXX.stack_name.resource_name.property_name). The data values associated with these names are stored in a global data-space. Reads are blocking, i.e., reading a value will block the execution resource/thread until the value is available. Writes are non-blocking, i.e., any thread can write a value and the write will succeed immediately.</font><br>
<br>
<font size="2" face="sans-serif">The ability to define resources at arbitrary levels of granularity together with the explicit specification of INPUTS/OUTPUTS allows us to reap the benefits G1 and G2 outlined above. Note that the ability to reason about the inputs/outputs of each resource and the induced dependencies will also allow Heat to detect dead-locks via dependence cycles (benefit G3). This is already done today in Heat for Refs, GetAttr on base-resources, but the proposal is to extend the same to arbitrary attributes for any resource. The blocking-read and non-blocking writes further structures the specification to avoid deadlocks and race conditions (benefit G3). </font><br>
<br>
<font size="2" face="sans-serif">As for G4, the conjecture, I can only give as evidence our experience with using our DSL with the proposed abstraction to deploy a few reasonably large applications :-)</font><br>
<br>
<font size="2" face="sans-serif">I would like to know your comments and suggestions. Also, if there is interest I can write a Blueprint / proposal with more details and use-cases.</font><br>
<br>
<font size="2" face="sans-serif">Thanks,</font><br>
<font size="2" face="sans-serif">LN</font><br>
<br>
<br>
<br>
<tt><font size="2">Clint Byrum <clint@fewbar.com> wrote on 10/11/2013 12:40:19 PM:<br>
<br>
> From: Clint Byrum <clint@fewbar.com></font></tt><br>
<tt><font size="2">> To: openstack-dev <openstack-dev@lists.openstack.org></font></tt><br>
<tt><font size="2">> Date: 10/11/2013 12:43 PM</font></tt><br>
<tt><font size="2">> Subject: Re: [openstack-dev] [Heat] HOT Software orchestration <br>
> proposal for workflows</font></tt><br>
<tt><font size="2">> <br>
> Excerpts from Stan Lagun's message of 2013-10-11 07:22:37 -0700:<br>
> > Hello,<br>
> > <br>
> > Thanks Angus, Clint, I've got your design.<br>
> > <br>
> > It seems that Murano can built on top of that. With service metadata<br>
> > knowledge Murano can generate HOT templates with set of interdependent<br>
> > configs.<br>
> > Here is what will be needed:<br>
> > <br>
> > 1. Ability to implement support for custom software configuration tool<br>
> > (type: OS::SoftwareConfig::MuranoAgent)<br>
> <br>
> These are really syntactic sugar, but you can implement them as resource<br>
> plugins for users who want Murano resources. In the absence of this,<br>
> just putting things in the free form metadata area of the resource that<br>
> the MuranoAgent can interpret would suffice.<br>
> <br>
> > 2. Ability to provide arbitrary input values for the config<br>
> <br>
> We already have that, there's a free-form json document called "metadata"<br>
> attached to every resource. Or maybe I missed what you mean here. The<br>
> new capability that is in the works that will make that better is to<br>
> have multiple reusable metadata blocks referenced on one instance.<br>
> <br>
> > 3. Ability to return arbitrary (JSON-compatible) data structure from config<br>
> > application and use attributes of that structure as an input for other<br>
> > configs<br>
> <br>
> Note that I'd like to see more use cases specified for this ability. The<br>
> random string generator that Steve Baker has put up should handle most<br>
> cases where you just need passwords. Generated key sharing might best<br>
> be deferred to something like Barbican which does a lot more than Heat<br>
> to try and keep your secrets safe.<br>
> <br>
> > 4. Ability to provide config body that is an input to Murano Agent of<br>
> > arbitrary size<br>
> <br>
> Isn't this the same as 2?<br>
> <br>
> > 5. Work well with large graph of configs with a lot of dependencies.<br>
> > Independent configs on different VMs should be applied in parallel.<br>
> ><br>
> <br>
> Yes this does look good. For dependent configs, the exsiting wait<br>
> condition can be used.<br>
> <br>
> > Does it confirm to your plans?<br>
> > <br>
> <br>
> I think it confirms that we're heading toward consensus on where to draw<br>
> the software config vs. infrastructure orchestration line. That is very<br>
> exciting. :)<br>
> <br>
> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> OpenStack-dev@lists.openstack.org<br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
> <br>
</font></tt></body></html>