<html><body>
<p><font size="2" face="sans-serif">Hi,</font><br>
<br>
<font size="2" face="sans-serif">In the last Openstack Heat meeting there was good interest in proposals for cross-vm synchronization and communication and I had mentioned the prototype I have built. I had also promised that I will post an outline of the prototype ... Here it is. I might have missed some details, please feel free to ask / comment and I would be happy to explain more.</font><br>
<font size="2" face="sans-serif">---</font><br>
<font size="2" face="sans-serif">Goal of the prototype: Enable cross-vm synchronization and communication using high-level declarative description (no wait-conditions) Use chef as the CM tool.</font><br>
<br>
<font size="2" face="sans-serif">Design rationale / choices of the prototype (note that these were made just for the prototype and I am not proposing them to be the choices for Heat/HOT):</font><br>
<br>
<font size="2" face="sans-serif">D1: No new construct in Heat template</font><br>
<font size="2" face="sans-serif"> => use metadata sections</font><br>
<font size="2" face="sans-serif">D2: No extensions to core Heat engine</font><br>
<font size="2" face="sans-serif"> => use a pre-processor that will produce a Heat template that the standard Heat engine can consume</font><br>
<font size="2" face="sans-serif">D3: Do not require chef recipes to be modified</font><br>
<font size="2" face="sans-serif"> => use a convention of accessing inputs/outputs from chef node[][]</font><br>
<font size="2" face="sans-serif"> => use ruby meta-programming to intercept reads/writes to node[][] forward values</font><br>
<font size="2" face="sans-serif">D4: Use a standard distributed coordinator (don't reinvent)</font><br>
<font size="2" face="sans-serif"> => use zookeeper as a coordinator and as a global data space for communciation</font><br>
<br>
<font size="2" face="sans-serif">Overall, the flow is the following:</font><br>
<font size="2" face="sans-serif">1. User specifies a Heat template with details about software config and dependences in the metadata section of resources (see step S1 below).</font><br>
<font size="2" face="sans-serif">2. A pre-processor consumes this augmented heat template and produces another heat template with user-data sections with cloud-init scripts and also sets up a zookeeper instance with enough information to coordinate between the resources at runtime to realize the dependences and synchronization (see step S2)</font><br>
<font size="2" face="sans-serif">3. The generated heat template is fed into standard heat engine to deploy. After the VMs are created the cloud-init script kicks in. The cloud init script installs chef solo and then starts the execution of the roles specified in the metadata section. During this execution of the recipes the coordination is realized (see steps S2 and S3 below).</font><br>
<br>
<font size="2" face="sans-serif">Implementation scheme:</font><br>
<font size="2" face="sans-serif">S1. Use metadata section of each resource to describe (see attached example)</font><br>
<font size="2" face="sans-serif"> - a list of roles</font><br>
<font size="2" face="sans-serif"> - inputs to and outputs from each role and their mapping to resource attrs (any attr)</font><br>
<font size="2" face="sans-serif"> - convention: these inputs/outputs will be through chef node attrs node[][]</font><br>
<br>
<font size="2" face="sans-serif">S2. Dependence analysis and cloud init script generation</font><br>
<font size="2" face="sans-serif"> </font><br>
<font size="2" face="sans-serif"> Dependence analysis:</font><br>
<font size="2" face="sans-serif"> - resolve every reference that can be statically resolved using Heat's fucntions (this step just uses Heat's current dependence analysis -- Thanks to Zane Bitter for helping me understand this)</font><br>
<font size="2" face="sans-serif"> - flag all unresolved references as values resolved at run-time at communicated via the coordinator</font><br>
<br>
<font size="2" face="sans-serif"> Use cloud-init in user-data sections:</font><br>
<font size="2" face="sans-serif"> - automatically generate a script that would bootstrap chef and will run the roles/recipes in the order specified in the metadata section</font><br>
<font size="2" face="sans-serif"> - generate dependence info for zookeeper to coordinate at runtime</font><br>
<br>
<font size="2" face="sans-serif">S3. Coordinate synchronization and communication at run-time</font><br>
<font size="2" face="sans-serif"> - intercept reads and writes to node[][]</font><br>
<font size="2" face="sans-serif"> - if it is a remote read, get it from Zookeeper</font><br>
<font size="2" face="sans-serif"> - execution will block till the value is available</font><br>
<font size="2" face="sans-serif"> - if write is for a value required by a remote resource, write the value to Zookeeper</font><br>
<br>
<font size="2" face="sans-serif">The prototype is implemented in Python and Ruby is used for chef interception. </font><br>
<br>
<font size="2" face="sans-serif">There are alternatives for many of the choices I have made for the prototype:</font><br>
<font size="2" face="sans-serif"> - zookeeper can be replaced with any other service that provides a data space and distributed coordination</font><br>
<font size="2" face="sans-serif"> - chef can be replaced by any other CM tool (a little bit of design / convention needed for other CM tools because of the interception used in the prototype to catch reads/writes to node[][])</font><br>
<font size="2" face="sans-serif"> - the whole dependence analysis can be integrated into the Heat's dependence analyzer</font><br>
<font size="2" face="sans-serif"> - the component construct proposed recently (by Steve Baker) for HOT/Heat can be used to specify much of what is specified using the metadata sections in this prototype.</font><br>
<br>
<font size="2" face="sans-serif">I am interested in using my experience with this prototype to contribute to HOT/Heat's cross-vm synchronization and communication design and code. I look forward to your comments.</font><br>
<br>
<font size="2" face="sans-serif">Thanks,</font><br>
<font size="2" face="sans-serif">LN</font><br>
</body></html>