<div dir="ltr"><div><div><div>Just to add my 2 cents. <br><br>1. YAML is already familiar to OpenStack developers as Heat and others use it. So at least the syntax (not to mess with semantics) doesn't have to be learned. <br>
</div><br>2. YAML parser is very flexible an can be extended with additional types or constructs like Key: <filename.yaml> to include content of other file etc. Murano DSL may operate on already deserialized (parsed) data leaving yaml files access to hosting operation. Thus the engine itself can be independent from how and where the YAML files are stored. This is very good for App Catalog that stores its data in Glance. Also it always possible to serialize it to some other format (XML, JSON, whatever) if it required for some purpose<br>
</div><br>3. YAML declarations can be processed by Murano dashboard, 3-rd party software and various tooling for Murano. If it wasn't YAML but some handcrafted syntax then we would have to provide them with embeddable parser for that syntax<br>
</div><br>4. Implementing parser for full-blown language is not a trivial task to say the least. It would require much more time for development and greatly increase probability to shoot ourself in the foot. And we really don't want to have like a year between Murano versions. <br>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Feb 15, 2014 at 12:18 PM, Alexander Tivelkov <span dir="ltr"><<a href="mailto:ativelkov@mirantis.com" target="_blank">ativelkov@mirantis.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-size:small">Hi Joshua,</div><div class="gmail_default" style="font-size:small">
<br></div><div class="gmail_default" style="font-size:small">Thank you very much for you feedback!</div>
<div class="gmail_default" style="font-size:small">This is really a great question, and it was the first question we've asked ourselves when we started thinking about this new design.</div><div class="gmail_default" style="font-size:small">
We've considered both options: to have our own syntax (python-like, java-like, something-else-like) or to use YAML. We've chosen the latest, and here are the reasons.<br></div><div class="gmail_default" style="font-size:small">
<br></div><div class="gmail_default" style="font-size:small">The most important moment: this is not a general-purpose language, but a domain-specific one. And Murano's domain is manipulating with complex nested data-structures, which are usually represented by YAML. Murano's environments are in YAML. Murano's VM-side commands are wrapped into YAML. The primary building blocks of Murano - Heat templates - are written on YAML. Actually, at a simplified level murano's workflows can be thought of as algorithms that just generate yaml fragments. So, it would be beneficial for Murano to manipulate with YAML-constructs at the DSL level. If we use YAML notation, yaml-blocks become first-class citizens in the language, while in a regular python-like language they would be just formatted-strings. For example, look at this code snippet which generates a heat template to deploy an instance: <a href="http://paste.openstack.org/show/65725/" target="_blank">http://paste.openstack.org/show/65725/</a></div>
<div class="gmail_default" style="font-size:small">As you may see, the code on lines 7-11 is a Heat-template, seamlessly embedded inside Murano's workflow code. It has the variables right inside the template, and the Murano engine will substitute them with a user-specified (or workflow-computed) data</div>
<div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Another reason for YAML: the notation is very easy to extend: you'll just have to add some new predefined key and a handler for its value: the format will not be broken, so older code will run out of the box, and even the newer code will most probably run fine on the older parser (the unknown sections will simply be ignored). This will allow us to extend the language without making any breaking-changes. The regular languages do not provide this flexibility: they will have problems if detecting unrecognised lexems or constructs.</div>
<div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Last but not least: the perception. You are absolutely right when you say that this is actually a full programming language. However, we don't want to rush all its capabilities to unprepared developers. If some developer does not want any complexity, they may think about it as about some fancy configuration markup language: a declarative, heat-template-like header followed by a sequence of simple actions. And only if needed the power comes at your service: variable assignments, flow control, flexible data contracts, complex compositions, inheritance trees.. I can imagine a lot of scary stuff here J</div>
<div class="gmail_default" style="font-size:small">But at the same time, YAML's indent-based syntax will look familiar to python developers.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">
Yes, everything comes at cost, and yaml may seem a bit bulky at the first glance. But I believe that people will get used to it soon enough, and the benefits are really important. </div><div class="gmail_default" style="font-size:small">
<br></div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">I hope this answers your concern. We'll come up with more examples and ideas: this thing has just emerged, nothing is set in stone yet, I am actively seeking for feedback and ideas. So thanks a loot for your question, I really appreciate it.</div>
<div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small"><br></div></div><div class="gmail_extra"><br clear="all"><div><div dir="ltr"><font>--<br></font><div dir="ltr">
<font>Regards,<br>Alexander Tivelkov</font></div></div></div><div><div class="h5">
<br><br><div class="gmail_quote">On Fri, Feb 14, 2014 at 6:41 PM, Joshua Harlow <span dir="ltr"><<a href="mailto:harlowja@yahoo-inc.com" target="_blank">harlowja@yahoo-inc.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="auto">
<div>An honest question,</div>
<div><br>
</div>
<div>U are mentioning what appears to be the basis for a full programming language (variables, calling other workflows - similar to functions) but then u mention this is being stuffed into yaml. </div>
<div><br>
</div>
<div>Why? </div>
<div><br>
</div>
<div>It appears like u might as well spend the effort and define a grammar and simplistic language that is not stuffed inside yaml. Shoving one into yaml syntax seems like it gets u none of the benefits of syntax checking, parsing, validation (highlighting...)
and all the pain of yaml. </div>
<div><br>
</div>
<div>Something doesn't seem right about the approach of creating languages inside the yaml format (in a way it becomes like xsl, yet xsl at least has a spec and is well defined). </div>
<div><br>
</div>
<div>My 2 cents....<br>
<br>
Sent from my really tiny device...</div><div><div>
<div><br>
On Feb 14, 2014, at 7:22 PM, "Alexander Tivelkov" <<a href="mailto:ativelkov@mirantis.com" target="_blank">ativelkov@mirantis.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<div dir="ltr">
<div class="gmail_default"><span>
<p dir="ltr" style="font-size:small;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Hi folks,</span></p>
<br>
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span>
<p dir="ltr" style="font-size:small;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Murano matures, and we are getting more and more feedback from our early adopters. The overall reception is very positive,
but at the same time there are some complaints as well. By now the most significant complaint is is hard to write workflows for application deployment and maintenance.</span></p>
<p dir="ltr" style="font-size:small;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Current version of workflow definition markup really have some design drawbacks which limit its potential adoption. They
are caused by the fact that it was never intended for use for Application Catalog use-cases.</span></p>
<br>
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span>
<p dir="ltr" style="font-size:small;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">I'll briefly touch these drawbacks first:</span></p>
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span>
<p dir="ltr" style="margin-top:0pt;margin-bottom:0pt"></p>
<ol>
<li style="font-size:small;line-height:1.15"><span style="line-height:1.15;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Murano's workflow engine is actually a state machine, however the workflow markup
does not explicitly define the states and transitions.</span> </li><li><span style="background-color:transparent"><font color="#000000" face="Arial"><span style="font-size:15px;line-height:1.15;white-space:pre-wrap">There is no data isolation within any environment, which causes both potential security vulnerabilities and
unpredictable workflow </span><span style="font-size:15px;line-height:17px;white-space:pre-wrap">behaviours</span><span style="font-size:15px;line-height:1.15;white-space:pre-wrap">.</span></font></span>
</li><li style="font-size:small;line-height:1.15"><span style="line-height:1.15;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">There is no easy way to reuse the workflows and their related procedures between
several applications.</span> </li><li style="font-size:small;line-height:1.15"><span style="line-height:1.15;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">The markup uses JSONPath, which relies on Python’s 'eval' function. This is insecure
and has to be avoided. </span> </li><li style="font-size:small;line-height:1.15"><span style="line-height:1.15;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">5. The workflow markup is XML-based, which is not a common practice in the OpenStack
community.</span> </li></ol>
<p></p>
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span>
<p dir="ltr" style="font-size:small;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">So, it turns out that we have to design and implement a new workflow definition notation, which will not have any of the
issues mentioned above.</span></p>
<p dir="ltr" style="font-size:small;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">At the same time, it should still allow to fully specify the configuration of any third-party Application, its dependencies
with other Applications and define specific actions which are required for Application deployment, configuration and life cycle management.</span></p>
<p dir="ltr" style="font-size:small;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">This new notation should allow to do the following:
</span></p>
<br>
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span>
<ul style="font-size:small;margin-top:0pt;margin-bottom:0pt">
<li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;vertical-align:baseline;white-space:pre-wrap">List all the required configuration parameters and dependencies for a given application</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Validate user input and match it to the defined parameters</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Define specific deployment actions and their execution order</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Define behaviors to handle the events of changes in application’s environment</span></p>
</li></ul>
<br>
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span>
<p dir="ltr" style="font-size:small;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Also, it should satisfy the following requirements:</span></p>
<br>
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span>
<ul style="font-size:small;margin-top:0pt;margin-bottom:0pt">
<li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Minimize the amount of configuration for common application parts, i.e. reuse existing configuration
parts and add only difference specific to the application.</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Allow to use different deployment tools with using the same markup constructs. i.e. provide a high-level
abstraction on the underlying tools (heat, shell, chef, puppet etc)</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;vertical-align:baseline;white-space:pre-wrap">For security reasons it should NOT allow to execute arbitrary operations - i.e. should allow to run
only predefined set of meaningful configuration actions. </span></p>
</li></ul>
<br>
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><br>
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span>
<p dir="ltr" style="font-size:small;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">So, I would suggest to introduce a simple and domain specific notation which would satisfy these needs:</span></p>
<ul style="font-size:small;margin-top:0pt;margin-bottom:0pt">
<li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Application dependencies and configuration properties are defined declaratively, in a way similar
to how it is done in Heat templates.</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Each property has special constraints and rules, allowing to validate the input and applications relationship
within the environment.</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;vertical-align:baseline;white-space:pre-wrap">The workflows are defined in imperative way: as a sequence of actions or method calls. This may include
assigning data variables or calling the workflows of other applications.</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:disc;font-size:15px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="background-color:transparent;vertical-align:baseline;white-space:pre-wrap">All of these may be packaged in a YAML format. The example may look something like this [1]</span></p>
</li></ul>
<br>
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span>
<p dir="ltr" style="font-size:small;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">The final version may become a bit more complicated, but as the starting point this should look fine. I suggest to cover
this in more details on our next IRC meeting on Tuesday.</span></p>
<br>
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span>
<p dir="ltr" style="font-size:small;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Any feedback or suggestions are appreciated.</span></p>
<br>
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span><br>
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span>
<p dir="ltr" style="font-size:small;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial">[1]
<a href="https://etherpad.openstack.org/p/murano-new-dsl-example" target="_blank">https://etherpad.openstack.org/p/murano-new-dsl-example</a></span></p>
<br>
<span style="vertical-align:baseline;font-size:15px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span></span></div>
<div>
<div dir="ltr"><font>--<br>
</font>
<div dir="ltr"><font>Regards,<br>
Alexander Tivelkov</font></div>
</div>
</div>
</div>
</div>
</blockquote>
</div></div><blockquote type="cite">
<div><span>_______________________________________________</span><br>
<span>OpenStack-dev mailing list</span><br>
<span><a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a></span><br>
<span><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></span><br>
</div>
</blockquote>
</div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br><div dir="ltr"><span style="border-collapse:separate;color:rgb(0,0,0);font-family:'Times New Roman';font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span style="font-family:arial;font-size:small">Sincerely yours<br>
Stanislav (Stan) Lagun<br>Senior Developer<br>Mirantis</span></span><br><span style="border-collapse:separate;color:rgb(0,0,0);font-family:'Times New Roman';font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-size:medium"><span style="font-family:arial;font-size:small"><span style="font-size:10.0pt;font-family:"Arial","sans-serif"" lang="EN-US">35b/3, Vorontsovskaya
St.</span><br>Moscow, Russia<br>Skype: stanlagun<br><a href="http://www.mirantis.com/" target="_blank">www.mirantis.com</a><br><a href="mailto:slagun@mirantis.com" target="_blank">slagun@mirantis.com</a></span></span></div>
</div>