<div dir="ltr">Hi Thomas,<div><br></div><div>I think we went to the second loop of the discussion about generic language concepts. Murano does not use a new language for the sole purpose of having parameters, constraints and polymorphism. These are generic concepts which are common for different languages, so keeping arguing about these generic concepts is just like a holy war like Python vs. C. Keeping these arguments is just like to say that we don't need Python as functions and parameters already exists in C which is used under the hood in Python.</div>
<div><br></div><div>Yes Murano DSL have some generic concepts similar to HOT. I think this is a benefit as user will see the familiar syntax constructions and it will be a lower threshold for him to start using Murano DSL.</div>
<div><br></div><div>In a simplified view Murano uses DSL for application definition to solve several particular problems:</div><div>a) control UI rendering of Application Catalog</div><div>b) control HOT template generation</div>
<div><br></div><div>These aspects are not covered in HOT and probably should not be covered. I don't like an idea of expressing HOT template generation in HOT as it sounds like a creation another Lisp like language :-)</div>
<div><br></div><div>I don't think that your statement that most of the people in the community are against new DSL is a right summary. There are some disagreements how it should look like and what are the goals. You will be probably surprised but we are not the first who use DSL for HOT templates generation. Here is an e-mail thread right about Ruby based DSL used in IBM for the same purpose: <a href="http://lists.openstack.org/pipermail/openstack-dev/2014-February/026606.html">http://lists.openstack.org/pipermail/openstack-dev/2014-February/026606.html</a></div>
<div><br></div>The term "Orchestration" is quite generic. Saying that orchestration should be Heat job sounds like a well know Henry Ford's phrase "You can have any colour as long as it's black.". <br>
I think this is again a lack of understanding of the difference between Orchestration program and Heat project. There are many aspects of Orchestration and OpenStack has the Orchestration program for the projects which are focused on some aspects of orchestration. Heat is one of the project inside Orchestration program but it does not mean that Heat should cover everything. That is why we discussed in this thread how workflows aspects should be aligned and how they should be placed into this Orchestration program.<div>
<br></div><div>Thanks</div><div>Georgy<br><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Mar 24, 2014 at 8:28 AM, Dmitry <span dir="ltr"><<a href="mailto:meytin@gmail.com" target="_blank">meytin@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">MuranoPL supposed to provide a solution for the real needs to manage services in the centralized manner and to allow cloud provider customers to create their own services.<div>
The application catalog similar to AppDirect (<a href="http://www.appdirect.com/" target="_blank">www.appdirect.com</a>) natively supported by OpenStack is a huge step forward.</div><div>Think about Amazon which provides different services for the different needs: Amazon Cloud Formation, Amazon OpsWorks and Amazon Beanstalk.</div>
<div>Following the similar logic (which is fully makes sense for me), OpenStack should provide resource reservation and orchestration (Heat and Climate), Application Catalog (Murano) and PaaS (Solum).</div><div>Every project can live in harmony with other and contribute for the cloud service provider service completeness.</div>
<div>This is my opinion and i would happy to use Murano in our internal solution.</div></div><br>_______________________________________________<br>OpenStack-dev mailing list<br><a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br><br></blockquote></div><br><br clear="all"><div><br></div>
-- <br><div dir="ltr"><font color="#999999">Georgy Okrokvertskhov<br>Architect,<br>OpenStack Platform Products,<br>Mirantis<br><a href="http://www.mirantis.com/" target="_blank">http://www.mirantis.com</a><br>Tel. +1 650 963 9828<br>
Mob. +1 650 996 3284</font></div></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Mar 24, 2014 at 5:13 AM, Thomas Herve <span dir="ltr"><<a href="mailto:thomas.herve@enovance.com" target="_blank">thomas.herve@enovance.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Stan,<br>
<br>
Comments inline.<br>
<div class=""><br>
> Zane,<br>
><br>
> I appreciate your explanations on Heat/HOT. This really makes sense.<br>
> I didn't mean to say that MuranoPL is better for Heat. Actually HOT is good<br>
> for Heat's mission. I completely acknowledge it.<br>
> I've tried to avoid comparison between languages and I'm sorry if it felt<br>
> that way. This is not productive as I don't offer you to replace HOT with<br>
> MuranoPL (although I believe that certain elements of MuranoPL syntax can be<br>
> contributed to HOT and be valuable addition there). Also people tend to<br>
> protect what they have developed and invested into and to be fair this is<br>
> what we did in this thread to great extent.<br>
><br>
> What I'm trying to achieve is that you and the rest of Heat team understand<br>
> why it was designed the way it is. I don't feel that Murano can become<br>
> full-fledged member of OpenStack ecosystem without a bless from Heat team.<br>
> And it would be even better if we agree on certain design, join our efforts<br>
> and contribute to each other for sake of Orchestration program.<br>
<br>
</div>Note that I feel that most people outside of the Murano project are against the idea of using a DSL. You should feel that it could block the integration in OpenStack.<br>
<div><div class="h5"><br>
> I'm sorry for long mail texts written in not-so-good English and appreciate<br>
> you patience reading and answering them.<br>
><br>
> Having said that let me step backward and explain our design decisions.<br>
><br>
> Cloud administrators are usually technical guys that are capable of learning<br>
> HOT and writing YAML templates. They know exact configuration of their cloud<br>
> (what services are available, what is the version of OpenStack cloud is<br>
> running) and generally understands how OpenStack works. They also know about<br>
> software they intent to install. If such guy wants to install Drupal he<br>
> knows exactly that he needs HOT template describing Fedora VM with Apache +<br>
> PHP + MySQL + Drupal itself. It is not a problem for him to write such HOT<br>
> template.<br>
><br>
> Note that such template would be designed for very particular configuration.<br>
> There are hundreds of combinations that may be used to install that Drupal -<br>
> use RHEL/Windows/etc instead of Fedora, use ngnix/IIS/etc instead of Apache,<br>
> use FastCGI instead of mod_php, PostgreSQL instead of MySQL. You may choose<br>
> to have all software on single VM or have one VM for database and another<br>
> for Drupal. There are also constraints to those combinations. For example<br>
> you cannot have Fedora + IIS on the same VM. You cannot have Apache and<br>
> Drupal on different VMs.<br>
><br>
> So the HOT template represent fixed combination of those software components.<br>
> HOT may have input parameters like "username" or "dbImageName" but the<br>
> overall structure of template is fixed. You cannot have template that choses<br>
> whether to use Windows or Linux based on parameter value. You cannot write<br>
> HOT that accepts number of instances it allowed to create and then decide<br>
> what would be installed on each of them. This is just not needed for Heat<br>
> users.<br>
><br>
> With Murano the picture looks the opposite. Typical Murano user is a guy who<br>
> bought account from cloud hosting vendor (cloud operator) and want to run<br>
> some software in the cloud. He may not even be aware that it is OpenStack.<br>
> He knows nothing about programming in general and Heat in particular. He<br>
> doesn't want to write YAMLs. He may not know how exactly Drupal is installed<br>
> and what components it consists of.<br>
<br>
</div></div>So that's where I want to make a first stop. If your primary user is not a developer, there is no reason to introduce a DSL for security reasons. The provider can trust the code he writes, and there is no need to create a dedicated language.<br>
<div><div class="h5"><br>
> So what he does is he goes to his cloud (Murano) dashboard, browses through<br>
> application catalog, finds Drupal and drags it onto his environment board<br>
> (think like Visio-style designer). He can stop at this point, click "deploy"<br>
> button and the system will deploy Drupal. In another words the system (or<br>
> maybe better to say cloud operator or application developer) decides what<br>
> set of components is going to be installed (like 1 Fedora VM for MySQL and 1<br>
> CentOS VM for Apache-PHP-Drupal). But user may decide he wants to customize<br>
> his environment. He digs down and sees that Drupal requires database<br>
> instance and the default is MySQL. He clicks on a button to see what are<br>
> other options available for that role.<br>
><br>
> In Heat HOT developer is the user. But in Murano those are completely<br>
> different roles. There are developers that write application definitions<br>
> (that is DSL code) and there are end users who compose environments from<br>
> those applications (components). Application developers may have nothing to<br>
> do with particular cloud their application deployed on. As for Drupal<br>
> application the developer knows that Drupal can be run with MySQL or<br>
> PostgreSQL. But there may be many compatible implementations of those DBMSes<br>
> - Galera MySQL, TroveMySQL, MMM MySQL etc. So to get a list of what<br>
> components can be placed in a database role Murano needs to look at all<br>
> applications in Application Catalog and find which of them are compatible<br>
> with MySQL and PostgreSQL so that user could choose what implementation is<br>
> better suits his needs (trade performance for high-availability etc.).<br>
><br>
> User can go deeper and to decide that he wants that MySQL instance (this can<br>
> be 1 or more VMs depending on implementation) to be shared between Drupal<br>
> and another application in that environment (say WordPress). He can go even<br>
> deeper to VM level and decide that he wants to have WordPress, Drupal,<br>
> Apcache and slave MySQL node on one VM and MySQL master node on another.<br>
><br>
> The ultimate goal is to create sort of construction kit for applications.<br>
> Cloud LEGO. And let users build environments of any complexity from small<br>
> components while providing reasonable default so that one-click deployment<br>
> would also be possible. And such system to be useful the design process need<br>
> to be guided and driven by UI. System must know what combination of<br>
> components are possible and what are not and not let user create Microsoft<br>
> IIS hosted on Fedora.<br>
<br>
</div></div>Those goals are really nice and I'd like to have something like that in OpenStack.<br>
<div class=""><br>
> This leads us to following design decisions:<br>
><br>
> a. We need a DSL that would allow describing those small components<br>
> independently in such a way that the environment could be composed from<br>
> components written by different people that are not aware of each other<br>
<br>
</div>You somewhat start by the end here. You haven't convinced us that you need a DSL.<br>
<div class=""><br>
> b. Components need to have properties (parameters in Heat terms). Properties<br>
> may be of different types (strings, numbers, booleans etc.). There can be<br>
> constraints on property values ("not null", "length > 5").<br>
<br>
</div>We have those constraints in Heat, they don't require imperative constructs.<br>
<div class=""><br>
> c. Components may reference (require, make use of) other components (Drupal<br>
> uses MySQL, hosted on Apache etc). There can be constraints for those<br>
> references.<br>
<br>
</div>Present in HOTs.<br>
<div class=""><br>
> d. Constraints for both properties and references can be *very* complex and<br>
> indirect. Like "IIS can be deployed on any VM who's image is known to be<br>
> Windows with version >= WinServer2003". Or "my app requires 10 to 20 Windows<br>
> instances that are all member of the same ActiveDirectory domain". Or "I<br>
> have 2 properties: minInstanceCount and maxInstanceCount and constraint<br>
> maxInstaceCount >= minInstanceCount". So we need a language to express<br>
> conditions of any complexity.<br>
<br>
</div>We support pluggable constraints in Heat, that you can write in Python.<br>
<div class=""><br>
> e. Both properties and references must be declared in a way that would make<br>
> possible for UI dashboard to discover their existence, ask user appropriate<br>
> input and validate the constraints on the fly.<br>
<br>
</div>Possible as well.<br>
<div class=""><br>
> f. Properties and especially references can have complex structure (lists,<br>
> maps, combination of them) rather than plain scalars. For example<br>
> ActiveDirectory consists of (e.g. references) *list* of domain controllers.<br>
> Constraints must respect that and be capable of validating things like<br>
> len(list) >= 2, all elements in list have some predicate true, there is at<br>
> least one element with desired set of properties etc.<br>
<br>
</div>If not possible, HOT should be extended to support this.<br>
<div class=""><br>
> g. Language need to have some sort mechanism to express polymorphism. E,g,<br>
> GaleraMySQL *is* MySQL (and thus can be used anywhere where "just MySQL" can<br>
> be). This also implies ability to have declare interfaces (class contracts)<br>
> and ability to implement those interface to that it would be possible that<br>
> two components are conform to the same interface.<br>
<br>
</div>You don't need to have polymorphism for this. I'm mostly familiar with the mechanism used by Juju, where you *declare* an interface. It's basically metadata, and it's much simpler.<br>
<div class=""><br>
> h. If we want components to be reusable and easy to write we need to have<br>
> ability to one component to inherit/extend another. Inheritance implies<br>
> ability to override and customize some methods<br>
><br>
> i. We don't want to do deployment ourself. We don't intend to replace Heat,<br>
> Puppet etc. So we need a way to construct Heat templates that will do the<br>
> job. We need to map component's properties to<br>
> HOT parameters. This requires a language to express those bindings<br>
><br>
> j. Because there can be lists of references (or properties) it should be<br>
> possible to express something like<br>
> for(domain_controller in self.controllers):<br>
> hot_template.insert(resources_for_particular_controller)<br>
><br>
> or<br>
><br>
> if self.deploy_on_windows:<br>
> generate_windows_hot_template()<br>
> else:<br>
> generate_linux_hot_template()<br>
><br>
> or<br>
><br>
> for i in range(self.instance_count):<br>
> hot_template.insert(generate_instance())<br>
<br>
</div>CloudFormation recently added support for conditionals in templates. I'm not a super fan of it, but we could presumably have those in HOT as well.<br>
<div class=""><br>
> k. Need to provide most simple method to compose stack from pre-made HOT<br>
> snippets and avoid dynamic generation when its not really needed<br>
><br>
> l. Need to have some standard format to represent view of the world that is<br>
> what was designed in dashboard. In Murano it is JSON that describes object<br>
> graph with object property values and references to another objects. This is<br>
> somewhat similar to simple HOT template but with hierarchical structure<br>
><br>
> I think at this point it becomes obvious that above describes object-oriented<br>
> Turing-complete language. And the most important property of that language<br>
> is that *everything* is declared in a way useful to be accessed from<br>
> dashboard/app catalog. You not often see properties/references/constraints<br>
> declared in dynamic language. I've answered several times already why this<br>
> language cannot be JavaScript or Lua. And this is just on the surface. There<br>
> are many things under the hood that makes MuranoPL stand out from all<br>
> embeddable languages I know and thats for a good reason.<br>
><br>
> And I didn't even mentioned ALM aspects, orchestration, and many many other<br>
> things that go far beyond what can be expressed by HOT-style DSL<br>
<br>
</div>Well orchestration should be Heat's job.<br>
<br>
What I can say is that I'm not convinced. The only use-case for a DSL would be if you have to upload user-written code, but what you mentioned is a Web interface, where the user doesn't use the DSL, and the cloud provider is the developer. There is no reason in this case to have a secure environment for the code.<br>
<br>
Cheers,<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Thomas<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><font color="#999999"><span style="background-color:rgb(255,255,255)">Georgy Okrokvertskhov<br>
Architect,<br><span style="font-family:arial;font-size:small">OpenStack Platform Products,</span><br>
Mirantis</span><br>
<a href="http://www.mirantis.com/" target="_blank">http://www.mirantis.com</a><br>
Tel. +1 650 963 9828<br>
Mob. +1 650 996 3284</font><br></div>
</div>