Open Stack

Hi Zane,

Thank you for your feedback. This is very important for us to find out how
our approach can be aligned with existing solutions. Let me try to explain
how we come up to specific solutions.


>It does seem really awkward to me (and not just because of all the
$signs), because it's duplicating >basically all of the functionality of
Heat. e.g. in MuranoPL you have:
>Properties:
>  name:
>    Contract: $.string().notNull()

>whereas in HOT this would be:

>parameters:
>  name:
>    type: string
>    constraints:
>      - length: {min: 1}

First of all parameters are the common entities in all different services.
Some of them use config files, some of them use yaml formats. I don't see
that this is a significant overlap with Heat. While Murano parameters can
be directly passed to Heat template generated, there is a major difference
between Murano parameters and Heat parameters. We use these parameters to
bind different entities together. The good example is here [1][2]

Properties:
  name:
    Contract: $.string().notNull()

  primaryController:
    Contract: $.class(PrimaryController).notNull()

  secondaryControllers:
    Contract: [$.class(SecondaryController).notNull()]

As you see, parameters here are used to bind different applications as
PrimaryController and Secondary controllers are different VMs with
different configurations.

Murano can't use Heat parameters for that as we need to bind
applications before Heat template is generated. If we even try to use
Heat for keeping such parameters it will lead to Murano parsing and
processing Heat templates which we want to avoid.


>Looking at e.g. [1], more  or less everything in here can be done already inside a Heat template, using get_f>ile and str_replace.


This definitely can be done inside Heat template, but these operations
(str_replace) are not specific to Heat. As we use these variables
before HOT template creation it is better to do string operations
inside the service then pass this to another service. As we can't
expose python functions for that we just wrap them in our DSL syntax
which is just a couple lines of Python code around pythons string
methods.


>It sounds like this is a DSL in which you write everything imperatively,
then it gets converted behind the >scenes into a declarative model in a
completely different language (but not using any of the advanced >features
of that language) and passed to Heat, which turns it back into a workflow
to execute. That seems >bizarre to me. Surely Murano should be focusing on
filling the gaps in Heat, rather than reimplementing it in >a different
paradigm?

We will be very happy to fill gaps in the HOT syntax to extend it. At the
same time, I need to mention, that the picture you drew does not perfectly
reflects what we are doing. Murano uses its own definition of application
which covers the specific aspects of application - actions and application
binding. As a part of application definition we store Heat snippets as
files. We don't hide anything from Heat under the Murano layer. All
features available in Heat can be used in Murano as user has an ability to
add Heat template to the application definition. As Heat main feature is to
manage different resource we pass this tasks to Heat. Murano does not do
any kind of infrastructure resource management outside of the Heat.

Heat template generation is just a first step in Application life. Murano
provides a way to define workflows for different aspects
of application which can be invoked by Heat during generation or out of
Heat by Mistral, Ceilometer and other services, including Murano.

>What I'm imagining here is something along the lines of:
>- Heat implements hooks to customise its workflows, as proposed in [2],
[3].
>- Deployments are defined declaratively using HOT syntax.
>- Workflows - to wrap the deployment operations, to customise the
deployment and to perform lifecycle >operations like backups - are defined
using a Mistral DSL (I assume this exists already? I haven't looked >into
it).
>- Murano provides a way to bundle the various workflow definitions, HOT
models, and other files into an >application package.

Murano is not focused on deployment itself. As soon as we have HOT Murano
is responsible to generate proper HOT template to make actual deployments.
As soon as HOT Software components are ready, application publisher will
use them in Heat templates.

>Can anybody enlighten me as to what features would be missing from this
that would warrant creating a >new programming language?

I see this in the following way - who will generate HOT template for my
complex multi-tier applications when I have only templates for components?
It looks like I will have to write a new template by myself understanding
all possible relation between components and their parameters. Then I have
to add some specific workflows to Mistral and somehow bind them to the
resources deployed by Heat. Then I have to understand what kind of
notifications to expect and somehow react on them or configure Mistral to
call some workflows. I think this was a motivation for a new language which
will reflect these specifics of applications and their bindings. Currently
Murano workflows are processed by Murano engine, but I expect to see in the
future, when Mistral is mature enough, Murano will just add corresponding
workflows as Mistral entities and will use it to do workflows execution.

I hope I cleared some of your questions. I don't say that Murano DSL is the
only solution. I probably miss something as it is not possible to know
everything. I will be glad to continue this discussion and find out the
right way of doing the tasks we are doing now in Murano.




[1] -
https://github.com/istalker2/MuranoDsl/blob/master/meta/com.mirantis.murano.services.windows.failoverCluster.FailoverCluster/manifest.yaml
[2] -
https://github.com/istalker2/MuranoDsl/blob/master/meta/com.mirantis.murano.services.windows.activeDirectory.ActiveDirectory/manifest.yaml


On Tue, Mar 18, 2014 at 10:35 AM, Zane Bitter <zbitter at redhat.com> wrote:

> (I added a couple of tags to the subject; hope this doesn't screw up
> anyone's threading.)
>
>
> On 09/03/14 16:26, Joshua Harlow wrote:
>
>> I'd be very interested in knowing the resource controls u plan to add.
>> Memory, CPU...
>>
>> I'm still trying to figure out where something like
>> https://github.com/istalker2/MuranoDsl/blob/master/meta/
>> com.mirantis.murano.demoApp.DemoInstance/manifest.yaml
>> would be beneficial, why not just spend effort sand boxing lua,
>> python... Instead of spending effort on creating a new language and then
>> having to sandbox it as well... Especially if u picked languages that
>> are made to be sandboxed from the start (not python)...
>>
>
> -1 to using a full-blown programming language like Python over a DSL,
> but...
>
>
>  Who is going to train people on muranoPL, write language books and
>> tutorials when the same amount of work has already been done for 10+
>> years for other languages....
>>
>> I fail to see where muranoPL is a DSL when it contains a full language
>> already with functions, objects, namespaces, conditionals... (what is
>> the domain of it?), maybe I'm just confused though (quite possible, haha).
>>
>
> ...I'm inclined to agree with this. Whenever you find yourself
> implementing a Turing-complete Object-Oriented DSL... well, you'd at least
> want to stop and think very carefully about whether you might have taken a
> wrong turn somewhere.
>
>
>  Does this not seem awkward to anyone else??
>>
>
> It does seem really awkward to me (and not just because of all the
> $signs), because it's duplicating basically all of the functionality of
> Heat. e.g. in MuranoPL you have:
>
> Properties:
>   name:
>     Contract: $.string().notNull()
>
> whereas in HOT this would be:
>
> parameters:
>   name:
>     type: string
>     constraints:
>       - length: {min: 1}
>
> In MuranoPL you reference it using "$this.name", vs. HOT using
> "{get_param: name}".
>
> Note that HOT (a) already exists in OpenStack, and (b) is being developed
> in conjunction with TOSCA folks to ensure easy translation to/from TOSCA
> Simple Profile YAML rendering.
>
> Looking at e.g. [1], more  or less everything in here can be done already
> inside a Heat template, using get_file and str_replace.
>
> It sounds like this is a DSL in which you write everything imperatively,
> then it gets converted behind the scenes into a declarative model in a
> completely different language (but not using any of the advanced features
> of that language) and passed to Heat, which turns it back into a workflow
> to execute. That seems bizarre to me. Surely Murano should be focusing on
> filling the gaps in Heat, rather than reimplementing it in a different
> paradigm?
>
> What I'm imagining here is something along the lines of:
> - Heat implements hooks to customise its workflows, as proposed in [2],
> [3].
> - Deployments are defined declaratively using HOT syntax.
> - Workflows - to wrap the deployment operations, to customise the
> deployment and to perform lifecycle operations like backups - are defined
> using a Mistral DSL (I assume this exists already? I haven't looked into
> it).
> - Murano provides a way to bundle the various workflow definitions, HOT
> models, and other files into an application package.
>
> Can anybody enlighten me as to what features would be missing from this
> that would warrant creating a new programming language?
>
> thanks,
> Zane.
>
> [1] https://github.com/istalker2/MuranoDsl/blob/master/meta/
> com.mirantis.murano.demoApp.DemoInstance/manifest.yaml
> [2] http://lists.openstack.org/pipermail/openstack-dev/2014-
> February/026329.html
> [3] http://lists.openstack.org/pipermail/openstack-dev/2014-
> March/030228.html
>
>  Sent from my really tiny device...
>>
>> On Mar 8, 2014, at 10:44 PM, "Stan Lagun" <slagun at mirantis.com
>> <mailto:slagun at mirantis.com>> wrote:
>>
>>  First of all MuranoPL is not a host but hosted language. It never
>>> intended to replace Python and if Python can do the job it is probably
>>> better than MuranoPL for that job.
>>> The problem with Python is that you cannot have Python code as a part
>>> of your DSL if you need to evaluate that DSL on server-side. Using
>>> Python's eval() is not secure. And you don't have enough control on
>>> what evaled code is allowed to do. MuranoPL on the contrary are fully
>>> sandboxed. You have absolute control over what functions/methods/APIs
>>> are exposed to DSL and DSL code can do nothing except for what it
>>> allowed to do. Besides you typically do want your DSL to be
>>> domain-specific so general-purpose language like Python can be
>>> suboptimal.
>>>
>>> I don't say MuranoPL is good for all projects. It has many
>>> Murano-specific things after all. In most cases you don't need all
>>> those OOP features that MuranoPL has. But the code organization, how
>>> it uses YAML, block structures and especially YAQL expressions can be
>>> of a great value to many projects
>>>
>>> For examples of MuranoPL classes you can browse
>>> https://github.com/istalker2/MuranoDsl/tree/master/meta folder. This
>>> is my private repository that I was using to develop PoC for MuranoPL
>>> engine. We are on the way to create production-quality implementation
>>> with unit-tests etc. in regular Murano repositories on stackforge
>>>
>>>
>>> On Sun, Mar 9, 2014 at 7:33 AM, Joshua Harlow <harlowja at yahoo-inc.com
>>> <mailto:harlowja at yahoo-inc.com>> wrote:
>>>
>>>     To continue from other thread....
>>>
>>>     """
>>>     Personally I believe that YAQL-based DSLs similar (but probably
>>>     simlet than) MuranoPL can be of a great value for many OpenStack
>>>     projects that have DSLs of different kind. Murano for App
>>>     Catatalog, Mistral for workflows, Heat for HOT templates,
>>>     Ceilometer for alarms & counter aggregation rules, Nova for
>>>     customizable resource scheduling and probably many more.
>>>     """
>>>
>>>     Isn't python a better host language for said DSLs than muranoPL? I
>>>     am still pretty weary of a DSL that starts to grow so many
>>>     features to encompass other DSLs since then it's not really a DSL
>>>     but a non-DSL, and we already have one that everyone is familiar
>>>     with (python).
>>>
>>>     Are there any good examples if muranoPL that I can look at that
>>>     take advantage of muranoPL classes, functions, namespaces which
>>>     can be compared to there python equivalents. Has such an
>>>     analysis/evaluation been done?
>>>
>>>     Sent from my really tiny device...
>>>     _______________________________________________
>>>     OpenStack-dev mailing list
>>>     OpenStack-dev at lists.openstack.org
>>>     <mailto:OpenStack-dev at lists.openstack.org>
>>>
>>>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>
>>>
>>> --
>>> Sincerely yours
>>> Stanislav (Stan) Lagun
>>> Senior Developer
>>> Mirantis
>>> 35b/3, Vorontsovskaya St.
>>> Moscow, Russia
>>> Skype: stanlagun
>>> www.mirantis.com <http://www.mirantis.com/>
>>> slagun at mirantis.com <mailto:slagun at mirantis.com>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> <mailto:OpenStack-dev at lists.openstack.org>
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Georgy Okrokvertskhov
Architect,
OpenStack Platform Products,
Mirantis
http://www.mirantis.com
Tel. +1 650 963 9828
Mob. +1 650 996 3284
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140318/e6546228/attachment.html>