[openstack-dev] MuranoPL questions?

Zane Bitter zbitter at redhat.com
Tue Mar 18 17:36:58 UTC 2014


On 18/03/14 08:01, Ruslan Kamaldinov wrote:
> Joshua, Clint,
>
> The only platform I'm aware about which fully supports true isolation and which
> has been used in production for this purpose is Java VM. I know people who
> developed systems for online programming competitions and really smart kids
> tried to break it without any luck :)
>
> Since we're speaking about Heat, Mistral and Murano DSLs and all of them need an
> execution engine. Do you think that JVM could become a host for that engine?

-2. Deploying OpenStack is hard enough already.

> JVM has a lot of potential:
> - it allows to fine-tune security manager to allow/disallow specific actions
> - it can execute a lot of programming languages - Python, Ruby, JS, etc
> - it has been used in production for this specific purpose for years
>
> But it also introduces another layer of complexity:
> - it's another component to deploy, configure and monitor
> - it's non-python, which means it should be supported by infra
> - we will need to run java service and potentially have some java code to
>    accept and process user code
>
>
> Thanks,
> Ruslan
>
> On Mon, Mar 17, 2014 at 10:40 PM, Joshua Harlow <harlowja at yahoo-inc.com> wrote:
>> So I guess this is similar to the other thread.
>>
>> http://lists.openstack.org/pipermail/openstack-dev/2014-March/030185.html
>>
>> I know that the way YQL has provided it could be a good example; where the
>> core DSL (the select queries and such) are augmented by the addition and
>> usage of JS, for example
>> http://developer.yahoo.com/yql/guide/yql-execute-examples.html#yql-execute-example-helloworld
>> (ignore that its XML, haha). Such usage already provides rate-limits and
>> execution-limits
>> (http://developer.yahoo.com/yql/guide/yql-execute-intro-ratelimits.html) and
>> afaik if something like what YQL is doing then u don't need to recreate
>> simialr features in your DSL (and then u also don't need to teach people
>> about a new language and syntax and ...)
>>
>> Just an idea (I believe lua offers similar controls/limits.., although its
>> not as popular of course as JS).
>>
>> From: Stan Lagun <slagun at mirantis.com>
>>
>> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
>> <openstack-dev at lists.openstack.org>
>> Date: Monday, March 17, 2014 at 3:59 AM
>>
>> To: "OpenStack Development Mailing List (not for usage questions)"
>> <openstack-dev at lists.openstack.org>
>> Subject: Re: [openstack-dev] MuranoPL questions?
>>
>> Joshua,
>>
>> Completely agree with you. We wouldn't be writing another language if we
>> knew how any of existing languages can be used for this particular purpose.
>> If anyone suggest such language and show how it can be used to solve those
>> issues DSL was designed to solve we will consider dropping MuranoPL. np
>>
>> Surely DSL hasn't stood the test of time. It just hasn't had a chance yet.
>> 100% of successful programming languages were in such position once.
>>
>> Anyway it is the best time to come with your suggestions. If you know how
>> exactly DSL can be replaced or improved we would like you to share
>>
>>
>> On Wed, Mar 12, 2014 at 2:05 AM, Joshua Harlow <harlowja at yahoo-inc.com>
>> wrote:
>>>
>>> I guess I might be a bit biased to programming; so maybe I'm not the
>>> target audience.
>>>
>>> I'm not exactly against DSL's, I just think that DSL's need to be really
>>> really proven to become useful (in general this applies to any language that
>>> 'joe' comp-sci student can create). Its not that hard to just make one, but
>>> the real hard part is making one that people actually like and use and
>>> survives the test of time. That's why I think its just nicer to use
>>> languages that have stood the test of time already (if we can), creating a
>>> new DSL (muranoPL seems to be slightly more than a DSL imho) means creating
>>> a new language that has not stood the test of time (in terms of lifetime,
>>> battle tested, supported over years) so that's just the concern I have.
>>>
>>> Of course we have to accept innovation and I hope that the DSL/s makes it
>>> easier/simpler, I just tend to be a bit more pragmatic maybe in this area.
>>>
>>> Here's hoping for the best! :-)
>>>
>>> -Josh
>>>
>>> From: Renat Akhmerov <rakhmerov at mirantis.com>
>>> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
>>> <openstack-dev at lists.openstack.org>
>>> Date: Monday, March 10, 2014 at 8:36 PM
>>> To: "OpenStack Development Mailing List (not for usage questions)"
>>> <openstack-dev at lists.openstack.org>
>>> Subject: Re: [openstack-dev] MuranoPL questions?
>>>
>>> Although being a little bit verbose it makes a lot of sense to me.
>>>
>>> @Joshua,
>>>
>>> Even assuming Python could be sandboxed and whatever else that's needed to
>>> be able to use it as DSL (for something like Mistral, Murano or Heat) is
>>> done  why do you think Python would be a better alternative for people who
>>> don't know neither these new DSLs nor Python itself. Especially, given the
>>> fact that Python has A LOT of things that they'd never use. I know many
>>> people who have been programming in Python for a while and they admit they
>>> don't know all the nuances of Python and actually use 30-40% of all of its
>>> capabilities. Even not in domain specific development. So narrowing a
>>> feature set that a language provides and limiting it to a certain domain
>>> vocabulary is what helps people solve tasks of that specific domain much
>>> easier and in the most expressive natural way. Without having to learn tons
>>> and tons of details that a general purpose language (GPL, hah :) ) provides
>>> (btw, the reason to write thick books).
>>>
>>> I agree with Stan, if you begin to use a technology you'll have to learn
>>> something anyway, be it TaskFlow API and principles or DSL. Well-designed
>>> DSL just encapsulates essential principles of a system it is used for. By
>>> learning DSL you're leaning the system itself, as simple as that.
>>>
>>> Renat Akhmerov
>>> @ Mirantis Inc.
>>>
>>>
>>>
>>> On 10 Mar 2014, at 05:35, Stan Lagun <slagun at mirantis.com> wrote:
>>>
>>>> I'd be very interested in knowing the resource controls u plan to add.
>>>> Memory, CPU...
>>> We haven't discussed it yet. Any suggestions are welcomed
>>>
>>>> I'm still trying to figure out where something like
>>>> https://github.com/istalker2/MuranoDsl/blob/master/meta/com.mirantis.murano.demoApp.DemoInstance/manifest.yaml
>>>> would be beneficial, why not > just spend effort sand boxing lua, python...
>>>> Instead of spending effort on creating a new language and then having to
>>>> sandbox it as well... Especially if u picked languages that are made to be
>>>> sandboxed from the start (not python)...
>>>
>>> 1. See my detailed answer in Mistral thread why haven't we used any of
>>> those languages. There are many reasons besides sandboxing.
>>>
>>> 2. You don't need to sandbox MuranoPL. Sandboxing is restricting some
>>> operations. In MuranoPL ALL operations (including operators in expressions,
>>> functions, methods etc.) are just those that you explicitly provided. So
>>> there is nothing to restrict. There are no builtins that throw
>>> AccessViolationError
>>>
>>> 3. Most of the value of MuranoPL comes not form the workflow code but from
>>> class declarations. In all OOP languages classes are just a convenient to
>>> organize your code. There are classes that represent real-life objects and
>>> classes that are nothing more than data-structures, DTOs etc. In Murano
>>> classes in MuranoPL are deployable entities like Heat resources application
>>> components, services etc. In dashboard UI user works with those entities. He
>>> (in UI!) creates instances of those classes, fills their property values,
>>> binds objects together (assigns on object to a property of another). And
>>> this is done without even a single MuranoPL line being executed! That is
>>> possible because everything in MuranoPL is a subject to declaration and
>>> because it is just plain YAML anyone can easily extract those declarations
>>> from MuranoPL classes.
>>> Now suppose it was Python instead of MuranoPL. Then you would have to
>>> parse *.py files to get list of declared classes (without executing
>>> anything). Suppose that you managed to solve this somehow. Probably you
>>> wrote regexp that finds all class declarations it text files. Are you done?
>>> No! There are no properties (attributes) declarations in Python. You cannot
>>> infer all possible class attributes just from class declaration. You can try
>>> to parse __init__ method to find attribute initialization code but then you
>>> find that you cannot infer property types. You would not know what values
>>> does class expect in those attributes, what are constraints etc. Besides
>>> there are plenty of ways to fool such naive algorithms. Classes can be
>>> created using type() built-in function without declarations at all.
>>> Attributes may be initialized anywhere. Everything can be made in runtime.
>>> As you can see Python is not good for the task. Ans all this true for all
>>> of the languages you mentioned. The reason is fundamental - all of those
>>> languages are dynamic.
>>> So maybe you can take some statically-typed language like C++ or Java to
>>> do the job? Again, no! There are other problems with those type of
>>> languages. Have you ever seen statically-typed embeddable language? Those
>>> language require compilation and linker, binary distribution of classes,
>>> handling of versioning problems, dependencies on 3-rd party libraries etc.
>>> This is the hell you don't want to step in. But even in statically typed
>>> languages there are no contracts. You know the list of properties and their
>>> types. But knowing that class foo has property bar of type int doesn't give
>>> a clue if value if 10 is okay for that property.
>>>
>>> Now I admit that we could probably spend a year of development and try to
>>> implement this on type Java with custom wrapping and heavy usage of
>>> decorators, embed somehow JVM into python process, do all possible
>>> sandboxing and then get a Frankenstein that nobody in OpenStack community
>>> would like or accept.
>>>
>>> My point here is that there is no language in the world that was designed
>>> for external object composition that is in Murano. No language in the world
>>> was design so that there would be huge repository of classes but no single
>>> control point, no main project, no trust between different classes. There is
>>> no language with tree-like heap organization (each object has single owner
>>> object). This what makes MuranoPL domain-specific and not deployments. There
>>> are no such languages not because nobody that smart to create them but
>>> because nobody needed to do so. Because MuranoPL is domain-specific unlike
>>> Python or Java.
>>>
>>> MuranoPL does contain many constructs that can be found in general-purpose
>>> languages but that doesn't make it general-purpose!
>>>
>>>
>>>> Who is going to train people on muranoPL, write language books and
>>>> tutorials when the same amount of work has already been done for 10+ years
>>>> for other languages....
>>> Let me say that most of MuranoPL audience are bash-developers that know
>>> Lua to the same level they know MuranoPL :)
>>>
>>> In Mirantis we deed interesting experiment. We asked several deployment
>>> engineers that were not involved in MuranoPL creation and knew nothing about
>>> it beforehand to write some application manifests using MuranoPL. We gave
>>> them 10-minute introduction, link to DSL documentation blueprint and
>>> examples in my github repository (you've seen both of them). The link above
>>> to DemoInstance manifest points to the class that was created by one of
>>> those guys. It took them several days to get used to it. But they never even
>>> used Python before! And the feedback was very positive.
>>>
>>> I don't really see a problem writing tutorials or even a book. Learning
>>> new framework like TaskFlow is nowhere easier than learning DSL. Anyway you
>>> need to learn something to start using new technology. There are hundreds of
>>> DSL languages that are used in production and I don't feel like having one
>>> more language is really a problem
>>>
>>>
>>>
>>>
>>> On Mon, Mar 10, 2014 at 12:26 AM, Joshua Harlow <harlowja at yahoo-inc.com>
>>> wrote:
>>>>
>>>> I'd be very interested in knowing the resource controls u plan to add.
>>>> Memory, CPU...
>>>>
>>>> I'm still trying to figure out where something like
>>>> https://github.com/istalker2/MuranoDsl/blob/master/meta/com.mirantis.murano.demoApp.DemoInstance/manifest.yaml
>>>> would be beneficial, why not just spend effort sand boxing lua, python...
>>>> Instead of spending effort on creating a new language and then having to
>>>> sandbox it as well... Especially if u picked languages that are made to be
>>>> sandboxed from the start (not python)...
>>>>
>>>> Who is going to train people on muranoPL, write language books and
>>>> tutorials when the same amount of work has already been done for 10+ years
>>>> for other languages....
>>>>
>>>> I fail to see where muranoPL is a DSL when it contains a full language
>>>> already with functions, objects, namespaces, conditionals... (what is the
>>>> domain of it?), maybe I'm just confused though (quite possible, haha).
>>>>
>>>> Does this not seem awkward to anyone else??
>>>>
>>>> Sent from my really tiny device...
>>>>
>>>> On Mar 8, 2014, at 10:44 PM, "Stan Lagun" <slagun at mirantis.com> wrote:
>>>>
>>>> First of all MuranoPL is not a host but hosted language. It never
>>>> intended to replace Python and if Python can do the job it is probably
>>>> better than MuranoPL for that job.
>>>> The problem with Python is that you cannot have Python code as a part of
>>>> your DSL if you need to evaluate that DSL on server-side. Using Python's
>>>> eval() is not secure. And you don't have enough control on what evaled code
>>>> is allowed to do. MuranoPL on the contrary are fully sandboxed. You have
>>>> absolute control over what functions/methods/APIs are exposed to DSL and DSL
>>>> code can do nothing except for what it allowed to do. Besides you typically
>>>> do want your DSL to be domain-specific so general-purpose language like
>>>> Python can be suboptimal.
>>>>
>>>> I don't say MuranoPL is good for all projects. It has many
>>>> Murano-specific things after all. In most cases you don't need all those OOP
>>>> features that MuranoPL has. But the code organization, how it uses YAML,
>>>> block structures and especially YAQL expressions can be of a great value to
>>>> many projects
>>>>
>>>> For examples of MuranoPL classes you can browse
>>>> https://github.com/istalker2/MuranoDsl/tree/master/meta folder. This is my
>>>> private repository that I was using to develop PoC for MuranoPL engine. We
>>>> are on the way to create production-quality implementation with unit-tests
>>>> etc. in regular Murano repositories on stackforge
>>>>
>>>>
>>>> On Sun, Mar 9, 2014 at 7:33 AM, Joshua Harlow <harlowja at yahoo-inc.com>
>>>> wrote:
>>>>>
>>>>> To continue from other thread....
>>>>>
>>>>> """
>>>>> Personally I believe that YAQL-based DSLs similar (but probably simlet
>>>>> than) MuranoPL can be of a great value for many OpenStack projects that have
>>>>> DSLs of different kind. Murano for App Catatalog, Mistral for workflows,
>>>>> Heat for HOT templates, Ceilometer for alarms & counter aggregation rules,
>>>>> Nova for customizable resource scheduling and probably many more.
>>>>> """
>>>>>
>>>>> Isn't python a better host language for said DSLs than muranoPL? I am
>>>>> still pretty weary of a DSL that starts to grow so many features to
>>>>> encompass other DSLs since then it's not really a DSL but a non-DSL, and we
>>>>> already have one that everyone is familiar with (python).
>>>>>
>>>>> Are there any good examples if muranoPL that I can look at that take
>>>>> advantage of muranoPL classes, functions, namespaces which can be compared
>>>>> to there python equivalents. Has such an analysis/evaluation been done?
>>>>>
>>>>> Sent from my really tiny device...
>>>>> _______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-dev at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Sincerely yours
>>>> Stanislav (Stan) Lagun
>>>> Senior Developer
>>>> Mirantis
>>>> 35b/3, Vorontsovskaya St.
>>>> Moscow, Russia
>>>> Skype: stanlagun
>>>> www.mirantis.com
>>>> slagun at mirantis.com
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>
>>>
>>>
>>> --
>>> Sincerely yours
>>> Stanislav (Stan) Lagun
>>> Senior Developer
>>> Mirantis
>>> 35b/3, Vorontsovskaya St.
>>> Moscow, Russia
>>> Skype: stanlagun
>>> www.mirantis.com
>>> slagun at mirantis.com
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>> --
>> Sincerely yours
>> Stanislav (Stan) Lagun
>> Senior Developer
>> Mirantis
>> 35b/3, Vorontsovskaya St.
>> Moscow, Russia
>> Skype: stanlagun
>> www.mirantis.com
>> slagun at mirantis.com
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>




More information about the OpenStack-dev mailing list