Open Stack

Joshua, Clint,

The only platform I'm aware about which fully supports true isolation and which
has been used in production for this purpose is Java VM. I know people who
developed systems for online programming competitions and really smart kids
tried to break it without any luck :)

Since we're speaking about Heat, Mistral and Murano DSLs and all of them need an
execution engine. Do you think that JVM could become a host for that engine?

JVM has a lot of potential:
- it allows to fine-tune security manager to allow/disallow specific actions
- it can execute a lot of programming languages - Python, Ruby, JS, etc
- it has been used in production for this specific purpose for years

But it also introduces another layer of complexity:
- it's another component to deploy, configure and monitor
- it's non-python, which means it should be supported by infra
- we will need to run java service and potentially have some java code to
  accept and process user code


Thanks,
Ruslan

On Mon, Mar 17, 2014 at 10:40 PM, Joshua Harlow <harlowja at yahoo-inc.com> wrote:
> So I guess this is similar to the other thread.
>
> http://lists.openstack.org/pipermail/openstack-dev/2014-March/030185.html
>
> I know that the way YQL has provided it could be a good example; where the
> core DSL (the select queries and such) are augmented by the addition and
> usage of JS, for example
> http://developer.yahoo.com/yql/guide/yql-execute-examples.html#yql-execute-example-helloworld
> (ignore that its XML, haha). Such usage already provides rate-limits and
> execution-limits
> (http://developer.yahoo.com/yql/guide/yql-execute-intro-ratelimits.html) and
> afaik if something like what YQL is doing then u don't need to recreate
> simialr features in your DSL (and then u also don't need to teach people
> about a new language and syntax and ...)
>
> Just an idea (I believe lua offers similar controls/limits.., although its
> not as popular of course as JS).
>
> From: Stan Lagun <slagun at mirantis.com>
>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev at lists.openstack.org>
> Date: Monday, March 17, 2014 at 3:59 AM
>
> To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev at lists.openstack.org>
> Subject: Re: [openstack-dev] MuranoPL questions?
>
> Joshua,
>
> Completely agree with you. We wouldn't be writing another language if we
> knew how any of existing languages can be used for this particular purpose.
> If anyone suggest such language and show how it can be used to solve those
> issues DSL was designed to solve we will consider dropping MuranoPL. np
>
> Surely DSL hasn't stood the test of time. It just hasn't had a chance yet.
> 100% of successful programming languages were in such position once.
>
> Anyway it is the best time to come with your suggestions. If you know how
> exactly DSL can be replaced or improved we would like you to share
>
>
> On Wed, Mar 12, 2014 at 2:05 AM, Joshua Harlow <harlowja at yahoo-inc.com>
> wrote:
>>
>> I guess I might be a bit biased to programming; so maybe I'm not the
>> target audience.
>>
>> I'm not exactly against DSL's, I just think that DSL's need to be really
>> really proven to become useful (in general this applies to any language that
>> 'joe' comp-sci student can create). Its not that hard to just make one, but
>> the real hard part is making one that people actually like and use and
>> survives the test of time. That's why I think its just nicer to use
>> languages that have stood the test of time already (if we can), creating a
>> new DSL (muranoPL seems to be slightly more than a DSL imho) means creating
>> a new language that has not stood the test of time (in terms of lifetime,
>> battle tested, supported over years) so that's just the concern I have.
>>
>> Of course we have to accept innovation and I hope that the DSL/s makes it
>> easier/simpler, I just tend to be a bit more pragmatic maybe in this area.
>>
>> Here's hoping for the best! :-)
>>
>> -Josh
>>
>> From: Renat Akhmerov <rakhmerov at mirantis.com>
>> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
>> <openstack-dev at lists.openstack.org>
>> Date: Monday, March 10, 2014 at 8:36 PM
>> To: "OpenStack Development Mailing List (not for usage questions)"
>> <openstack-dev at lists.openstack.org>
>> Subject: Re: [openstack-dev] MuranoPL questions?
>>
>> Although being a little bit verbose it makes a lot of sense to me.
>>
>> @Joshua,
>>
>> Even assuming Python could be sandboxed and whatever else that's needed to
>> be able to use it as DSL (for something like Mistral, Murano or Heat) is
>> done  why do you think Python would be a better alternative for people who
>> don't know neither these new DSLs nor Python itself. Especially, given the
>> fact that Python has A LOT of things that they'd never use. I know many
>> people who have been programming in Python for a while and they admit they
>> don't know all the nuances of Python and actually use 30-40% of all of its
>> capabilities. Even not in domain specific development. So narrowing a
>> feature set that a language provides and limiting it to a certain domain
>> vocabulary is what helps people solve tasks of that specific domain much
>> easier and in the most expressive natural way. Without having to learn tons
>> and tons of details that a general purpose language (GPL, hah :) ) provides
>> (btw, the reason to write thick books).
>>
>> I agree with Stan, if you begin to use a technology you'll have to learn
>> something anyway, be it TaskFlow API and principles or DSL. Well-designed
>> DSL just encapsulates essential principles of a system it is used for. By
>> learning DSL you're leaning the system itself, as simple as that.
>>
>> Renat Akhmerov
>> @ Mirantis Inc.
>>
>>
>>
>> On 10 Mar 2014, at 05:35, Stan Lagun <slagun at mirantis.com> wrote:
>>
>> > I'd be very interested in knowing the resource controls u plan to add.
>> > Memory, CPU...
>> We haven't discussed it yet. Any suggestions are welcomed
>>
>> > I'm still trying to figure out where something like
>> > https://github.com/istalker2/MuranoDsl/blob/master/meta/com.mirantis.murano.demoApp.DemoInstance/manifest.yaml
>> > would be beneficial, why not > just spend effort sand boxing lua, python...
>> > Instead of spending effort on creating a new language and then having to
>> > sandbox it as well... Especially if u picked languages that are made to be
>> > sandboxed from the start (not python)...
>>
>> 1. See my detailed answer in Mistral thread why haven't we used any of
>> those languages. There are many reasons besides sandboxing.
>>
>> 2. You don't need to sandbox MuranoPL. Sandboxing is restricting some
>> operations. In MuranoPL ALL operations (including operators in expressions,
>> functions, methods etc.) are just those that you explicitly provided. So
>> there is nothing to restrict. There are no builtins that throw
>> AccessViolationError
>>
>> 3. Most of the value of MuranoPL comes not form the workflow code but from
>> class declarations. In all OOP languages classes are just a convenient to
>> organize your code. There are classes that represent real-life objects and
>> classes that are nothing more than data-structures, DTOs etc. In Murano
>> classes in MuranoPL are deployable entities like Heat resources application
>> components, services etc. In dashboard UI user works with those entities. He
>> (in UI!) creates instances of those classes, fills their property values,
>> binds objects together (assigns on object to a property of another). And
>> this is done without even a single MuranoPL line being executed! That is
>> possible because everything in MuranoPL is a subject to declaration and
>> because it is just plain YAML anyone can easily extract those declarations
>> from MuranoPL classes.
>> Now suppose it was Python instead of MuranoPL. Then you would have to
>> parse *.py files to get list of declared classes (without executing
>> anything). Suppose that you managed to solve this somehow. Probably you
>> wrote regexp that finds all class declarations it text files. Are you done?
>> No! There are no properties (attributes) declarations in Python. You cannot
>> infer all possible class attributes just from class declaration. You can try
>> to parse __init__ method to find attribute initialization code but then you
>> find that you cannot infer property types. You would not know what values
>> does class expect in those attributes, what are constraints etc. Besides
>> there are plenty of ways to fool such naive algorithms. Classes can be
>> created using type() built-in function without declarations at all.
>> Attributes may be initialized anywhere. Everything can be made in runtime.
>> As you can see Python is not good for the task. Ans all this true for all
>> of the languages you mentioned. The reason is fundamental - all of those
>> languages are dynamic.
>> So maybe you can take some statically-typed language like C++ or Java to
>> do the job? Again, no! There are other problems with those type of
>> languages. Have you ever seen statically-typed embeddable language? Those
>> language require compilation and linker, binary distribution of classes,
>> handling of versioning problems, dependencies on 3-rd party libraries etc.
>> This is the hell you don't want to step in. But even in statically typed
>> languages there are no contracts. You know the list of properties and their
>> types. But knowing that class foo has property bar of type int doesn't give
>> a clue if value if 10 is okay for that property.
>>
>> Now I admit that we could probably spend a year of development and try to
>> implement this on type Java with custom wrapping and heavy usage of
>> decorators, embed somehow JVM into python process, do all possible
>> sandboxing and then get a Frankenstein that nobody in OpenStack community
>> would like or accept.
>>
>> My point here is that there is no language in the world that was designed
>> for external object composition that is in Murano. No language in the world
>> was design so that there would be huge repository of classes but no single
>> control point, no main project, no trust between different classes. There is
>> no language with tree-like heap organization (each object has single owner
>> object). This what makes MuranoPL domain-specific and not deployments. There
>> are no such languages not because nobody that smart to create them but
>> because nobody needed to do so. Because MuranoPL is domain-specific unlike
>> Python or Java.
>>
>> MuranoPL does contain many constructs that can be found in general-purpose
>> languages but that doesn't make it general-purpose!
>>
>>
>> > Who is going to train people on muranoPL, write language books and
>> > tutorials when the same amount of work has already been done for 10+ years
>> > for other languages....
>> Let me say that most of MuranoPL audience are bash-developers that know
>> Lua to the same level they know MuranoPL :)
>>
>> In Mirantis we deed interesting experiment. We asked several deployment
>> engineers that were not involved in MuranoPL creation and knew nothing about
>> it beforehand to write some application manifests using MuranoPL. We gave
>> them 10-minute introduction, link to DSL documentation blueprint and
>> examples in my github repository (you've seen both of them). The link above
>> to DemoInstance manifest points to the class that was created by one of
>> those guys. It took them several days to get used to it. But they never even
>> used Python before! And the feedback was very positive.
>>
>> I don't really see a problem writing tutorials or even a book. Learning
>> new framework like TaskFlow is nowhere easier than learning DSL. Anyway you
>> need to learn something to start using new technology. There are hundreds of
>> DSL languages that are used in production and I don't feel like having one
>> more language is really a problem
>>
>>
>>
>>
>> On Mon, Mar 10, 2014 at 12:26 AM, Joshua Harlow <harlowja at yahoo-inc.com>
>> wrote:
>>>
>>> I'd be very interested in knowing the resource controls u plan to add.
>>> Memory, CPU...
>>>
>>> I'm still trying to figure out where something like
>>> https://github.com/istalker2/MuranoDsl/blob/master/meta/com.mirantis.murano.demoApp.DemoInstance/manifest.yaml
>>> would be beneficial, why not just spend effort sand boxing lua, python...
>>> Instead of spending effort on creating a new language and then having to
>>> sandbox it as well... Especially if u picked languages that are made to be
>>> sandboxed from the start (not python)...
>>>
>>> Who is going to train people on muranoPL, write language books and
>>> tutorials when the same amount of work has already been done for 10+ years
>>> for other languages....
>>>
>>> I fail to see where muranoPL is a DSL when it contains a full language
>>> already with functions, objects, namespaces, conditionals... (what is the
>>> domain of it?), maybe I'm just confused though (quite possible, haha).
>>>
>>> Does this not seem awkward to anyone else??
>>>
>>> Sent from my really tiny device...
>>>
>>> On Mar 8, 2014, at 10:44 PM, "Stan Lagun" <slagun at mirantis.com> wrote:
>>>
>>> First of all MuranoPL is not a host but hosted language. It never
>>> intended to replace Python and if Python can do the job it is probably
>>> better than MuranoPL for that job.
>>> The problem with Python is that you cannot have Python code as a part of
>>> your DSL if you need to evaluate that DSL on server-side. Using Python's
>>> eval() is not secure. And you don't have enough control on what evaled code
>>> is allowed to do. MuranoPL on the contrary are fully sandboxed. You have
>>> absolute control over what functions/methods/APIs are exposed to DSL and DSL
>>> code can do nothing except for what it allowed to do. Besides you typically
>>> do want your DSL to be domain-specific so general-purpose language like
>>> Python can be suboptimal.
>>>
>>> I don't say MuranoPL is good for all projects. It has many
>>> Murano-specific things after all. In most cases you don't need all those OOP
>>> features that MuranoPL has. But the code organization, how it uses YAML,
>>> block structures and especially YAQL expressions can be of a great value to
>>> many projects
>>>
>>> For examples of MuranoPL classes you can browse
>>> https://github.com/istalker2/MuranoDsl/tree/master/meta folder. This is my
>>> private repository that I was using to develop PoC for MuranoPL engine. We
>>> are on the way to create production-quality implementation with unit-tests
>>> etc. in regular Murano repositories on stackforge
>>>
>>>
>>> On Sun, Mar 9, 2014 at 7:33 AM, Joshua Harlow <harlowja at yahoo-inc.com>
>>> wrote:
>>>>
>>>> To continue from other thread....
>>>>
>>>> """
>>>> Personally I believe that YAQL-based DSLs similar (but probably simlet
>>>> than) MuranoPL can be of a great value for many OpenStack projects that have
>>>> DSLs of different kind. Murano for App Catatalog, Mistral for workflows,
>>>> Heat for HOT templates, Ceilometer for alarms & counter aggregation rules,
>>>> Nova for customizable resource scheduling and probably many more.
>>>> """
>>>>
>>>> Isn't python a better host language for said DSLs than muranoPL? I am
>>>> still pretty weary of a DSL that starts to grow so many features to
>>>> encompass other DSLs since then it's not really a DSL but a non-DSL, and we
>>>> already have one that everyone is familiar with (python).
>>>>
>>>> Are there any good examples if muranoPL that I can look at that take
>>>> advantage of muranoPL classes, functions, namespaces which can be compared
>>>> to there python equivalents. Has such an analysis/evaluation been done?
>>>>
>>>> Sent from my really tiny device...
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>
>>>
>>> --
>>> Sincerely yours
>>> Stanislav (Stan) Lagun
>>> Senior Developer
>>> Mirantis
>>> 35b/3, Vorontsovskaya St.
>>> Moscow, Russia
>>> Skype: stanlagun
>>> www.mirantis.com
>>> slagun at mirantis.com
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>> --
>> Sincerely yours
>> Stanislav (Stan) Lagun
>> Senior Developer
>> Mirantis
>> 35b/3, Vorontsovskaya St.
>> Moscow, Russia
>> Skype: stanlagun
>> www.mirantis.com
>> slagun at mirantis.com
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Sincerely yours
> Stanislav (Stan) Lagun
> Senior Developer
> Mirantis
> 35b/3, Vorontsovskaya St.
> Moscow, Russia
> Skype: stanlagun
> www.mirantis.com
> slagun at mirantis.com
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>