Open Stack

Sure, I understand how this could make it harder.

Its a hard question to answer, which one is more worth it, creating a bunch of DSL's that u now have to implement correctly in a runtime that is actually pretty hard to isolate/control the execution of, or should people bite the bullet and move to something that actually has a runtime that provides these features. Since it appears the DSL's that are being created are nearly turing complete it appears inevitable that said runtime will be needed (but of course I'm all for being proven wrong).

-Josh

From: Zane Bitter <zbitter at redhat.com<mailto:zbitter at redhat.com>>
Organization: Red Hat
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Tuesday, March 18, 2014 at 9:36 AM
To: "openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] MuranoPL questions?

On 18/03/14 08:01, Ruslan Kamaldinov wrote:
Joshua, Clint,

The only platform I'm aware about which fully supports true isolation and which
has been used in production for this purpose is Java VM. I know people who
developed systems for online programming competitions and really smart kids
tried to break it without any luck :)

Since we're speaking about Heat, Mistral and Murano DSLs and all of them need an
execution engine. Do you think that JVM could become a host for that engine?

-2. Deploying OpenStack is hard enough already.

JVM has a lot of potential:
- it allows to fine-tune security manager to allow/disallow specific actions
- it can execute a lot of programming languages - Python, Ruby, JS, etc
- it has been used in production for this specific purpose for years

But it also introduces another layer of complexity:
- it's another component to deploy, configure and monitor
- it's non-python, which means it should be supported by infra
- we will need to run java service and potentially have some java code to
    accept and process user code


Thanks,
Ruslan

On Mon, Mar 17, 2014 at 10:40 PM, Joshua Harlow <harlowja at yahoo-inc.com<mailto:harlowja at yahoo-inc.com>> wrote:
So I guess this is similar to the other thread.

http://lists.openstack.org/pipermail/openstack-dev/2014-March/030185.html

I know that the way YQL has provided it could be a good example; where the
core DSL (the select queries and such) are augmented by the addition and
usage of JS, for example
http://developer.yahoo.com/yql/guide/yql-execute-examples.html#yql-execute-example-helloworld
(ignore that its XML, haha). Such usage already provides rate-limits and
execution-limits
(http://developer.yahoo.com/yql/guide/yql-execute-intro-ratelimits.html) and
afaik if something like what YQL is doing then u don't need to recreate
simialr features in your DSL (and then u also don't need to teach people
about a new language and syntax and ...)

Just an idea (I believe lua offers similar controls/limits.., although its
not as popular of course as JS).

From: Stan Lagun <slagun at mirantis.com<mailto:slagun at mirantis.com>>

Reply-To: "OpenStack Development Mailing List (not for usage questions)"
<openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Monday, March 17, 2014 at 3:59 AM

To: "OpenStack Development Mailing List (not for usage questions)"
<openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] MuranoPL questions?

Joshua,

Completely agree with you. We wouldn't be writing another language if we
knew how any of existing languages can be used for this particular purpose.
If anyone suggest such language and show how it can be used to solve those
issues DSL was designed to solve we will consider dropping MuranoPL. np

Surely DSL hasn't stood the test of time. It just hasn't had a chance yet.
100% of successful programming languages were in such position once.

Anyway it is the best time to come with your suggestions. If you know how
exactly DSL can be replaced or improved we would like you to share


On Wed, Mar 12, 2014 at 2:05 AM, Joshua Harlow <harlowja at yahoo-inc.com<mailto:harlowja at yahoo-inc.com>>
wrote:

I guess I might be a bit biased to programming; so maybe I'm not the
target audience.

I'm not exactly against DSL's, I just think that DSL's need to be really
really proven to become useful (in general this applies to any language that
'joe' comp-sci student can create). Its not that hard to just make one, but
the real hard part is making one that people actually like and use and
survives the test of time. That's why I think its just nicer to use
languages that have stood the test of time already (if we can), creating a
new DSL (muranoPL seems to be slightly more than a DSL imho) means creating
a new language that has not stood the test of time (in terms of lifetime,
battle tested, supported over years) so that's just the concern I have.

Of course we have to accept innovation and I hope that the DSL/s makes it
easier/simpler, I just tend to be a bit more pragmatic maybe in this area.

Here's hoping for the best! :-)

-Josh

From: Renat Akhmerov <rakhmerov at mirantis.com<mailto:rakhmerov at mirantis.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)"
<openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Monday, March 10, 2014 at 8:36 PM
To: "OpenStack Development Mailing List (not for usage questions)"
<openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] MuranoPL questions?

Although being a little bit verbose it makes a lot of sense to me.

@Joshua,

Even assuming Python could be sandboxed and whatever else that's needed to
be able to use it as DSL (for something like Mistral, Murano or Heat) is
done  why do you think Python would be a better alternative for people who
don't know neither these new DSLs nor Python itself. Especially, given the
fact that Python has A LOT of things that they'd never use. I know many
people who have been programming in Python for a while and they admit they
don't know all the nuances of Python and actually use 30-40% of all of its
capabilities. Even not in domain specific development. So narrowing a
feature set that a language provides and limiting it to a certain domain
vocabulary is what helps people solve tasks of that specific domain much
easier and in the most expressive natural way. Without having to learn tons
and tons of details that a general purpose language (GPL, hah :) ) provides
(btw, the reason to write thick books).

I agree with Stan, if you begin to use a technology you'll have to learn
something anyway, be it TaskFlow API and principles or DSL. Well-designed
DSL just encapsulates essential principles of a system it is used for. By
learning DSL you're leaning the system itself, as simple as that.

Renat Akhmerov
@ Mirantis Inc.



On 10 Mar 2014, at 05:35, Stan Lagun <slagun at mirantis.com<mailto:slagun at mirantis.com>> wrote:

I'd be very interested in knowing the resource controls u plan to add.
Memory, CPU...
We haven't discussed it yet. Any suggestions are welcomed

I'm still trying to figure out where something like
https://github.com/istalker2/MuranoDsl/blob/master/meta/com.mirantis.murano.demoApp.DemoInstance/manifest.yaml
would be beneficial, why not > just spend effort sand boxing lua, python...
Instead of spending effort on creating a new language and then having to
sandbox it as well... Especially if u picked languages that are made to be
sandboxed from the start (not python)...

1. See my detailed answer in Mistral thread why haven't we used any of
those languages. There are many reasons besides sandboxing.

2. You don't need to sandbox MuranoPL. Sandboxing is restricting some
operations. In MuranoPL ALL operations (including operators in expressions,
functions, methods etc.) are just those that you explicitly provided. So
there is nothing to restrict. There are no builtins that throw
AccessViolationError

3. Most of the value of MuranoPL comes not form the workflow code but from
class declarations. In all OOP languages classes are just a convenient to
organize your code. There are classes that represent real-life objects and
classes that are nothing more than data-structures, DTOs etc. In Murano
classes in MuranoPL are deployable entities like Heat resources application
components, services etc. In dashboard UI user works with those entities. He
(in UI!) creates instances of those classes, fills their property values,
binds objects together (assigns on object to a property of another). And
this is done without even a single MuranoPL line being executed! That is
possible because everything in MuranoPL is a subject to declaration and
because it is just plain YAML anyone can easily extract those declarations
from MuranoPL classes.
Now suppose it was Python instead of MuranoPL. Then you would have to
parse *.py files to get list of declared classes (without executing
anything). Suppose that you managed to solve this somehow. Probably you
wrote regexp that finds all class declarations it text files. Are you done?
No! There are no properties (attributes) declarations in Python. You cannot
infer all possible class attributes just from class declaration. You can try
to parse __init__ method to find attribute initialization code but then you
find that you cannot infer property types. You would not know what values
does class expect in those attributes, what are constraints etc. Besides
there are plenty of ways to fool such naive algorithms. Classes can be
created using type() built-in function without declarations at all.
Attributes may be initialized anywhere. Everything can be made in runtime.
As you can see Python is not good for the task. Ans all this true for all
of the languages you mentioned. The reason is fundamental - all of those
languages are dynamic.
So maybe you can take some statically-typed language like C++ or Java to
do the job? Again, no! There are other problems with those type of
languages. Have you ever seen statically-typed embeddable language? Those
language require compilation and linker, binary distribution of classes,
handling of versioning problems, dependencies on 3-rd party libraries etc.
This is the hell you don't want to step in. But even in statically typed
languages there are no contracts. You know the list of properties and their
types. But knowing that class foo has property bar of type int doesn't give
a clue if value if 10 is okay for that property.

Now I admit that we could probably spend a year of development and try to
implement this on type Java with custom wrapping and heavy usage of
decorators, embed somehow JVM into python process, do all possible
sandboxing and then get a Frankenstein that nobody in OpenStack community
would like or accept.

My point here is that there is no language in the world that was designed
for external object composition that is in Murano. No language in the world
was design so that there would be huge repository of classes but no single
control point, no main project, no trust between different classes. There is
no language with tree-like heap organization (each object has single owner
object). This what makes MuranoPL domain-specific and not deployments. There
are no such languages not because nobody that smart to create them but
because nobody needed to do so. Because MuranoPL is domain-specific unlike
Python or Java.

MuranoPL does contain many constructs that can be found in general-purpose
languages but that doesn't make it general-purpose!


Who is going to train people on muranoPL, write language books and
tutorials when the same amount of work has already been done for 10+ years
for other languages....
Let me say that most of MuranoPL audience are bash-developers that know
Lua to the same level they know MuranoPL :)

In Mirantis we deed interesting experiment. We asked several deployment
engineers that were not involved in MuranoPL creation and knew nothing about
it beforehand to write some application manifests using MuranoPL. We gave
them 10-minute introduction, link to DSL documentation blueprint and
examples in my github repository (you've seen both of them). The link above
to DemoInstance manifest points to the class that was created by one of
those guys. It took them several days to get used to it. But they never even
used Python before! And the feedback was very positive.

I don't really see a problem writing tutorials or even a book. Learning
new framework like TaskFlow is nowhere easier than learning DSL. Anyway you
need to learn something to start using new technology. There are hundreds of
DSL languages that are used in production and I don't feel like having one
more language is really a problem




On Mon, Mar 10, 2014 at 12:26 AM, Joshua Harlow <harlowja at yahoo-inc.com<mailto:harlowja at yahoo-inc.com>>
wrote:

I'd be very interested in knowing the resource controls u plan to add.
Memory, CPU...

I'm still trying to figure out where something like
https://github.com/istalker2/MuranoDsl/blob/master/meta/com.mirantis.murano.demoApp.DemoInstance/manifest.yaml
would be beneficial, why not just spend effort sand boxing lua, python...
Instead of spending effort on creating a new language and then having to
sandbox it as well... Especially if u picked languages that are made to be
sandboxed from the start (not python)...

Who is going to train people on muranoPL, write language books and
tutorials when the same amount of work has already been done for 10+ years
for other languages....

I fail to see where muranoPL is a DSL when it contains a full language
already with functions, objects, namespaces, conditionals... (what is the
domain of it?), maybe I'm just confused though (quite possible, haha).

Does this not seem awkward to anyone else??

Sent from my really tiny device...

On Mar 8, 2014, at 10:44 PM, "Stan Lagun" <slagun at mirantis.com<mailto:slagun at mirantis.com>> wrote:

First of all MuranoPL is not a host but hosted language. It never
intended to replace Python and if Python can do the job it is probably
better than MuranoPL for that job.
The problem with Python is that you cannot have Python code as a part of
your DSL if you need to evaluate that DSL on server-side. Using Python's
eval() is not secure. And you don't have enough control on what evaled code
is allowed to do. MuranoPL on the contrary are fully sandboxed. You have
absolute control over what functions/methods/APIs are exposed to DSL and DSL
code can do nothing except for what it allowed to do. Besides you typically
do want your DSL to be domain-specific so general-purpose language like
Python can be suboptimal.

I don't say MuranoPL is good for all projects. It has many
Murano-specific things after all. In most cases you don't need all those OOP
features that MuranoPL has. But the code organization, how it uses YAML,
block structures and especially YAQL expressions can be of a great value to
many projects

For examples of MuranoPL classes you can browse
https://github.com/istalker2/MuranoDsl/tree/master/meta folder. This is my
private repository that I was using to develop PoC for MuranoPL engine. We
are on the way to create production-quality implementation with unit-tests
etc. in regular Murano repositories on stackforge


On Sun, Mar 9, 2014 at 7:33 AM, Joshua Harlow <harlowja at yahoo-inc.com<mailto:harlowja at yahoo-inc.com>>
wrote:

To continue from other thread....

"""
Personally I believe that YAQL-based DSLs similar (but probably simlet
than) MuranoPL can be of a great value for many OpenStack projects that have
DSLs of different kind. Murano for App Catatalog, Mistral for workflows,
Heat for HOT templates, Ceilometer for alarms & counter aggregation rules,
Nova for customizable resource scheduling and probably many more.
"""

Isn't python a better host language for said DSLs than muranoPL? I am
still pretty weary of a DSL that starts to grow so many features to
encompass other DSLs since then it's not really a DSL but a non-DSL, and we
already have one that everyone is familiar with (python).

Are there any good examples if muranoPL that I can look at that take
advantage of muranoPL classes, functions, namespaces which can be compared
to there python equivalents. Has such an analysis/evaluation been done?

Sent from my really tiny device...
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
slagun at mirantis.com<mailto:slagun at mirantis.com>

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
slagun at mirantis.com<mailto:slagun at mirantis.com>
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Sincerely yours
Stanislav (Stan) Lagun
Senior Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
slagun at mirantis.com<mailto:slagun at mirantis.com>


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140318/575b653d/attachment.html>