Open Stack

Hi Keith,

The question with Heat plugins is a good one. It actually has several
different answers.

The most important one is that the actual implementation of this approach
may be considered as "out of scope" for Murano. Think of Murano as an
application integration engine, which allows existing application to
communicate with each other and with the openstack infrastructure via
multiple interfaces. The possibility of this interaction does matter a lot,
while the actual implementation of the interfaces is much less important.
Application Publishers should be able to execute something like
'instance.takeSnapshot()' or
'databaseInstance.Migrate(anotherDatabaseInstance)' from the DSL-code of
their workflows, while the actual implementation of these actions may be
different for different "classes". Murano will definitely come bundled with
the support of Heat templates as its primary OpenStack interface, so, if an
application publisher decides to execute some specific action via a custom
heat plugin, Murano will provide all means to use this plugin
out-of-the-box.
So, in general, the plugin-approach is perfectly fine, it already fits into
Murano, and that is intentional: our ultimate goal is to integrate Murano
into existing OpenStack infrastructure as much as possible. The final
desicion which approach to choose is up to the publisher of the specific
application though.

However, in some cases relying on the plugins seems to be a bad idea. When
you create a custom plugin for a custom task you have to install the plugin
to the Heat at the target cloud first. And this contradicts with the whole
purpose of application catalog, where all the needed dependencies of the
application are placed into its package by the publisher, and once the
package is put into the catalog it is immediately available for usage. We
cannot allow to put plugins into the packages, as plugins are actually
written in python, while allowing to upload and run an arbitrary python
code to untrusted users is definitely a security breach.
So, plugins are ok for the most common and popular tasks (but in this case
these types of resources should be eventually placed into the Heat itself,
right?), but may be a bad choice for custom tasks.

And the last but not least, the thing which Georgy has already mentioned: I
personally feel a bit uncomfortable when trying to think about the
processes and actions as objects persisted in a Heat stack. These are
different kinds of entities, and in my opinion events are dynamic and
transient, while the Heat stack is stable and persisted. Events can modify
the stack - and that is why we need the workflows - but they should not be
persisted on their own.  I don't know how to express it in a more formal
way, and that is just my gut feeling which definitely has to be discussed
more. I am sure we will come back to this topic when the DSL is finished
and we started implementing the bases class library for Murano and its
external interfaces with Heat and other services.

Thanks!

--
Regards,
 from 

Alexander Tivelkov


On Tue, Feb 25, 2014 at 1:44 AM, Keith Bray <keith.bray at rackspace.com>wrote:

>  Have you considered writing Heat resource plug-ins that perform (or
> configure within other services) instance snapshots, backups, or whatever
> other maintenance workflow possibilities you want that don't exist?  Then
> these maintenance workflows you mention could be expressed in the Heat
> template forming a single place for the application architecture
> definition, including defining the configuration for services that need to
> be application aware throughout the application's life .  As you describe
> things in Murano, I interpret that you are layering application
> architecture specific information and workflows into a DSL in a layer above
> Heat, which means information pertinent to the application as an ongoing
> concern would be disjoint.  Fragmenting the necessary information to wholly
> define an infrastructure/application architecture could make it difficult
> to share the application and modify the application stack.
>
>  I would be interested in a library that allows for composing Heat
> templates from "snippets" or "fragments" of pre-written Heat DSL... The
> library's job could be to ensure that the snippets, when combined, create a
> valid Heat template free from conflict amongst resources, parameters, and
> outputs.  The interaction with the library, I think, would belong in
> Horizon, and the "Application Catalog" and/or "Snippets Catalog" could be
> implemented within Glance.
>
>  >>>Also, there may be workflow steps which are not covered by Heat by
> design. For example, application publisher may include creating instance
> snapshots, data migrations, backups etc into the deployment or maintenance
> workflows. I don't see how these may be done by Heat, while Murano should
> definitely support this scenarios.
>
>   From: Alexander Tivelkov <ativelkov at mirantis.com>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev at lists.openstack.org>
> Date: Monday, February 24, 2014 12:18 PM
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev at lists.openstack.org>
> Subject: Re: [openstack-dev] [Murano] Object-oriented approach for
> defining Murano Applications
>
>   Hi Stan,
>
>  It is good that we are on a common ground here :)
>
>  Of course this can be done by Heat. In fact - it will be, in the very
> same manner as it always was, I am pretty sure we've discussed this many
> times already. When Heat Software config is fully implemented, it will be
> possible to use it instead of our Agent execution plans for software
> configuration - it the very same manner as we use "regular" heat templates
> for resource allocation.
>
>  Heat does indeed support template composition - but we don't want our
> end-users to do learn how to do that: we want them just to combine existing
> application on higher-level. Murano will use the template composition under
> the hood, but only in the way which is designed by application publisher.
> If the publisher has decided to configure the software with using Heat
> Software Config, then this option will be used. If some other (probably
> some legacy ) way of doing this was preferred, Murano should be able to
> support that and allow to create such workflows.
>
>  Also, there may be workflow steps which are not covered by Heat by
> design. For example, application publisher may include creating instance
> snapshots, data migrations, backups etc into the deployment or maintenance
> workflows. I don't see how these may be done by Heat, while Murano should
> definitely support this scenarios.
>
>  So, as a conclusion, Murano should not be though of as a Heat
> alternative: it is a different tool located on the different layer of the
> stack, aiming different user audience - and, the most important - using the
> Heat underneath.
>
>
>  --
>  Regards,
> Alexander Tivelkov
>
>
> On Mon, Feb 24, 2014 at 8:36 PM, Stan Lagun <slagun at mirantis.com> wrote:
>
>>  Hi Alex,
>>
>>  Personally I like the approach and how you explain it. I just would like
>> to know your opinion on how this is better from someone write Heat template
>> that creates Active Directory  lets say with one primary and one secondary
>> controller and then publish it somewhere. Since Heat do supports software
>> configuration as of late and has concept of environments [1] that Steven
>> Hardy generously pointed out in another mailing thread that can be used for
>> composition as well it seems like everything you said can be done by Heat
>> alone
>>
>> [1]:
>> http://hardysteven.blogspot.co.uk/2013/10/heat-providersenvironments-101-ive.html
>>
>>
>>  On Mon, Feb 24, 2014 at 7:51 PM, Alexander Tivelkov <
>> ativelkov at mirantis.com> wrote:
>>
>>>   Sorry folks, I didn't put the proper image url. Here it is:
>>>
>>>
>>>  https://creately.com/diagram/hrxk86gv2/kvbckU5hne8C0r0sofJDdtYgxc%3D
>>>
>>>
>>>  --
>>>  Regards,
>>> Alexander Tivelkov
>>>
>>>
>>> On Mon, Feb 24, 2014 at 7:39 PM, Alexander Tivelkov <
>>> ativelkov at mirantis.com> wrote:
>>>
>>>>  Hi,
>>>>
>>>>  I would like to initiate one more discussion about an approach we
>>>> selected to solve a particular problem in Murano.
>>>>
>>>> The problem statement is the following: We have multiple entities like
>>>> low level resources and high level application definitions. Each entity
>>>> does some specific actions for example to create a VM or deploy application
>>>> configuration. We want each entity's workflow code reusable in order to
>>>> simplify development for a new application as the current approach with XML
>>>> based rules requires significant efforts.
>>>>
>>>> After internal discussions inside Murano team we come up to the
>>>> solution which uses a well known programmatic concept - classes, their
>>>> inheritance and composition.
>>>>
>>>> In this thread I would like to share our ideas and discuss the
>>>> implementation details.
>>>>
>>>>  We want to represent each and every entity being manipulated by
>>>> Murano, as an instance of some "class". These classes will define structure
>>>> of the entities and their behavior. Different entities may be combined
>>>> together, interacting with each other, forming a composite environment. The
>>>> inheritance may be used to extract common structure and functionality into
>>>> generic superclasses, while having their subclasses to define only their
>>>> specific attributes and actions.
>>>>
>>>>  This approach is better to explain on some example. Let's consider
>>>> the Active Directory windows service. This is one of the currently present
>>>> Murano Applications, and its structure and deployment workflow is pretty
>>>> complex. Let's see how it may be simplified by using the proposed
>>>> object-oriented approach.
>>>>
>>>>  First, let's just describe an Active Directory service in plain
>>>> English.
>>>>
>>>> Active Directory service consists of several Controllers: exactly one
>>>> Primary Domain Controller and, optionally, several Secondary Domain
>>>> Controllers. Controllers (both primary and Secondary) are special Windows
>>>> Instances, having an active directory server role activated. Their specific
>>>> difference is in the configuration scripts which are executed on them after
>>>> the roles are activated. Also, Secondary Domain Controllers have an ability
>>>> to join to a domain, while Primary Domain Controller cannot do it.
>>>>
>>>> Windows Instances are regular machines having some limitations on the
>>>> their images (it should, obviously, be Windows image) and hardware flavor
>>>> (windows is usually demanding on resources). Also, windows machines may
>>>> have some specific operations, like configuring windows firewall rules or
>>>> defining local administrator password.
>>>>
>>>> And the machines in general (both Windows and any others) are simple
>>>> entities which know how to create virtual machines in OpenStack clouds.
>>>>
>>>>  Now, let's map this model to object-oriented concept. We get the
>>>> following classes:
>>>>
>>>>
>>>>    1.
>>>>
>>>>    Instance. Defines common properties of virtual machines (flavor,
>>>>    image, hostname) and deployment workflow which executes a HEAT template to
>>>>    create an instance in the cloud.
>>>>    2.
>>>>
>>>>    WindowsInstance - inherits Instance. Defines local administrator
>>>>    account password and extends base deployment workflow to set this password
>>>>    and configure windows firewall - after the instance is deployed.
>>>>    3.
>>>>
>>>>    DomainMember - inherits Windows instance, defines a machine which
>>>>    can join an Active Directory. Adds a "join domain" workflow step
>>>>    4.
>>>>
>>>>    DomainController - inherits Windows instance, adds an "Install AD
>>>>    Role" workflow steps and extends the "Deploy" step to call it.
>>>>    5.
>>>>
>>>>    PrimaryController - inherits DomainContoller, adds a "Configure as
>>>>    Primary DC" workflow step and extends "Deploy" step to call it. Also adds a
>>>>    domainIpAddress property which is set during the deployment.
>>>>    6.
>>>>
>>>>    SecondaryController, inherits both DomainMember and
>>>>    DomainController. Adds a "Configure as Secondary DC" worflow step and
>>>>    extends Deploy() step to call it and the "join domain" step inherited from
>>>>    the "Domain Member" class.
>>>>    7.
>>>>
>>>>    ActiveDirectory -  a primary class which defines an Active
>>>>    Directory application. Defines properties for PrimaryController and
>>>>    SecondaryControllers and a "Deploy" workflow which call appropriate
>>>>    workflows on the controllers.
>>>>
>>>>
>>>>  The simplified class diagram may look like this:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>  So, this approach allows to decompose the AD deployment workflow into
>>>> simple isolated parts, explicitly manage the state and create reusable
>>>> entities (of course classes like Instance, WindowsInstance, DomainMember
>>>> may be used by other Murano Applications). For me this looks much, much
>>>> better than the current implicit state machine which we run based on XML
>>>> rules.
>>>>
>>>>  What do you think about this approach, folks? Do you think it will be
>>>> easily understood by application developers? Will it be easy to write
>>>> workflows this way? Do you see any drawbacks here?
>>>>
>>>> Waiting for your feedback.
>>>>
>>>>
>>>>   --
>>>>  Regards,
>>>> Alexander Tivelkov
>>>>
>>>
>>>
>>>  _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Sincerely yours
>> Stanislav (Stan) Lagun
>> Senior Developer
>> Mirantis
>> 35b/3, Vorontsovskaya St.
>> Moscow, Russia
>> Skype: stanlagun
>> www.mirantis.com
>> slagun at mirantis.com
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140225/8927945a/attachment.html>