Open Stack

Hi Steve, Thomas

I'm glad the discussion is so constructive!

If we add type interfaces to HOT this may do the job.
Applications in AppCatalog need to be portable across OpenStack clouds.
Thus if we use some globally-unique type naming system applications could
identify their dependencies in unambiguous way.

We also would need to establish relations between between interfaces.
Suppose there is My::Something::Database interface and 7 compatible
materializations:
My::Something::TroveMySQL
My::Something::GaleraMySQL
My::Something::PostgreSQL
My::Something::OracleDB
My::Something::MariaDB
My::Something::MongoDB
My::Something::HBase

There are apps that (say SQLAlchemy-based apps) are fine with any
relational DB. In that case all templates except for MongoDB and HBase
should be matched. There are apps that design to work with MySQL only. In
that case only TroveMySQL, GaleraMySQL and MariaDB should be matched. There
are application who uses PL/SQL and thus require OracleDB (there can be
several Oracle implementations as well). There are also applications
(Marconi and Ceilometer are good example) that can use both some SQL and
NoSQL databases. So conformance to Database interface is not enough and
some sort of interface hierarchy required.

Another thing that we need to consider is that even compatible
implementations may have different set of parameters. For example
clustered-HA PostgreSQL implementation may require additional parameters
besides those needed for plain single instance variant. Template that
consumes *any* PostgreSQL will not be aware of those additional parameters.
Thus they need to be dynamically added to environment's input parameters
and resource consumer to be patched to pass those parameters to actual
implementation



On Fri, Apr 4, 2014 at 9:53 AM, Thomas Spatzier
<thomas.spatzier at de.ibm.com>wrote:

> Hi Steve,
>
> your indexing idea sounds interesting, but I am not sure it would work
> reliably. The kind of matching based on names of parameters and outputs and
> internal get_attr uses has very strong assumptions and I think there is a
> not so low risk of false positives. What if the templates includes some
> internal details that would not affect the matching but still change the
> behavior in a way that would break the composition. Or what if a user by
> chance built a template that by pure coincidence uses the same parameter
> and output names as one of those abstract types that was mention, but the
> template is simply not built for composition?
>
> I think it would be much cleaner to have an explicit attribute in the
> template that says "this template can be uses as realization of type
> My::SomeType" and use that for presenting the user choice and building the
> environment.
>
> Regards,
> Thomas
>
> Steve Baker <sbaker at redhat.com> wrote on 04/04/2014 06:12:38:
> > From: Steve Baker <sbaker at redhat.com>
> > To: openstack-dev at lists.openstack.org
> > Date: 04/04/2014 06:14
> > Subject: Re: [openstack-dev] [Heat] [Murano] [Solum] applications inthe
> cloud
> >
> > On 03/04/14 13:04, Georgy Okrokvertskhov wrote:
> > Hi Steve,
> >
> > I think this is exactly the place where we have a boundary between
> > Murano catalog and HOT.
> >
> > In your example one can use abstract resource type and specify a
> > correct implementation via environment file. This is how it will be
> > done on the final stage in Murano too.
> >
> > Murano will solve another issue. In your example user should know
> > what template to use as a provider template. In Murano this will be
> > done in the following way:
> > 1) User selects an app which requires a DB
> > 2) Murano sees this requirement for DB and do a search in the app
> > catalog to find all apps which expose this functionality. Murano
> > uses app package definitions for that.
> > 3) User select in UI specific DB implementation he wants to use.
> >
> > As you see, in Murano case user has no preliminary knowledge of
> > available apps\templates and it uses catalog to find it. A search
> > criteria can be quite complex with using different application
> > attribute. If we think about moving application definition to HOT
> > format it should provide all necessary information for catalog.
> >
> > In order to search apps in catalog which uses HOT format we need
> > something like that:
> >
> > One needs to define abstract resource like
> > OS:HOT:DataBase
> >
> > Than in each DB implementation of DB resource one has to somehow
> > refer this abstract resource as a parent like
> >
> > Resource OS:HOT:MySQLDB
> >   Parent: OS:HOT:DataBase
> >
> > Then catalog part can use this information and build a list of all
> > apps\HOTs with resources with parents OS:HOT:DataBase
> >
> > That is what we are looking for. As you see, in this example I am
> > not talking about version and other attributes which might be
> > required for catalog.
> >
> >
> > This sounds like a vision for Murano that I could get behind. It
> > would be a tool which allows fully running applications to be
> > assembled and launched from a catalog of Heat templates (plus some
> > app lifecycle workflow beyond the scope of this email).
> >
> > We could add type interfaces to HOT but I still think duck typing
> > would be worth considering. To demonstrate, lets assume that when a
> > template gets cataloged, metadata is also indexed about what
> > parameters and outputs the template has. So for the case above:
> > 1) User selects an app to launch from the catalog
> > 2) Murano performs a heat resource-type-list and compares that with
> > the types in the template. The resource-type list is missing
> > My::App::Database for a resource named my_db
> > 3) Murano analyses the template and finds that My::App::Database is
> > assigned 2 properties (db_username, db_password) and elsewhere in
> > the template is a {get_attr: [my_db, db_url]} attribute access.
> > 4) Murano queries glance for templates, filtering by templates which
> > have parameters [db_username, db_password] and outputs [db_url]
> > (plus whatever appropriate metadata filters)
> > 5) Glance returns 2 matches. Murano prompts the user for a choice
> > 6) Murano constructs an environment based on the chosen template,
> > mapping My::App::Database to the chosen template
> > 7) Murano launches the stack
> >
> > Sure, there could be a type interface called My::App::Database which
> > declares db_username, db_password and db_url, but since a heat
> > template is in a readily parsable declarative format, all required
> > information is available to analyze, both during glance indexing and
> > app launching.
> >
>
> >
>
> > On Wed, Apr 2, 2014 at 3:30 PM, Steve Baker <sbaker at redhat.com> wrote:
> > On 03/04/14 10:39, Ruslan Kamaldinov wrote:
> > > This is a continuation of the "MuranoPL questions" thread.
> > >
> > > As a result of ongoing discussions, we figured out that definitionof
> layers
> > > which each project operates on and has responsibility for is not yet
> agreed
> > > and discussed between projects and teams (Heat, Murano, Solum (in
> > > alphabetical order)).
> > >
> > > Our suggestion and expectation from this working group is to have such
> > > a definition of layers, agreement on it and an approach of reaching it.
> > >
> > > As a starting point, we suggest the following:
> > >
> > > There are three layers of responsibility:
> > > 1. Resources of the cloud
> > > 2. Applications of the cloud
> > > 3. Layers specific for Murano and Solum (to be further discussed and
> > >    clarified, for this discussion it's out of scope)
> > >
> > > Layer 1 is obviously covered by Heat.
> > >
> > > Most of the disagreement is around layer 2. Our suggestion is to figure
> out
> > > the way where we can have common engine, DSL and approach to apps
> > description.
> > > For this we'll take HOT and TOSCA as a basis and will work on addition
> of
> > > functionality missing from Murano and Solum point of view.
> > >
> > > We'll be happy if existing Heat team continue working on it, having the
> full
> > > control of the project, provided that we agree on functionality
> > missing there
> > > from Murano and Solum point of view and addition of this functionality
> in a
> > > systematic way.
> > >
> > > Let me outline the main concern in regards to HOT from Murano
> perspective.
> > > Others concerns could be figured out later. The typical use case
> > for Murano is
> > > the following:
> > > Application writer sets a requirement for her application to use RDBMS
> > > assuming that it can be MySQL, PostgreSQL or MSSQL. HOT engine
> > should be able
> > > to understand such requirements and be able to satisfy them by
> instantiating
> > > instances with DB. End user should be able to control which
> > particular type of
> > > DB should be used. This is quite similar to abstract requirements
> > described in
> > > TOSCA. As Heat wants to cover TOSCA format too this Murano requirement
> is
> > > pretty well aligned with what HOT\TOSCA can provide.
> > >
> > >
> > > Hopefully this functionality can be introduced into HOT. Solum and
> > Murano will
> > > leverage it by using some common API, and implementing layer 3 as thin
> as
> > > possible.
> > >
> > > Again, this is only suggested starting point, something to begin with.
> > >
> > Thanks for bring this discussion back to solving specific problems. The
> > above database abstraction can be nicely represented using HOT
> > environments and provider resources [1].
> >
> > Take the following template snippet:
> >
> > resources:
> >   my_db:
> >     type: My::Database
> >     properties:
> >       # ...
> >
> >   my_server
> >     type: OS::Nova::Server
> >     properties:
> >       # ... including a {get_attr: [my_db, connection_url]}
> >
> > My::Database is a type defined in the environment which is included in
> > the stack-create call. Different environments can provide different
> > database implementations, for example:
> >
> > resource_registry:
> >   "My::Database": path/to/trove_wrapper.yaml
> >
> > resource_registry:
> >   "My::Database": path/to/nova_galera_ubercluster.yaml
> >
> > resource_registry:
> >   "My::Database": path/to/postgresql.yaml
> >
> > Each of these templates could be implemented to have the same parameters
> > and outputs so the template will work with any of these resource
> > provider templates.
> >
> > One thing we could consider adding to HOT is allow the defining of a
> > type interface, and validating that a given template/resource implements
> > its declared interfaces. However the above example works just fine
> > without interfaces, and this duck-typing approach may provide a better
> > user experience anyway.
> >
> > [1]
> http://docs.openstack.org/developer/heat/template_guide/environment.html
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Sincerely yours
Stanislav (Stan) Lagun
Principal Developer
Mirantis
35b/3, Vorontsovskaya St.
Moscow, Russia
Skype: stanlagun
www.mirantis.com
slagun at mirantis.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140404/f35b6028/attachment.html>