Open Stack

One thing I'm not seeing handled is cross-app knowledge.

 From an earlier post:

 > 1) User selects an app which requires a DB
 > 2) Murano sees this requirement for DB and do a search in the app
 > catalog to find all apps which expose this functionality. Murano uses
 > app package definitions for that.
 > 3) User select in UI specific DB implementation he wants to use.

For example, take TripleO deploying OpenStack and just look at Nova and 
Glance. Presumably they are two different apps, allowing the user fine 
grained control over how many of each type to deploy and what not.

Both apps would indicate they need a message broker. But they need the 
same message broker, so it's invalid for QPID for one app and rabbit for 
another.

Perhaps I'm viewing it wrong, and instead of having different apps 
there's just one big "OpenStack" app, but that still has the same issue.

On 04/04/2014 05:54 AM, Stan Lagun wrote:
> Hi Steve, Thomas
>
> I'm glad the discussion is so constructive!
>
> If we add type interfaces to HOT this may do the job.
> Applications in AppCatalog need to be portable across OpenStack clouds.
> Thus if we use some globally-unique type naming system applications
> could identify their dependencies in unambiguous way.
>
> We also would need to establish relations between between interfaces.
> Suppose there is My::Something::Database interface and 7 compatible
> materializations:
> My::Something::TroveMySQL
> My::Something::GaleraMySQL
> My::Something::PostgreSQL
> My::Something::OracleDB
> My::Something::MariaDB
> My::Something::MongoDB
> My::Something::HBase
>
> There are apps that (say SQLAlchemy-based apps) are fine with any
> relational DB. In that case all templates except for MongoDB and HBase
> should be matched. There are apps that design to work with MySQL only.
> In that case only TroveMySQL, GaleraMySQL and MariaDB should be matched.
> There are application who uses PL/SQL and thus require OracleDB (there
> can be several Oracle implementations as well). There are also
> applications (Marconi and Ceilometer are good example) that can use both
> some SQL and NoSQL databases. So conformance to Database interface is
> not enough and some sort of interface hierarchy required.

Along those lines, a variation is if a particular version is needed. 
It's an odd parallel, but think of the differences in running a Python 
2.4 app v. 2.6 or higher. There's a pretty clear line there of things 
that won't run on 2.4 and require higher.

Assuming a parallel in these resources, would we end up with multiple 
MongoDB templates with slightly different names? I think the interface 
hierarchy concept comes into play here because we may need to express 
something like "matches at least".

> Another thing that we need to consider is that even compatible
> implementations may have different set of parameters. For example
> clustered-HA PostgreSQL implementation may require additional parameters
> besides those needed for plain single instance variant. Template that
> consumes *any* PostgreSQL will not be aware of those additional
> parameters. Thus they need to be dynamically added to environment's
> input parameters and resource consumer to be patched to pass those
> parameters to actual implementation



>
>
> On Fri, Apr 4, 2014 at 9:53 AM, Thomas Spatzier
> <thomas.spatzier at de.ibm.com <mailto:thomas.spatzier at de.ibm.com>> wrote:
>
>     Hi Steve,
>
>     your indexing idea sounds interesting, but I am not sure it would work
>     reliably. The kind of matching based on names of parameters and
>     outputs and
>     internal get_attr uses has very strong assumptions and I think there
>     is a
>     not so low risk of false positives. What if the templates includes some
>     internal details that would not affect the matching but still change the
>     behavior in a way that would break the composition. Or what if a user by
>     chance built a template that by pure coincidence uses the same parameter
>     and output names as one of those abstract types that was mention,
>     but the
>     template is simply not built for composition?
>
>     I think it would be much cleaner to have an explicit attribute in the
>     template that says "this template can be uses as realization of type
>     My::SomeType" and use that for presenting the user choice and
>     building the
>     environment.
>
>     Regards,
>     Thomas
>
>     Steve Baker <sbaker at redhat.com <mailto:sbaker at redhat.com>> wrote on
>     04/04/2014 06:12:38:
>      > From: Steve Baker <sbaker at redhat.com <mailto:sbaker at redhat.com>>
>      > To: openstack-dev at lists.openstack.org
>     <mailto:openstack-dev at lists.openstack.org>
>      > Date: 04/04/2014 06:14
>      > Subject: Re: [openstack-dev] [Heat] [Murano] [Solum] applications
>     inthe
>     cloud
>      >
>      > On 03/04/14 13:04, Georgy Okrokvertskhov wrote:
>      > Hi Steve,
>      >
>      > I think this is exactly the place where we have a boundary between
>      > Murano catalog and HOT.
>      >
>      > In your example one can use abstract resource type and specify a
>      > correct implementation via environment file. This is how it will be
>      > done on the final stage in Murano too.
>      >
>      > Murano will solve another issue. In your example user should know
>      > what template to use as a provider template. In Murano this will be
>      > done in the following way:
>      > 1) User selects an app which requires a DB
>      > 2) Murano sees this requirement for DB and do a search in the app
>      > catalog to find all apps which expose this functionality. Murano
>      > uses app package definitions for that.
>      > 3) User select in UI specific DB implementation he wants to use.
>      >
>      > As you see, in Murano case user has no preliminary knowledge of
>      > available apps\templates and it uses catalog to find it. A search
>      > criteria can be quite complex with using different application
>      > attribute. If we think about moving application definition to HOT
>      > format it should provide all necessary information for catalog.
>      >
>      > In order to search apps in catalog which uses HOT format we need
>      > something like that:
>      >
>      > One needs to define abstract resource like
>      > OS:HOT:DataBase
>      >
>      > Than in each DB implementation of DB resource one has to somehow
>      > refer this abstract resource as a parent like
>      >
>      > Resource OS:HOT:MySQLDB
>      >   Parent: OS:HOT:DataBase
>      >
>      > Then catalog part can use this information and build a list of all
>      > apps\HOTs with resources with parents OS:HOT:DataBase
>      >
>      > That is what we are looking for. As you see, in this example I am
>      > not talking about version and other attributes which might be
>      > required for catalog.
>      >
>      >
>      > This sounds like a vision for Murano that I could get behind. It
>      > would be a tool which allows fully running applications to be
>      > assembled and launched from a catalog of Heat templates (plus some
>      > app lifecycle workflow beyond the scope of this email).
>      >
>      > We could add type interfaces to HOT but I still think duck typing
>      > would be worth considering. To demonstrate, lets assume that when a
>      > template gets cataloged, metadata is also indexed about what
>      > parameters and outputs the template has. So for the case above:
>      > 1) User selects an app to launch from the catalog
>      > 2) Murano performs a heat resource-type-list and compares that with
>      > the types in the template. The resource-type list is missing
>      > My::App::Database for a resource named my_db
>      > 3) Murano analyses the template and finds that My::App::Database is
>      > assigned 2 properties (db_username, db_password) and elsewhere in
>      > the template is a {get_attr: [my_db, db_url]} attribute access.
>      > 4) Murano queries glance for templates, filtering by templates which
>      > have parameters [db_username, db_password] and outputs [db_url]
>      > (plus whatever appropriate metadata filters)
>      > 5) Glance returns 2 matches. Murano prompts the user for a choice
>      > 6) Murano constructs an environment based on the chosen template,
>      > mapping My::App::Database to the chosen template
>      > 7) Murano launches the stack
>      >
>      > Sure, there could be a type interface called My::App::Database which
>      > declares db_username, db_password and db_url, but since a heat
>      > template is in a readily parsable declarative format, all required
>      > information is available to analyze, both during glance indexing and
>      > app launching.
>      >
>
>      >
>
>      > On Wed, Apr 2, 2014 at 3:30 PM, Steve Baker <sbaker at redhat.com
>     <mailto:sbaker at redhat.com>> wrote:
>      > On 03/04/14 10:39, Ruslan Kamaldinov wrote:
>      > > This is a continuation of the "MuranoPL questions" thread.
>      > >
>      > > As a result of ongoing discussions, we figured out that
>     definitionof
>     layers
>      > > which each project operates on and has responsibility for is
>     not yet
>     agreed
>      > > and discussed between projects and teams (Heat, Murano, Solum (in
>      > > alphabetical order)).
>      > >
>      > > Our suggestion and expectation from this working group is to
>     have such
>      > > a definition of layers, agreement on it and an approach of
>     reaching it.
>      > >
>      > > As a starting point, we suggest the following:
>      > >
>      > > There are three layers of responsibility:
>      > > 1. Resources of the cloud
>      > > 2. Applications of the cloud
>      > > 3. Layers specific for Murano and Solum (to be further
>     discussed and
>      > >    clarified, for this discussion it's out of scope)
>      > >
>      > > Layer 1 is obviously covered by Heat.
>      > >
>      > > Most of the disagreement is around layer 2. Our suggestion is
>     to figure
>     out
>      > > the way where we can have common engine, DSL and approach to apps
>      > description.
>      > > For this we'll take HOT and TOSCA as a basis and will work on
>     addition
>     of
>      > > functionality missing from Murano and Solum point of view.
>      > >
>      > > We'll be happy if existing Heat team continue working on it,
>     having the
>     full
>      > > control of the project, provided that we agree on functionality
>      > missing there
>      > > from Murano and Solum point of view and addition of this
>     functionality
>     in a
>      > > systematic way.
>      > >
>      > > Let me outline the main concern in regards to HOT from Murano
>     perspective.
>      > > Others concerns could be figured out later. The typical use case
>      > for Murano is
>      > > the following:
>      > > Application writer sets a requirement for her application to
>     use RDBMS
>      > > assuming that it can be MySQL, PostgreSQL or MSSQL. HOT engine
>      > should be able
>      > > to understand such requirements and be able to satisfy them by
>     instantiating
>      > > instances with DB. End user should be able to control which
>      > particular type of
>      > > DB should be used. This is quite similar to abstract requirements
>      > described in
>      > > TOSCA. As Heat wants to cover TOSCA format too this Murano
>     requirement
>     is
>      > > pretty well aligned with what HOT\TOSCA can provide.
>      > >
>      > >
>      > > Hopefully this functionality can be introduced into HOT. Solum and
>      > Murano will
>      > > leverage it by using some common API, and implementing layer 3
>     as thin
>     as
>      > > possible.
>      > >
>      > > Again, this is only suggested starting point, something to
>     begin with.
>      > >
>      > Thanks for bring this discussion back to solving specific
>     problems. The
>      > above database abstraction can be nicely represented using HOT
>      > environments and provider resources [1].
>      >
>      > Take the following template snippet:
>      >
>      > resources:
>      >   my_db:
>      >     type: My::Database
>      >     properties:
>      >       # ...
>      >
>      >   my_server
>      >     type: OS::Nova::Server
>      >     properties:
>      >       # ... including a {get_attr: [my_db, connection_url]}
>      >
>      > My::Database is a type defined in the environment which is
>     included in
>      > the stack-create call. Different environments can provide different
>      > database implementations, for example:
>      >
>      > resource_registry:
>      >   "My::Database": path/to/trove_wrapper.yaml
>      >
>      > resource_registry:
>      >   "My::Database": path/to/nova_galera_ubercluster.yaml
>      >
>      > resource_registry:
>      >   "My::Database": path/to/postgresql.yaml
>      >
>      > Each of these templates could be implemented to have the same
>     parameters
>      > and outputs so the template will work with any of these resource
>      > provider templates.
>      >
>      > One thing we could consider adding to HOT is allow the defining of a
>      > type interface, and validating that a given template/resource
>     implements
>      > its declared interfaces. However the above example works just fine
>      > without interfaces, and this duck-typing approach may provide a
>     better
>      > user experience anyway.
>      >
>      > [1]
>     http://docs.openstack.org/developer/heat/template_guide/environment.html
>      > _______________________________________________
>      > OpenStack-dev mailing list
>      > OpenStack-dev at lists.openstack.org
>     <mailto:OpenStack-dev at lists.openstack.org>
>      > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>     _______________________________________________
>     OpenStack-dev mailing list
>     OpenStack-dev at lists.openstack.org
>     <mailto:OpenStack-dev at lists.openstack.org>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> --
> Sincerely yours
> Stanislav (Stan) Lagun
> Principal Developer
> Mirantis
> 35b/3, Vorontsovskaya St.
> Moscow, Russia
> Skype: stanlagun
> www.mirantis.com <http://www.mirantis.com/>
> slagun at mirantis.com <mailto:slagun at mirantis.com>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>