[openstack-dev] [app-catalog] [glance] [murano] Data Assets API for App Catalog

Ian Cordasco ian.cordasco at RACKSPACE.COM
Sat Oct 31 18:41:08 UTC 2015

Does that mean that https://review.openstack.org/177397 will be updated to
fully address the API-WG and Defcore feedback? How soon can we expect the
spec to reflect that feedback and the code will be updated for it?

On 10/30/15, 18:54, "Alexander Tivelkov" <ativelkov at mirantis.com> wrote:

>Hi folks,
>Thanks for the fruitful discussion on the summit: seems like we are in
>agreement and have a good plan to proceed.
>Here is the video recording of the PoC I've made to serve the current
>AppCatalog's data using Glance Artifacts. As the demo was recorded before
>the summit, it's called Glance V3 everywhere. Now we've agreed that this
>will be a Glance Artifacts Repository
> (aka Glare) API v1 - but everything else about its functionality is
>still valid :)
>Please feel free to ask any questions you may have on the demo and Glare
>functionality in general.
>Thanks again
>On Wed, Oct 28, 2015 at 11:38 AM Flavio Percoco <flavio at redhat.com> wrote:
>On 23/10/15 19:25 +0000, Fox, Kevin M wrote:
>>The "Glance: Artefacts API" session was scheduled right on top of the
>>Cross Service Issues: Service Lock Server, Service Tokens, Instance
>>Users (TBC)
>>" session. The latter having been really hard to schedule since it
>>involves so
>>many different projects and something I must attend. So I won't be able
>>attend the glance session.
>>The Murano folks have kindly offered to use one of their sessions:
>>Murano contributors meetup (9:00am-12:30pm)
>>to discuss Glance Artefacts/Murano integration/Community App Catalog
>>Can folks from the glance team that are interested in the discussion
>Kevin, thanks for the heads up! I'll help spreading the word in case
>some folks missed this email.
>>From: Alexander Tivelkov [ativelkov at mirantis.com]
>>Sent: Thursday, October 15, 2015 3:04 AM
>>To: OpenStack Development Mailing List (not for usage questions)
>>Subject: [openstack-dev] [app-catalog] [glance] [murano] Data Assets API
>>App Catalog
>>Hi folks,
>>I’ve noticed that the Community Application Catalog has begun to
>>implement its
>>own API, and it seems to me that we are going to have some significant
>>duplication of efforts with the code which has already been implemented
>>Glance as Artifact Repository initiative (also known as Glance V3).
>>From the very beginning of the App Catalog project (and I’ve been
>>involved in
>>it since February) I’ve been proposing to use Glance as its backend,
>>from my point of view it covers like 90% of the needed functionality.
>>But it
>>looks like we have some kind of miscommunication here, as I am getting
>>confusing questions and assumptions, like the vision of Glance V3 being
>>dedicated backend for Murano (which is definitely incorrect).
>>So, I am writing the email to clarify my vision of what Glance V3 is and
>>its features may be used to provide the REST API for Community App
>>1.  Versioned schema
>>First of all, Glance V3 operates on entities called “artifacts”, and
>>these ones
>>perfectly map to the Data Assets of community app catalog. The artifacts
>>strongly typed: there are artifact types for murano packages, glance
>>heat templates - and there may be (and will be) more. Each artifact type
>>represented by a plugin, defining the schema of objects’ data and
>>metadata and
>>- optionally - custom logic. So, this thing is extensible: when a new
>>type of
>>asset needs to be added to a catalog it can be done really quickly by
>>defining the schema and putting that schema into a plugin. Also, these
>>are versioned, so the possible changes in the schema are handled
>>2. Generic properties
>>Next, all the artifact types in Glance V3 have some generic metadata
>>(i.e. part of the schema which is common for all the types), including
>>name, the version, description, authorship information and so on. This
>>corresponds to the data schema of community app catalog. The mapping is
>>1:1, but we can work together on this to make sure that these generic
>>properties match the expectations of the catalog.
>>3. Versioning
>>Versions are very important for catalogs of objects, so Glance V3 was
>>designed keeping versioning questions in mind: each artifact has a
>>version assigned, so the artifacts having the same name but different
>>are grouped into the proper sequences. API is able to query artifacts
>>based on
>>their version spec, e.g. it is possible to fetch the latest artifact
>>with the
>>name “foo” having the version greater than 2.1 and less than 3.5.7 - or
>>other version spec, similar to pip or any other similar tool. As far as
>>I know,
>>community app catalog does not have such capabilities right now - and I
>>strongly believe that it is really a must have feature for a catalog to
>>successful. At least it is absolutely mandatory for Murano packages,
>>which are
>>the only “real apps” among the asset types right now.
>>4. Cross artifact dependencies
>>Glance V3 also has the dependency relations from the very beginning, so
>>may be defined as part of artifact type schema. As a result, an artifact
>>“reference” any number of other artifacts with various semantic. For
>>murano package may define a set of references to other murano packages
>>and call
>>it “requires” - and this will act similar to the requirements of a python
>>package. Similar properties may be defined for heat templates and glance
>>- they may reference each other with various semantics.
>>Of course, the definitions of such dependencies may be done internally
>>the packages, so they may be resolved locally by the service which is
>>going to
>>use it, but letting the catalog know about them will allow us to do the
>>import-export operations for any given artifacts and its dependencies
>>automatically, only by the means of the catalog itself.
>>5. Search and filtering API
>>Right now Glance V3 API is in experimental state (we plan to stabilize it
>>during the Mitaka cycle), but it already provides quite good
>>capabilities to
>>discover things. It can search artifacts by their type, name and
>>aforementioned version specs, by tag or even by arbitrary set of metadata
>>properties. We have plans to integrate Glance V3 with the Searchlight
>>to have even more index and search capabilities using its elastic search
>>6. Data storage
>>As you probably know, Glance does not own the binary data of its images.
>>Instead, it provides an abstraction of the backend storage, which may be
>>ceph, s3 or something else. The same approach is used in Glance V3 for
>>artifacts data, but with more per-type control: particular artifact
>>types may
>>be configured independently to store their blobs in different backends.
>>may be of use for Community App Catalog which operates on different
>>for its assets.
>>7. Sharing and access control.
>>Glance V3 inherits the same access mechanics present in Glance V2: an
>>may be visible to its owner tenant only, be public (i.e. visible to all
>>tenants) or directly shared by the owner to a specific tenant. Also,
>>Glance can
>>act in the anonymous mode (i.e. without an access token), thus providing
>>to public artifacts even to unauthenticated users.
>>This can be easily applied to a public web service, such as community app
>>catalog: regular unauthenticated users use anonymous mode to access only
>>public assets (this is the current behavior of apps.o.o), while
>>users will have their own private spaces (“tenants”) with various access
>>8. The federation.
>>The ultimate goal for Glance Artifact Repository is ability to build
>>trees of
>>artifact repos in different clouds. The top node of that tree is some
>>Application Catalog (and the
>apps.openstack.org <http://apps.openstack.org> may be this global catalog
>- if
>>it is glance-based or at least supports glance v3 federation), then
>>there are
>>repositories of particular openstack vendors or distributors, then - the
>>catalogs of enterprises operating different openstack clouds. The
>>glance deployments in that clouds are the leafs of that tree, being able
>>search for data assets in all the upstream repositories, download them
>>there or - if permitted - submit their local assets back upstream. This
>>will be
>>the ultimate network for application delivery and exchange in openstack
>>world -
>>and this is one of the main reasons we’ve began the Artifacts initiative
>>Unlike other aforementioned features this one is not implemented yet,
>>but we
>>are planning to add it as soon as we are done with API stabilization
>>There are many other features which are present in V3’s roadmap and may
>>useful for the app catalog, such as ability to sign artifacts with their
>>developers’ keys and verify that keys on usage to ensure the
>>authenticity of
>>the artifact.
>>What we don’t have right now is the ability to associate ratings
>>(“stars”) and
>>comments to the artifact, as well as aggregating different usage and
>>statistics: such features are really needed only for the public website
>>such as
>>apps.o.o but are not required for Glance’s in particular clouds. But we
>>find some way to solve this, either by wrapping glance API with
>>middleware which would add appropriate info from a different data
>>source, or by
>>having custom plugins which are able to do that, or in some other way: I
>>sure we may find a solution for this.
>>So, this was just a brief description of what Glance v3 has to offer as a
>>backend for App Catalog API.
>>It also worths to mention that this API is in “EXPERIMENTAL” state right
>>which means that it is not fixed and we may modify it significantly if
>>there is
>>a need to. So we may work closer together to adopt it for the needs of
>>Community App Catalog.
>>I would really prefer to not create any overlaps between Glance v3 and
>>community app catalog: if the app catalog builds its own incompatible
>>implementation of assets discovery and distribution API then we’ll have
>>a huge
>>duplication of efforts for developers and lots of confusion to the
>>who will get two entirely different ways to do the same task.
>>So, I’d propose to discuss these potential overlaps, look at the
>>features need
>>by App Catalog and see how Glance V3 may be of use here. I’ll be more
>>happy to help with that. We can dive deeper into the details here in the
>>mailing list or meet in person in Tokyo. I'll try to have a
>>prototype by that time.
>>Alexander Tivelkov
>>OpenStack Development Mailing List (not for usage questions)
>OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>Flavio Percoco
>OpenStack Development Mailing List (not for usage questions)
>OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>Alexander Tivelkov

More information about the OpenStack-dev mailing list