[openstack-dev] [app-catalog] [glance] [murano] Data Assets API for App Catalog

Flavio Percoco flavio at redhat.com
Wed Oct 28 02:33:27 UTC 2015


On 23/10/15 19:25 +0000, Fox, Kevin M wrote:
>The "Glance: Artefacts API" session was scheduled right on top of the "Nova:
>Cross Service Issues: Service Lock Server, Service Tokens, Instance Users (TBC)
>" session. The latter having been really hard to schedule since it involves so
>many different projects and something I must attend. So I won't be able to
>attend the glance session.
>
>The Murano folks have kindly offered to use one of their sessions:
>Murano contributors meetup (9:00am-12:30pm)
>http://mitakadesignsummit.sched.org/event/5e244fb7f342854dc5c112e76e77c930
>to discuss Glance Artefacts/Murano integration/Community App Catalog needs.
>
>Can folks from the glance team that are interested in the discussion please
>attend?

Kevin, thanks for the heads up! I'll help spreading the word in case
some folks missed this email.

Flavio

>
>Thanks,
>Kevin
>
>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
>From: Alexander Tivelkov [ativelkov at mirantis.com]
>Sent: Thursday, October 15, 2015 3:04 AM
>To: OpenStack Development Mailing List (not for usage questions)
>Subject: [openstack-dev] [app-catalog] [glance] [murano] Data Assets API for
>App Catalog
>
>Hi folks,
>
>I’ve noticed that the Community Application Catalog has begun to implement its
>own API, and it seems to me that we are going to have some significant
>duplication of efforts with the code which has already been implemented in
>Glance as Artifact Repository initiative (also known as Glance V3).
>From the very beginning of the App Catalog project (and I’ve been involved in
>it since February) I’ve been proposing to use Glance as its backend, because
>from my point of view it covers like 90% of the needed functionality. But it
>looks like we have some kind of miscommunication here, as I am getting some
>confusing questions and assumptions, like the vision of Glance V3 being
>dedicated backend for Murano (which is definitely incorrect).
>So, I am writing the email to clarify my vision of what Glance V3 is and how
>its features may be used to provide the REST API for Community App Catalog.
>
>1.  Versioned schema
>First of all, Glance V3 operates on entities called “artifacts”, and these ones
>perfectly map to the Data Assets of community app catalog. The artifacts are
>strongly typed: there are artifact types for murano packages, glance images,
>heat templates - and there may be (and will be) more. Each artifact type is
>represented by a plugin, defining the schema of objects’ data and metadata and
>- optionally - custom logic. So, this thing is extensible: when a new type of
>asset needs to be added to a catalog it can be done really quickly by just
>defining the schema and putting that schema into a plugin. Also, these plugins
>are versioned, so the possible changes in the schema are handled properly. 
>
>2. Generic properties
>Next, all the artifact types in Glance V3 have some generic metadata properties
>(i.e. part of the schema which is common for all the types), including the
>name, the version, description, authorship information and so on. This also
>corresponds to the data schema of community app catalog. The mapping is not
>1:1, but we can work together on this to make sure that these generic
>properties match the expectations of the catalog.
>
>3. Versioning
>Versions are very important for catalogs of objects, so Glance V3 was initially
>designed keeping versioning questions in mind: each artifact has a semver-based
>version assigned, so the artifacts having the same name but different versions
>are grouped into the proper sequences. API is able to query artifacts based on
>their version spec, e.g. it is possible to fetch the latest artifact with the
>name “foo” having the version greater than 2.1 and less than 3.5.7 - or any
>other version spec, similar to pip or any other similar tool. As far as I know,
>community app catalog does not have such capabilities right now - and I
>strongly believe that it is really a must have feature for a catalog to be
>successful. At least it is absolutely mandatory for Murano packages, which are
>the only “real apps” among the asset types right now.
>
>4. Cross artifact dependencies
>Glance V3 also has the dependency relations from the very beginning, so they
>may be defined as part of artifact type schema. As a result, an artifact may
>“reference” any number of other artifacts with various semantic. For example,
>murano package may define a set of references to other murano packages and call
>it “requires” - and this will act similar to the requirements of a python
>package. Similar properties may be defined for heat templates and glance images
>- they may reference each other with various semantics.
>Of course, the definitions of such dependencies may be done internally inside
>the packages, so they may be resolved locally by the service which is going to
>use it, but letting the catalog know about them will allow us to do the
>import-export operations for any given artifacts and its dependencies
>automatically, only by the means of the catalog itself. 
>
>5. Search and filtering API
>Right now Glance V3 API is in experimental state (we plan to stabilize it
>during the Mitaka cycle), but it already provides quite good capabilities to
>discover things. It can search artifacts by their type, name and (optionally)
>aforementioned version specs, by tag or even by arbitrary set of metadata
>properties. We have plans to integrate Glance V3 with the Searchlight project
>to have even more index and search capabilities using its elastic search
>engine.
>
>6. Data storage
>As you probably know, Glance does not own the binary data of its images.
>Instead, it provides an abstraction of the backend storage, which may be swift,
>ceph, s3 or something else. The same approach is used in Glance V3 for
>artifacts data, but with more per-type control: particular artifact types may
>be configured independently to store their blobs in different backends. This
>may be of use for Community App Catalog which operates on different storages
>for its assets. 
>
>7. Sharing and access control. 
>Glance V3 inherits the same access mechanics present in Glance V2: an artifact
>may be visible to its owner tenant only, be public (i.e. visible to all the
>tenants) or directly shared by the owner to a specific tenant. Also, Glance can
>act in the anonymous mode (i.e. without an access token), thus providing access
>to public artifacts even to unauthenticated users. 
>This can be easily applied to a public web service, such as community app
>catalog: regular unauthenticated users use anonymous mode to access only the
>public assets (this is the current behavior of apps.o.o), while registered
>users will have their own private spaces (“tenants”) with various access
>restrictions.
>
>8. The federation. 
>The ultimate goal for Glance Artifact Repository is ability to build trees of
>artifact repos in different clouds. The top node of that tree is some Global
>Application Catalog (and the apps.openstack.org may be this global catalog - if
>it is glance-based or at least supports glance v3 federation), then there are
>repositories of particular openstack vendors or distributors, then - the
>catalogs of enterprises operating different openstack clouds. The particular
>glance deployments in that clouds are the leafs of that tree, being able to
>search for data assets in all the upstream repositories, download them from
>there or - if permitted - submit their local assets back upstream. This will be
>the ultimate network for application delivery and exchange in openstack world -
>and this is one of the main reasons we’ve began the Artifacts initiative in
>Glance. 
>Unlike other aforementioned features this one is not implemented yet, but we
>are planning to add it as soon as we are done with API stabilization goal.
> 
>
>There are many other features which are present in V3’s roadmap and may be
>useful for the app catalog, such as ability to sign artifacts with their
>developers’ keys and verify that keys on usage to ensure the authenticity of
>the artifact.
>
>What we don’t have right now is the ability to associate ratings (“stars”) and
>comments to the artifact, as well as aggregating different usage and download
>statistics: such features are really needed only for the public website such as
>apps.o.o but are not required for Glance’s in particular clouds. But we may
>find some way to solve this, either by wrapping glance API with additional
>middleware which would add appropriate info from a different data source, or by
>having custom plugins which are able to do that, or in some other way: I am
>sure we may find a solution for this.
>
>So, this was just a brief description of what Glance v3 has to offer as a
>backend for App Catalog API. 
>It also worths to mention that this API is in “EXPERIMENTAL” state right now,
>which means that it is not fixed and we may modify it significantly if there is
>a need to. So we may work closer together to adopt it for the needs of
>Community App Catalog.
>
>I would really prefer to not create any overlaps between Glance v3 and the
>community app catalog: if the app catalog builds its own incompatible
>implementation of assets discovery and distribution API then we’ll have a huge
>duplication of efforts for developers and lots of confusion to the end-users
>who will get two entirely different ways to do the same task.
>
>So, I’d propose to discuss these potential overlaps, look at the features need
>by App Catalog and see how Glance V3 may be of use here. I’ll be more than
>happy to help with that. We can dive deeper into the details here in the
>mailing list or meet in person in Tokyo. I'll try to have a demonstratable
>prototype by that time.
>
>--
>Regards,
>Alexander Tivelkov

>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-- 
@flaper87
Flavio Percoco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151028/6371ed5c/attachment-0001.pgp>


More information about the OpenStack-dev mailing list