[openstack-dev] [new][app-catalog] App Catalog next steps

Fox, Kevin M Kevin.Fox at pnnl.gov
Fri May 29 16:55:34 UTC 2015


I've ran into the opposite problem though with Glance. As an Op, I really really want to replace one image with a new one atomically with security updates preapplied. Think shellshock, ghost, etc. It will be basically be the same exact image as before, but patched. Referencing local ID's explicitly makes it harder to ensure things are patched, since new vm's will tend to pop up after things are patched with new vulnerabilities.

Thanks,
Kevin

________________________________
From: Alexander Tivelkov [ativelkov at mirantis.com]
Sent: Friday, May 29, 2015 9:24 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [new][app-catalog] App Catalog next steps

Hi Kevin,

I don't suggest to use random IDs as artifact identifiers in the community app catalog. Of course we will need to have some globally unique names there (my initial idea on that is to have a combination of fully-qualified namespace-based name + version + signature) - and such names should be used to replicate artifacts across the cloud boundaries.

By "Referencing by ID" I mean only the local referencing: when the artifact is already present in cloud's local Glance (be it imported from the community catalog, copied from other cloud or uploaded directly), the particular service (Heat in our example) should be able to consume it by ID, same as Nova currently does with Images.
This has its own purpose to guarantee objects' immutability: once the user has selected an object in the local catalog, she may be sure that nobody will interfere and modify it, as the object itself is immutable and the id is not reusable. If the object is referenced only by name, then it may be deleted and a different artifact with the same name may be uploaded instead, which may introduce a potential security issue. Using IDs will prevent such behavior.
Fully qualified object names are still needed, of course - but that's Glance goal to locate an artifact based on its FQN and return the id for it.
At least, this was the design idea of the initial artifact concept.

But that's an off-topic here, as this concept is related only to the local artifact repos, and world-global app catalog has nothing to do with this.


--
Regards,
Alexander Tivelkov

On Fri, May 29, 2015 at 6:47 PM, Fox, Kevin M <Kevin.Fox at pnnl.gov<mailto:Kevin.Fox at pnnl.gov>> wrote:
Hi Alexander,

Sweet. I'll have to kick the tires on the current state of Liberty soon. :)

Reference by artifact IDs is going to be problematic I think. How do you release a generic set of resources to the world that reference specific randomly generated ID's?

What about by Name? If not, then it will need to be some kind of mapping mechanism. :/

Thanks,
Kevin

________________________________
From: Alexander Tivelkov [ativelkov at mirantis.com<mailto:ativelkov at mirantis.com>]
Sent: Friday, May 29, 2015 4:19 AM

To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [new][app-catalog] App Catalog next steps

Hi Kevin,

Has the Glance Artifact Repository implemented enough bits to have Heat and/or Murano artefacts yet?

​Most of the code is there already, couple of patchsets are still on review but we'll land them soon.​ L1 is a likely milestone to have it ready in master.


Also, has there been any work on Exporting/Importing them through some defined format (tarball?) that doesn't depend on the artefact type?

​This one is not completely implemented: the design is ready (the spec had this feature from the very beginning) and a PoC was done. The final implementation is likely to happen in L cycle.


I've been talking with the Heat folks on starting a blueprint to allow heat templates to use relative URL's instead of absolute ones. That would allow a set of Heat templates to be stored in one artefact in Glance.

​That's awesome.
Also I'd consider allowing Heat to reference Templates by their artifact IDs in Glance, same as Nova does it for images. ​


________________________________
From: Alexander Tivelkov [ativelkov at mirantis.com<mailto:ativelkov at mirantis.com>]
Sent: Thursday, May 28, 2015 4:46 AM

To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [new][app-catalog] App Catalog next steps

Hi folks,

I believe that at least part of the filtering we are discussing here may be done at the client side if the client is sophisticated enough to be aware about the capabilities of the local cloud.
And by "sophisticated client" I mean "Glance V3" (previously known as "Artifact Repository"), which may (and, in my vision, should) become the ultimate consumer of the app catalog on the cloud side.

Each asset type (currently Image, Murano Package, Heat template, more to come) should be implemented as Glance Artifact type (i.e. a plugin), and may define the required capabilities as its type specific metadata fields (for example, Heat-template type may list plugins which are required to run this template; Murano-package type may set the minimum required version of Core library etc). The logic which is needed to validate this capabilities may be put into this type-specific plugin as well. This custom logic method will gets executed when the artifact is being exported from app catalog into the particular cloud.

In this case the compatibility of particular artifact with particular cloud will be validated by that cloud itself when the app catalog is browsed. Also, if the cloud does not have support of some artifact types at all (e.g. does not have Murano installed and thus cannot utilize Murano Packages), then it does not have the Murano plugin in its glance and thus will not be able to import murano-artifacts from the Catalog.

Hope this makes sense.


--
Regards,
Alexander Tivelkov

On Thu, May 28, 2015 at 10:29 AM, Morgan Fainberg <morgan.fainberg at gmail.com<mailto:morgan.fainberg at gmail.com>> wrote:


On Wed, May 27, 2015 at 5:33 PM, Joe Gordon <joe.gordon0 at gmail.com<mailto:joe.gordon0 at gmail.com>> wrote:


On Wed, May 27, 2015 at 4:27 PM, Fox, Kevin M <Kevin.Fox at pnnl.gov<mailto:Kevin.Fox at pnnl.gov>> wrote:
I'd say, tools that utilize OpenStack, like the knife openstack plugin, are not something that you would probably go to the catalog to find. And also, the recipes that you would use with knife would not be specific to OpenStack in any way, so you would just be duplicating the config management system's own catalog in the OpenStack catalog, which would be error prone. Duplicating all the chef recipes, and docker containers, puppet stuff, and ..... is a lot of work...

I am very much against duplicating things, including chef recipes that use the openstack plugin for knife. But we can still easily point to external resources from apps.openstack.org<http://apps.openstack.org>. In fact we already do (http://apps.openstack.org/#tab=heat-templates&asset=Lattice).


The vision I have for the Catalog (I can be totally wrong here, lets please discuss) is a place where users (non computer scientists) can visit after logging into their Cloud, pick some app of interest, hit launch, and optionally fill out a form. They then have a running piece of software, provided by the greater OpenStack Community, that they can interact with, and their Cloud can bill them for. Think of it as the Apple App Store for OpenStack.  Having a reliable set of deployment engines (Murano, Heat, whatever) involved is critical to the experience I think. Having too many of them though will mean it will be rare to have a cloud that has all of them, restricting the utility of the catalog. Too much choice here may actually be a detriment.


calling this a catalog, which it sounds accurate, is confusing since keystone already has a catalog.   Naming things is unfortunately a difficult problem.

This is in itself turns into a really unfortunately usability issue for a number of reason; colliding namespaces that end users need to be aware of serves to generate confusion. Even the choices made naming things currently in use by OpenStack (I openly admit Keystone is particularly bad in this light) have this issue. I would support a "catalog-like" name that limits confusion especially when it comes to conveying this information to the end users (not just deployers and operators).

I will reiterate Joe's statement: Naming things is unfortunately a difficult problem.


I respectfully disagree with this vision. I mostly agree with the first part about it being somewhere users can go to find applications that can be quickly deployed on OpenStack (note all the gotchas that Monty described here). The part I disagree with is about limiting the deployment engines to invented here. Even if we have 100 deployment engines on apps.openstack.org<http://apps.openstack.org>, it would be very easy for a user to filter by the deployment engines they use so I do not agree with your concern about too many choices here being a detriment (after all isn't OpenStack about choices?).


++

We should be as inclusive as we can be. There are many cases of prior art where (as long as it's workable) we can do filtering (someone brought up the mobile app stores). Even if we want to be measured in ensuring the filtering works before opening the flood gates, allowing alternate deployment engines is a good thing. It makes OpenStack more usable and more desirable as a platform

Secondly IMHO the notion that 'if it wasn't invented here we shouldn't support it' [0] is a dangerous one that results on us constantly re-inventing the wheel while alienating the larger developer community by saying there solutions are no good, you should use the OpenStack version of it.


OpenStack isn't a single 'thing' it is a collection of 'things' and user's should be able to pick and choose which components they want and which components they want to get from elsewhere.

[0] http://en.wikipedia.org/wiki/Not_invented_here


If chef, or what ever other configuration management system became multitenant aware, and integrated into OpenStack and provided by the Cloud providers, then maybe it would fit into the app store vision?

I am not sure why this matters?  As a dependency you simply state chef, and either require users to provide it or tell them to use a chef heat template, glance image, etc.


Thanks,
Kevin
________________________________
From: Joe Gordon [joe.gordon0 at gmail.com<mailto:joe.gordon0 at gmail.com>]
Sent: Wednesday, May 27, 2015 3:20 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [new][app-catalog] App Catalog next steps



On Fri, May 22, 2015 at 9:06 PM, Christopher Aedo <caedo at mirantis.com<mailto:caedo at mirantis.com>> wrote:
I want to start off by thanking everyone who joined us at the first
working session in Vancouver, and those folks who have already started
adding content to the app catalog. I was happy to see the enthusiasm
and excitement, and am looking forward to working with all of you to
build this into something that has a major impact on OpenStack
adoption by making it easier for our end users to find and share the
assets that run on our clouds.

Great job. This is very exciting to see, I have been wanting something like this for some time now.


The catalog: http://apps.openstack.org
The repo: https://github.com/stackforge/apps-catalog
The wiki: https://wiki.openstack.org/wiki/App-Catalog

Please join us via IRC at #openstack-app-catalog on freenode.

Our initial core team is Christopher Aedo, Tom Fifield, Kevin Fox,
Serg Melikyan.

I’ve started a doodle poll to vote on the initial IRC meeting
schedule, if you’re interested in helping improve and build up this
catalog please vote for the day/time that works best and get involved!
http://doodle.com/vf3husyn4bdkui8w

At the summit we managed to get one planning session together. We
captured that on etherpad[1], but I’d like to highlight here a few of
the things we talked about working on together in the near term:

-More information around asset dependencies (like clarifying
requirements for Heat templates or Glance images for instance),
potentially just by providing better guidance in what should be in the
description and attributes sections.
-With respect to the assets that are listed in the catalog, there’s a
need to account for tagging, rating/scoring, and a way to have
comments or a forum for each asset so potential users can interact
outside of the gerrit review system.
-Supporting more resource types (Sahara, Trove, Tosca, others)

What about expanding the scope of the application catalog to any application that can run *on* OpenStack, versus the implied scope of applications that can be deployed *by* (heat, murano, etc.) OpenStack and *on* OpenStack services (nova, cinder etc.). This would mean adding room for Ansible roles that provision openstack resources [0]. And more generally it would reinforce the point that there is no 'blessed' method of deploying applications on OpenStack, you can use tools developed specifically for OpenStack or tools developed elsewhere.


[0] https://github.com/ansible/ansible-modules-core/blob/1f99382dfb395c1b993b2812122761371da1bad6/cloud/openstack/os_server.py

-Discuss using glance artifact repository as the backend rather than
flat YAML files
-REST API, enable searching/sorting, this would ease native
integration with other projects
-Federated catalog support (top level catalog including contents from
sub-catalogs)
- I’ll be working with the OpenStack infra team to get the server and
CI set up in their environment (though that work will not impact the
catalog as it stands today).

I am pleased to see moving this to OpenStack Infra is a high priority.

A quick nslookup of http://apps.openstack.org shows it us currently hosted on linode at http://nb-23-239-6-45.fremont.nodebalancer.linode.com/. And last I checked linode isn't OpenStack powered.  apps.openstack.org<http://apps.openstack.org> is a great example of the type of application that should be easy to deploy with OpenStack, since as far as I can tell it just needs a web server and that is it. So wearing my OpenStack developer hat on, why did you go with linode and not any one of the OpenStack based public clouds [1]? If OpenStack is not a good solution for workloads like this, then it would be great to know how what needs work.


[1] https://www.openstack.org/marketplace/public-clouds/


There were a ton of great ideas that came up and it was clear there
was WAY more to discuss than we could accomplish in one short session
at the summit.  I’m looking forward to continuing the conversation
here on the mailing list, on IRC, and in Tokyo as well!

[1] https://etherpad.openstack.org/p/YVR-app-catalog-plans

-Christopher

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150529/1b6c5927/attachment-0001.html>


More information about the OpenStack-dev mailing list