Open Stack

Colleagues,
having attended many sessions and talked to many customers, partners
and contributors in Austin I’d like to suggest several improvements to how
we develop OpenStack apps and work with the Community App Catalog (
https://apps.openstack.org/).

Key goals to achieve are:
- Provide contributors with an ability to collaborate on OpenStack
apps development
- Provide contributors and consumers with transparent workflow to
manage their apps
- Provide consumers with information about apps - how it was developed
and tested
- To summarize - introduce the way to build community working on
OpenStack apps

*What is OpenStack application*
OpenStack is about 6 years young and all these years discussions about
it are in progress. Variety of applications is huge, from LAMP stacks
and legacy Java apps to telco workloads and VNF apps. There is working
group which works on a definition of "What is OpenStack application",
hopefully community will agree on definition soon.

For the sake of our discussion below let us agree on a simple approach:
an OpenStack application is any software asset which 1. can be executed on
an OpenStack cloud, 2. lives in apps.openstack.org.  So far there are
Murano applications, Heat templates, Glance images and TOSCA templates.

There are many good OpenStack applications in the world which don't live in
OpenStack App Catalog. However, let us for now concentrate on those which
do, just for the sake of this discussion.

*Introduction to OpenStack development ecosystem*
OpenStack was introduced about 6 years ago. Over these years
community grown significantly. There were 8 companies contributing to
OpenStack in Austin (1-st release). In Mitaka (13-th release) there were 64
companies contributing.

One of the reasons for this growth is the set of sophisticated tools
the OpenStack contributor ecosystem has chosen to use or build to
enable collaboration:
- software repositories: http://git.openstack.org/cgit/openstack/nova,
http://git.openstack.org/cgit/openstack/neutron, ..
- bug trackers: https://launchpad.net/nova, https://launchpad.net/neutron
, ...
- same instance of gerrit for all the projects for code review:
https://review.openstack.org/
- gating test infrastructure http://zuul.openstack.org/
- common approach to release management, repositories management,
naming, tons of other things managed by review in
https://review.openstack.org/#/q/status:open+project:openstack/governance
- IRC channels, etherpads, meetings and mailing lists
- governance to manage all of the above

All of the above is what we can call OpenStack collaboration ecosystem
and it is a key factor for OpenStack community success.

*Introduction to OpenStack apps development ecosystem*
Now when OpenStack is mature and have it up and running is not a big
deal, focus of community and customers shifts from "how do I get a running
cloud" to "what do I do with running cloud".

Use cases of different cloud users are very different, however one
can identify and develop standard building blocks which can be reused by
cloud users (service providers, DevTest teams, ...). Many cloud users want
to contribute their homegrown apps upstream because:
- it allows to other people to use it and improve it
- community can implement missing parts
- community can help with testing and maintaining an app

Year ago we introduced Community App Catalog for OpenStack
http://apps.openstack.org as an integration/distribution point of
customer experience/apps. This initiative is successful, there are about
100 software assets of various kinds which can be run on OpenStack. For
further growth we need to make several changes in a way we approach
collaboration around OpenStack Apps. Namely, we need to provide an ability
to apps developers to collaborate on application development.

*OpenStack Community App Catalog is there, what else?*
Community App Catalog http://apps.openstack.org allows to
publish/consume apps to/from it.

"The OpenStack Community App Catalog is designed to use the same tools
for submission and review as other OpenStack projects. As such we follow
the OpenStack development workflow" [0].

To follow OpenStack development workflow, apps developers need to have:
- dedicated repositories & code review system to collaborate on code
- mailing lists, IRC channels, core reviewers teams
- common approach to QA
- governance model to manage all of the above

Most of the above is missing for apps developers now.  App Catalog
provides central place to store final artifacts (ready apps, like .exe
files in Win world) but there is no centralized infrastructure to
collaborate on development of source code of apps.

Apps developers partially use infrastructure of OpenStack core
projects (Heat & Murano) - repositories and bug trackers. Other than that
they are on their own, there are no teams, no mailing lists, no IRC
channels for apps developers - most of the items from the list above is
missing.

[0] https://wiki.openstack.org/wiki/App-Catalog#How_to_contribute

*All right, we need to change something. What exactly?*

1.  We need to introduce a team which manages content of Community
App Catalog, decides which new assets can be added, decides on a workflow
for apps publishing, maintaining, consuming. This team could be a
complimentary team working alongside the Community App Catalog
implementation team (engine, backend, frontend); or within the team itself

2. There should be separated repositories for source code of apps
from Community App Catalog. These repositories should not be stored
under openstack label as they do not relate to core OpenStack projects.
Core project teams are not responsible for maintaining apps.

3. Bug tracking for apps should be separated from bug tracking for
core projects.

4. There should be teams working on apps with core reviewers, IRC
channels and mailing lists. These teams should differ from core projects
teams.

5. Given #1 - #4 it seems reasonable to develop governance model
specific for OpenStack Apps Community, which differs (when it’s necessary)
from governance model of OpenStack Community.

Let us develop such a governance model, implement changes described
above and build community of OpenStack apps developers.

PS. There is representative discussion in comments to
https://review.openstack.org/#/c/309383/. Some team wants to add new
repository for CI/CD Murano app. Should it be part of Murano core project?
Rather not. Should it be part of BigTent? Well, rather not as BigTent is
for core OpenStack services, not for workloads on top of it. At the same
time team wants to use some OpenStack infra resources (at least gerrit for
now) as this project is obviously beneficial for OpenStack. We need to have
an ability to resolve similar requests in a centralized manner - there are
more teams who want to publish
source code of their OpenStack apps and there is no established
workflow for that.

*Agree. What’s next?*
Suggested plan:
- Introduce label openstack-apps, put repositories with source code
of OpenStack Apps under it, i.e.:
  * http://git.openstack.org/cgit/openstack-apps/murano-apps
  * http://git.openstack.org/cgit/openstack-apps/heat-templates
  * http://git.openstack.org/cgit/openstack-apps/tosca-workflows
- Agree with OpenStack Community App Catalog team on how content of
App Catalog is managed and by whom
- Describe workflow of how to add source code of new application
to repositories, who approves its addition
- Introduce simplified workflow of publishing new Application to
the catalog:
  * register/login
  * push/update
  * done
- Introduce teams (core reviewers) contributing to/maintaining Murano
apps, Heat templates, ...
- Establish channels of communications with potential contributors
(mailing list, meetings, IRC/slack channel, ... )
- Agree with contributors on QA process for applications and how we
track it in Community App Catalog

To simplify commenting and tracking of the plan above I put last
two sections with suggested steps to the etherpad
https://etherpad.openstack.org/p/OpenStackAppsCommunity

We're also discussing this topic with OpenStack Application Ecosystem
Working Group and several members of OpenStack App Catalog team support
this idea:
http://lists.openstack.org/pipermail/user-committee/2016-May/000854.html

Please share your thoughts and comments.

Regards,
Igor Marnat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160525/25201577/attachment.html>