<div dir="ltr">Colleagues,<div>let me share an update with our progress with initiative I sent across in late May about improving our collaboration around development of OpenStack applications and Community App Catalog. Several teams have been busily working on it, including App Catalog team, Glare/Glance, Murano and Murano Apps.</div><div><br></div><div>You can also find suggested plan and status update quoted below in the second part of this etherpad </div><div><a href="https://etherpad.openstack.org/p/OpenStackAppsCommunity" target="_blank">https://etherpad.openstack.org/p/OpenStackAppsCommunity</a>.</div><br><b>Suggested plan and progress with it</b><div><b><br>1. Introduce label openstack-apps, put repositories with source code of OpenStack Apps under it, i.e.:</b><br> * <a href="http://git.openstack.org/cgit/openstack-apps/murano-apps" target="_blank">http://git.openstack.org/cgit/openstack-apps/murano-apps</a><br> * <a href="http://git.openstack.org/cgit/openstack-apps/heat-templates" target="_blank">http://git.openstack.org/cgit/openstack-apps/heat-templates</a><br> * <a href="http://git.openstack.org/cgit/openstack-apps/tosca-workflows" target="_blank">http://git.openstack.org/cgit/openstack-apps/tosca-workflows</a><br><b> Status: </b><br> OpenStack Infra team suggested not to introduce yet another label. Thus we just changed ownership of <a href="http://git.openstack.org/cgit/openstack/murano-apps" target="_blank">http://git.openstack.org/cgit/openstack/murano-apps</a> repository to murano-apps-core team. Didn't make progress with other projects (Heat, Tosca), TBD if they need/want it.<br> <br>Besides we agreed with OpenStack Infra that for complicated Murano Applications we'll be creating and maintaining separated repositories, specific for application (<a href="http://lists.openstack.org/pipermail/openstack-dev/2016-June/096471.html" target="_blank">http://lists.openstack.org/pipermail/openstack-dev/2016-June/096471.html</a>). I.e. we use <a href="http://git.openstack.org/cgit/openstack/murano-apps" target="_blank">http://git.openstack.org/cgit/openstack/murano-apps</a> for simple applications and dedicated repositories for mostly complex apps (i.e. <a href="http://git.openstack.org/cgit/openstack/ci-cd-pipeline-app-murano" target="_blank">http://git.openstack.org/cgit/openstack/ci-cd-pipeline-app-murano</a>).<br> <br><b>2. Agree with OpenStack Community App Catalog team on how content of App Catalog is managed and by whom</b><br><b>Status:</b><br>To change an approach to managing content in Community App Catalog we need first to switch its backend to Glare. Current implementation of backend doesn't allow to make changes in apps management workflow, it's mostly manual and has to be done by App Catalog Core team. ETA for switching - mid/end Aug, 2016 (work in progress).<br><br>There is a test instance of App Catalog based on Glare V 1.0, <a href="http://r-ci.tk:8100" target="_blank">http://r-ci.tk:8100</a>. It is a very early version under development, can be updated/turned off w/o prior notification. App Catalog team works on implementation of existing App Catalog functionality based on this backend. <br><br>App Catalog team is about to start discussions on what new upload workflows look like. Please follow [app-catalog] mailing list on openstack-dev or come to #openstack-app-catalog in IRC for more information.<br><br><b>3. Describe workflow of how to add source code of new application to repositories, who approves its addition</b><br><b>Status: </b><br>App Catalog team is about to start discussing this/working on it once switched to new backend. Please follow [app-catalog] mailing list on openstack-dev or come to openstack-app-catalog in IRC for updates.<br><br><b>4. Introduce simplified workflow of publishing new Application to the catalog:</b><br> * register/login<br> * push/update<br> * done<br><b>Status:</b><br>See #2, #3<br> <br><b>5. Introduce teams (core reviewers) contributing to/maintaining Murano apps, Heat templates, ...</b><br><b>Status:</b><br>Done for Murano (introduced murano-apps-core team): <a href="http://lists.openstack.org/pipermail/openstack-dev/2016-May/096268.html" target="_blank">http://lists.openstack.org/pipermail/openstack-dev/2016-May/096268.html</a><br><br><b>6. Establish channels of communications with potential contributors (mailing list, meetings, IRC/slack channel, ... ?)</b><br><b>Status:</b><br>Introduced murano-apps IRC channel, rest will be added working on #3.<br><br><b>7. Agree with contributors on QA process for applications and how we track it in Community App Catalog:</b><br>7.1 Get from Heat, Murano and TOSCA team list of documentation resources, tools and best practices on QA of templates, apps and workflows<br>7.2 Make this list structured and visible for apps developers and apps users in app-catalog documentation<br>7.3 Implement automated checks of apps integrity using tests/checks already available in Heat, Glance, Murano, TOSCA<br>7.4 Discuss with Heat, Murano and TOSCA teams additions to their roadmaps for verification tools for their assets</div><div><br><b>Status:</b><br> Good progress with 7.3. Most of the results described below are work in progress, still needs to be polished and documented. However, they allow to understand progress and approach<br> - implemented yaml + bash syntax check for Murano Apps ( <a href="https://review.openstack.org/#/c/344922/" target="_blank">https://review.openstack.org/#/c/344922/</a> )<br> - implemented third-party CI for Murano Apps for <a href="http://git.openstack.org/cgit/openstack/ci-cd-pipeline-app-murano" target="_blank">http://git.openstack.org/cgit/openstack/ci-cd-pipeline-app-murano</a> (see for example feedback from murano-ci-cd-bot in <a href="https://review.openstack.org/#/c/316209/" target="_blank">https://review.openstack.org/#/c/316209/</a>). <br> - validation of TOSCA blueprints on upload to Glare v 1.0 using tosca-parser (<a href="https://review.openstack.org/#/c/337633/" target="_blank">https://review.openstack.org/#/c/337633/</a>). If template doesn't pass validation it can't be uploaded to the catalog. To be elaborated in the future based on new upload workflows we agree with community.</div><div> - validation of Heat templates and Glance images is being discussed with Heat and Glance teams correspondingly<br> <br> Next steps - improve 7.3, work on 7.1, 7.2, 7.4, and to discuss with OpenStack Infra an approach to switch CIs from third-party CIs to OpenStack Infra CIs.</div><div><br></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Regards,<div>Igor Marnat</div></div></div></div>
<br><div class="gmail_quote">On Wed, May 25, 2016 at 2:37 PM, Igor Marnat <span dir="ltr"><<a href="mailto:imarnat@mirantis.com" target="_blank">imarnat@mirantis.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Colleagues,<br>having attended many sessions and talked to many customers, partners and contributors in Austin I’d like to suggest several improvements to how we develop OpenStack apps and work with the Community App Catalog (<a href="https://apps.openstack.org/" target="_blank">https://apps.openstack.org/</a>).<br><br>Key goals to achieve are:<br>- Provide contributors with an ability to collaborate on OpenStack apps development<br>- Provide contributors and consumers with transparent workflow to manage their apps<br>- Provide consumers with information about apps - how it was developed and tested<br>- To summarize - introduce the way to build community working on OpenStack apps<br><br>*What is OpenStack application*<br>OpenStack is about 6 years young and all these years discussions about it are in progress. Variety of applications is huge, from LAMP stacks and legacy Java apps to telco workloads and VNF apps. There is working group which works on a definition of "What is OpenStack application", hopefully community will agree on definition soon.<br><br>For the sake of our discussion below let us agree on a simple approach: an OpenStack application is any software asset which 1. can be executed on an OpenStack cloud, 2. lives in <a href="http://apps.openstack.org" target="_blank">apps.openstack.org</a>. So far there are Murano applications, Heat templates, Glance images and TOSCA templates.<div><br></div><div>There are many good OpenStack applications in the world which don't live in OpenStack App Catalog. However, let us for now concentrate on those which do, just for the sake of this discussion. <br><br>*Introduction to OpenStack development ecosystem*<br>OpenStack was introduced about 6 years ago. Over these years community grown significantly. There were 8 companies contributing to OpenStack in Austin (1-st release). In Mitaka (13-th release) there were 64 companies contributing.<br><br>One of the reasons for this growth is the set of sophisticated tools the OpenStack contributor ecosystem has chosen to use or build to enable collaboration:<br>- software repositories: <a href="http://git.openstack.org/cgit/openstack/nova" target="_blank">http://git.openstack.org/cgit/openstack/nova</a>, <a href="http://git.openstack.org/cgit/openstack/neutron" target="_blank">http://git.openstack.org/cgit/openstack/neutron</a>, ..<br>- bug trackers: <a href="https://launchpad.net/nova" target="_blank">https://launchpad.net/nova</a>, <a href="https://launchpad.net/neutron" target="_blank">https://launchpad.net/neutron</a>, ...<br>- same instance of gerrit for all the projects for code review: <a href="https://review.openstack.org/" target="_blank">https://review.openstack.org/</a><br>- gating test infrastructure <a href="http://zuul.openstack.org/" target="_blank">http://zuul.openstack.org/</a><br>- common approach to release management, repositories management, naming, tons of other things managed by review in <a href="https://review.openstack.org/#/q/status:open+project:openstack/governance" target="_blank">https://review.openstack.org/#/q/status:open+project:openstack/governance</a><br>- IRC channels, etherpads, meetings and mailing lists<br>- governance to manage all of the above<br><br>All of the above is what we can call OpenStack collaboration ecosystem and it is a key factor for OpenStack community success.<br><br>*Introduction to OpenStack apps development ecosystem*<br>Now when OpenStack is mature and have it up and running is not a big deal, focus of community and customers shifts from "how do I get a running cloud" to "what do I do with running cloud".<br><br>Use cases of different cloud users are very different, however one can identify and develop standard building blocks which can be reused by cloud users (service providers, DevTest teams, ...). Many cloud users want to contribute their homegrown apps upstream because:<br>- it allows to other people to use it and improve it<br>- community can implement missing parts<br>- community can help with testing and maintaining an app<br><br>Year ago we introduced Community App Catalog for OpenStack <a href="http://apps.openstack.org" target="_blank">http://apps.openstack.org</a> as an integration/distribution point of customer experience/apps. This initiative is successful, there are about 100 software assets of various kinds which can be run on OpenStack. For further growth we need to make several changes in a way we approach collaboration around OpenStack Apps. Namely, we need to provide an ability to apps developers to collaborate on application development.<br><br>*OpenStack Community App Catalog is there, what else?*<br>Community App Catalog <a href="http://apps.openstack.org" target="_blank">http://apps.openstack.org</a> allows to publish/consume apps to/from it.<br><br>"The OpenStack Community App Catalog is designed to use the same tools for submission and review as other OpenStack projects. As such we follow the OpenStack development workflow" [0].<br><br>To follow OpenStack development workflow, apps developers need to have:<br>- dedicated repositories & code review system to collaborate on code<br>- mailing lists, IRC channels, core reviewers teams<br>- common approach to QA<br>- governance model to manage all of the above<br><br>Most of the above is missing for apps developers now. App Catalog provides central place to store final artifacts (ready apps, like .exe files in Win world) but there is no centralized infrastructure to collaborate on development of source code of apps.<br><br>Apps developers partially use infrastructure of OpenStack core projects (Heat & Murano) - repositories and bug trackers. Other than that they are on their own, there are no teams, no mailing lists, no IRC channels for apps developers - most of the items from the list above is missing.<br><br>[0] <a href="https://wiki.openstack.org/wiki/App-Catalog#How_to_contribute" target="_blank">https://wiki.openstack.org/wiki/App-Catalog#How_to_contribute</a><br><br>*All right, we need to change something. What exactly?*<br><br>1. We need to introduce a team which manages content of Community App Catalog, decides which new assets can be added, decides on a workflow for apps publishing, maintaining, consuming. This team could be a complimentary team working alongside the Community App Catalog implementation team (engine, backend, frontend); or within the team itself<br><br>2. There should be separated repositories for source code of apps from Community App Catalog. These repositories should not be stored under openstack label as they do not relate to core OpenStack projects. Core project teams are not responsible for maintaining apps.<br><br>3. Bug tracking for apps should be separated from bug tracking for core projects.<br><br>4. There should be teams working on apps with core reviewers, IRC channels and mailing lists. These teams should differ from core projects teams.<br><br>5. Given #1 - #4 it seems reasonable to develop governance model specific for OpenStack Apps Community, which differs (when it’s necessary) from governance model of OpenStack Community.<br><br>Let us develop such a governance model, implement changes described above and build community of OpenStack apps developers.<br><br>PS. There is representative discussion in comments to <a href="https://review.openstack.org/#/c/309383/" target="_blank">https://review.openstack.org/#/c/309383/</a>. Some team wants to add new repository for CI/CD Murano app. Should it be part of Murano core project? Rather not. Should it be part of BigTent? Well, rather not as BigTent is for core OpenStack services, not for workloads on top of it. At the same time team wants to use some OpenStack infra resources (at least gerrit for now) as this project is obviously beneficial for OpenStack. We need to have an ability to resolve similar requests in a centralized manner - there are more teams who want to publish<br>source code of their OpenStack apps and there is no established workflow for that.<br><br>*Agree. What’s next?*<br>Suggested plan:<br>- Introduce label openstack-apps, put repositories with source code of OpenStack Apps under it, i.e.:<br> * <a href="http://git.openstack.org/cgit/openstack-apps/murano-apps" target="_blank">http://git.openstack.org/cgit/openstack-apps/murano-apps</a><br> * <a href="http://git.openstack.org/cgit/openstack-apps/heat-templates" target="_blank">http://git.openstack.org/cgit/openstack-apps/heat-templates</a><br> * <a href="http://git.openstack.org/cgit/openstack-apps/tosca-workflows" target="_blank">http://git.openstack.org/cgit/openstack-apps/tosca-workflows</a><br>- Agree with OpenStack Community App Catalog team on how content of App Catalog is managed and by whom<br>- Describe workflow of how to add source code of new application to repositories, who approves its addition<br>- Introduce simplified workflow of publishing new Application to the catalog:<br> * register/login<br> * push/update<br> * done<br>- Introduce teams (core reviewers) contributing to/maintaining Murano apps, Heat templates, ...<br>- Establish channels of communications with potential contributors (mailing list, meetings, IRC/slack channel, ... )<br>- Agree with contributors on QA process for applications and how we track it in Community App Catalog<br><br>To simplify commenting and tracking of the plan above I put last two sections with suggested steps to the etherpad<br><a href="https://etherpad.openstack.org/p/OpenStackAppsCommunity" target="_blank">https://etherpad.openstack.org/p/OpenStackAppsCommunity</a><br><br><div>We're also discussing this topic with OpenStack Application Ecosystem Working Group and several members of OpenStack App Catalog team support this idea:</div><div><a href="http://lists.openstack.org/pipermail/user-committee/2016-May/000854.html" target="_blank">http://lists.openstack.org/pipermail/user-committee/2016-May/000854.html</a><br></div><div><br></div><div>Please share your thoughts and comments.<div><br></div></div><div><div><div dir="ltr">Regards,<div>Igor Marnat</div></div></div></div>
</div></div>
</blockquote></div><br></div></div>