<div dir="ltr">Hi, Alexander,<br><br>In general I am completely agree with Clint and Robert, and as one of contributors of Murano I don't see any practical reasons for repositories reorganization. And regarding of your proposal I have a few thoughts that I would like to share below: <div>
<br>><span style="font-family:arial,sans-serif">This enourmous amount of repositories adds too much infrustructural complexity<br></span>Creating a new repository is a quick, easy and completely automated procedure that requires only simple commit to Zuul configuration. All infrastructure related to repositories is handled by Openstack CI and supported by Openstack Infra Team, and actually don't require anything from project development team. About what infrastructure complexity you are talking about?<br>
<br>><span style="font-family:arial,sans-serif;font-size:13px">I actually think keeping them separate is a great </span><span style="font-family:arial,sans-serif;font-size:13px">way to make sure you have ongoing API stability. (c) Clint</span></div>
<div><font face="arial, sans-serif">I would like to share a little statistic gathered by Stan Lagun a little time ago regarding repositories count in different PaaS solution. If you are concerned about large number of repositories used by Murano, you will be quite amused:</font></div>
<div><font face="arial, sans-serif"><div><ul><li><a href="https://github.com/heroku">https://github.com/heroku</a> - 275<br></li><li><a href="https://github.com/cloudfoundry">https://github.com/cloudfoundry</a> - 132<br>
</li>
<li><a href="https://github.com/openshift">https://github.com/openshift</a> - 49<br></li><li><a href="https://github.com/CloudifySource">https://github.com/CloudifySource</a> - 46</li></ul></div></font></div><div>><span style="font-family:arial,sans-serif">First of all, I would suggest to have a single reposository for all the three main components of Murano: main murano API (the contents of the present), workflow execution engine (currently murano-conductor; also it was suggested to rename the component itself to murano-engine for more consistent naming) and metadata repository (currently murano-repository). <br>
<b><br></b></span></div><div><span style="font-family:arial,sans-serif"><b>murano-api</b> and <b>murano-repository</b> have many things in common, they are both present HTTP API to the user, and I hope would be rewritten to common framework (Pecan?). But <b>murano-conductor</b> have only one thing in common with other two components: code shared under <b>murano-common</b>. That repository may be eventually eliminated by moving to Oslo (as it should be done).</span></div>
<div><span style="font-family:arial,sans-serif"><br></span></div><div><span style="font-family:arial,sans-serif">></span><span style="font-family:arial,sans-serif">Also, it has been suggested to move our agents (both windows and unified python) into the main repository as well - just to put them into a separate subfolder. I don't see any reasons why they should be separated from core Murano: I don't believe we are going to have any third-party implementations of our "Unified agent" proposals, while this possibility was the main reason for separatinng them.<br>
<br>Main reason for murano-agent to have separate repository was not a possibility to have another implementation, but that all sources that should be able to be built as package, have tests and can be uploaded to PyPI (or any other gate job) should be placed in different repository. OpenStack CI have several rules regarding how repositories should be organized to support running different gate jobs. For example, to run tests <i>tox.ini</i> is </span><span style="font-family:arial,sans-serif">need</span><span style="font-family:arial,sans-serif"> </span><span style="font-family:arial,sans-serif">to be present in root directory, to build package <i>setup.py</i> should be present in root directory. So we could not simply move them to separate directories in main repository and have same capabilities as in separate repository.</span></div>
<div><span style="font-family:arial,sans-serif"><br></span></div><div><span style="font-family:arial,sans-serif">></span><span style="font-family:arial,sans-serif">Next, deployment scripts and auto-generated docs: are there reasons why they should be in their own repositories, instead of "docs" and "tools/deployment" folders of the primary repo? I would prefer the latter: docs and deployment scripts have no meaning without the sources which they document/deploy - so it is better to have them consistent.<br>
We have <i>developers documentation</i> alongside with all sources: <a href="https://github.com/stackforge/murano-conductor/tree/master/doc/source">murano-conductor</a>, <a href="https://github.com/stackforge/murano-api/tree/master/doc/source">murano-api</a> and so on. It is true that we have not so much documentation there, and not much code is documented to add auto-generated documentation. Documentation that is found in <b>murano-docs</b> repository actually is a docbook documentation, that is presented in book manner, and follows documentation patterns found in core projects itself: <a href="https://github.com/openstack/openstack-manuals/tree/master/doc">openstack-manuals</a>.</span></div>
<div><br></div><div><span style="font-family:arial,sans-serif"><b>murano-deployment</b> contains scripts and other artefacts related to deployment, but not necessary to source code. This repository don't use much of CI capabilities, but raise it is logical place where we can place different thing related to deployment: various scripts, specs, patches and so on. Also with separate repository we can not to spam our deployment engineers with software engineers related commits. </span></div>
<div><span style="font-family:arial,sans-serif"> </span></div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jan 21, 2014 at 11:55 PM, Alexander Tivelkov <span dir="ltr"><<a href="mailto:ativelkov@mirantis.com" target="_blank">ativelkov@mirantis.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-size:small"><div class="gmail_default">Hi folks,</div><div class="gmail_default">
<br></div><div class="gmail_default">As we are moving towards incubation application, I took a closer look at what is going on with our repositories.</div>
<div class="gmail_default">An here is what I found. We currently have 11 repositories at stackforge:</div><div class="gmail_default"><ul><li>murano-api<br></li><li>murano-conductor<br></li><li>murano-repository<br></li><li>
murano-dashboard<br></li><li>murano-common<br></li><li>python-muranoclient<br></li><li>murano-metadataclient<br></li><li>murano-agent<br></li><li>murano-docs<br></li><li>murano-tests<br></li><li>murano-deployment</li></ul>
</div><div class="gmail_default">This enourmous amount of repositories adds too much infrustructural complexity, and maintaining the changes in in consistent and reliable manner becomes a really tricky tasks. We often have changes which require modifing two or more repositories - and thus we have to make several changesets in gerrit, targeting different repositories. Quite often the dependencies between these changesets are not obvious, the patches get reviewed and approved on wrong order (yes, this also questions the quality of the code review, but that is a different topic), which causes in inconsostent state of the repositories. </div>
<div class="gmail_default"><br></div><div class="gmail_default">Well, anyway, this has to be changed in some way or another.</div><div class="gmail_default">I suggest to initiate the discussions on how to do all this.</div>
<div class="gmail_default"><br></div><div class="gmail_default">Below you may find my position. This is not final in any meaning, just a proposal. Please, feel free to discuss :)</div><div class="gmail_default"><br></div>
<div class="gmail_default">First of all, I would suggest to have a single reposository for all the three main components of Murano: main murano API (the contents of the present), workflow execution engine (currently murano-conductor; also it was suggested to rename the component itself to murano-engine for more consistent naming) and metadata repository (currently murano-repository). </div>
<div class="gmail_default">These should remain as independent modules, being able to run as different daemons, but stored within a single repository (similar to how heat has heat-api, heat-cfn and heat-engine under the same hood). The name of this repository is tentative: I think none of the existing match, so I would suggest to create a new repo (simple "murano" seems to fit the best), and then relocate all the content from other 3 repos and remove them aftwerwards. </div>
<div class="gmail_default"><br></div><div class="gmail_default">When the api, the repository and the engine are merged into a single repo, there will be no sense in having murano-common repo for storing their common classes: instead, there should be a "common" package inside the main murano repository.</div>
<div class="gmail_default"><br></div><div class="gmail_default">Also, it has been suggested to move our agents (both windows and unified python) into the main repository as well - just to put them into a separate subfolder. I don't see any reasons why they should be separated from core Murano: I don't believe we are going to have any third-party implementations of our "Unified agent" proposals, while this possibility was the main reason for separatinng them.</div>
<div class="gmail_default"><br></div><div class="gmail_default">Next, deployment scripts and auto-generated docs: are there reasons why they should be in their own repositories, instead of "docs" and "tools/deployment" folders of the primary repo? I would prefer the latter: docs and deployment scripts have no meaning without the sources which they document/deploy - so it is better to have them consistent.</div>
<div class="gmail_default"><br></div><div class="gmail_default"><br></div><div class="gmail_default">Then, the python bindings: "There can be only one" (c). Yes, the metadata API and the main murano API are different indeed, however there is no reason in having two repositories for their clients: let's have a single repo, containing two packages inside. Are there any technical reasons preventing us from doing that?</div>
<div class="gmail_default">CLI should be common as well - I think there should be a single command-line utility ("murano" should be the name), allowing to query both APIs. This CLI will eventually evolve into the developer's utility: it will get commands to package, sign and submit application packages.</div>
<div class="gmail_default"><br></div><div class="gmail_default">Openstack Dashboard plugin - aka Murano-dashboard - should remain in a separated repo, I have no objections here :)</div><div class="gmail_default"><br></div>
<div class="gmail_default">murano-tests may reamin independent as well - however, this repository is not likely to be transferred when we go to incubation: incubated projects should have tempest test in their repositories, shoudn't they? Our our test may remain on stackforge - this is irrelevant.</div>
<div class="gmail_default"><br></div><div class="gmail_default">And finally, we need some place to store sources of our metadata objects: the definition of core murano library, as well as example services, with all their stuff - metadata and ui definitions, heat templates, scripts etc. Here I propose to create a new repo, specially dedicated for this purpose. If we succeed in building the ecosystem for application developers and publishers, this will be the repo in which they should contribute, while the core murano repo's will remain relativele stable. </div>
<div class="gmail_default"><br></div><div class="gmail_default"><br></div><div class="gmail_default">So, this brings us to the following list of repos:</div><div class="gmail_default"><ul><li><b>murano</b> - main services, common, agents docs, deployments scripts<br>
</li><li><b>python-muranoclient</b> - python bindings and CLI<br></li><li><b>murano-dashboard</b> - OS Dashboard plugin<br></li><li><b>murano-apps</b> - new repo for metadata, including core library and example apps. <br>
</li><li><b>murano-tests</b> - existing test-repo, not going to be transferred when incubated.<br></li></ul></div><div class="gmail_default"><br></div><div class="gmail_default">This leaves us with just 4 repositories (plus one additional which will remain on stackforge), with clear separation of concerns.</div>
<div class="gmail_default"><br></div><div class="gmail_default">There may be technical issues in doing this mergement (we do not want to loose revision history, do we?), but they should be solvable (I'll write to infra asking on what is possible and what is not), but in general this is the direction in which we should be moving, as it seems to me.</div>
<div><br></div></div><div><div dir="ltr"><font>--<br></font><div dir="ltr"><font>Regards,<br>Alexander Tivelkov</font></div></div></div>
</div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><div>Serg Melikyan, Senior Software Engineer at Mirantis, Inc.<br></div><div><a href="http://mirantis.com/" target="_blank">http://mirantis.com</a> | <a href="mailto:smelikyan@mirantis.com" target="_blank">smelikyan@mirantis.com</a><br>
<div><br>+7 (495) 640-4904, 0261</div><div>+7 (903) 156-0836</div></div></div>
</div>