Open Stack

Excerpts from Jeremy Stanley's message of 2017-10-24 15:05:34 +0000:
> On 2017-10-24 09:42:25 -0400 (-0400), Doug Hellmann wrote:
> [...]
> > It looks like the publish-to-pypi-neutron template modifies
> > publish-to-pypi by adding openstack/neutron to the required-repositories
> > list for the release-openstack-python job. That repository was also at
> > some point added directly to the release-openstack-python job. So
> > technically the extension via the template is not needed. The same
> > applies to publish-to-pypi-horizon.
> [...]
> 
> Both are stop-gap solutions and I think either is fine in the short
> term to get us through the bulk of the release request backlog.
> Longer term we want to have neither of those. Python projects should
> be able to build sdists/wheels[1], documentation[2] and release
> notes[3] without tox (a behavior change which significantly
> complicates preinstalling source code from other projects, so best
> if their release jobs simply don't rely on doing that at all).
> 
> > As we find other projects with more dependencies, we're going to
> > end up with more custom templates.
> [...]
> 
> If we do, this only accelerates the need for something like a
> community goal for fixing this in those respective Python
> plugin/extension projects.

Right. I know the neutron team has been working on their library system
for quite a while. Maybe someone can report on the progress there?

I don't know if the horizon team is working on that (I'm just uninformed
about what they're doing). So maybe someone from that team can comment?

> 
> > One alternative to keeping multiple variants, and defining more in
> > the future, is to add required-repositories to the release-openstack-python
> > job directly, as we discover they are needed. Of course that means
> > we will clone repositories for some jobs that don't actually use
> > them. I don't know how big of an issue that really is, but the issue
> > of not knowing which instances of a job actually need a particular
> > dependency seems like more of a justification for keeping separate
> > templates than any runtime savings we would have by skipping a
> > couple of extra calls to git clone.
> [...]
> 
> Well, in this case you're talking about Zuul needing to calculate
> merge states for the neutron and horizon repos and then push these
> into every build of the affected jobs, which is no small amount of
> overhead on its own given the size of those particular repos. Also,
> once the tox-siblings role[4] is back in action (likely later this
> week?) Zuul will go back to preinstalling any required-projects from
> source into tox virtualenvs for these jobs, which is quite a bit
> more overhead still at that point.

Yes, we don't want that, so I think that means we want to retain the
multiple project templates for the short-term.

> 
> [1] http://git.openstack.org/cgit/openstack/governance/commit/?id=2678231
> [2] http://git.openstack.org/cgit/openstack/governance/commit/?id=2c0cdd2
> [3] http://git.openstack.org/cgit/openstack/governance/commit/?id=df438a7
> [4] http://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/tox-siblings