Open Stack

Excerpts from corvus's message of 2018-06-15 08:46:40 -0700:
> Doug Hellmann <doug at doughellmann.com> writes:
> 
> > Excerpts from Ghanshyam's message of 2018-06-15 09:04:35 +0900:
> 
> >> Yes, It will not be set on LIBS_FROM_GIT as we did not set it
> >> explicitly. But gate running on any repo does run job on current
> >> change set of that repo which is nothing but "master + current patch
> >> changes" . For example, any job running on oslo.config patch will
> >> take oslo.config source code from that patch which is "master +
> >> current change". You can see the results in this patch -
> >> https://review.openstack.org/#/c/575324/ . Where I deleted a module
> >> and gate jobs (including tempest-full-py3) fails as they run on
> >> current change set of neutron-lib code not on pypi version(which
> >> would pass the tests).
> >
> > The tempest-full-py3 job passed for that patch, though. Which seems to
> > indicate that the neutron-lib repository was not used in the test job,
> > even though it was checked out.
> 
> The automatic generation of LIBS_FROM_GIT only includes projects which
> appear in required-projects.  So in this case neutron-lib does not
> appear in LIBS_FROM_GIT[1], so the change is not actually tested by that
> job.
> 
> Doug's approach of adding {{zuul.project}} to LIBS_FROM_GIT would work,
> but anytime LIBS_FROM_GIT is set explicitly, it turns off the automatic
> generation, so more complex jobs (which may want to inherit from that
> job but need multiple libraries) would also have to override
> LIBS_FROM_GIT and add the full set of projects.
> 
> The code that automatically sets LIBS_FROM_GIT is fairly simple and
> could be modified to automatically add the project of the change under
> test.  We could do that for all jobs, or we could add a flag which
> toggles the behavior.  The question to answer here is: is there ever a
> case where a devstack job should not install the change under test from
> source?  I think the answer is no, and even though under Zuul v2
> devstack-gate didn't automatically add the project under test to
> LIBS_FROM_GIT, we probably had that behavior anyway due to some JJB
> templating which did.

Adding the project-under-test to LIBS_FROM_GIT unconditionally feels
like the behavior I would expect from the job.

> A further thing to consider is what the desired behavior is for a series
> of changes.  If a change to neutron-lib depends on a change to
> oslo.messaging, when the forward testing job runs on neutron-lib, should
> it also add oslo.messaging to LIBS_FROM_GIT?  That's equally easy to
> implement (but would certainly need a flag as it essentially would add
> everything in the change series to LIBS_FROM_GIT which defeats the
> purpose of the restriction for the jobs which need it), but I honestly
> am not certain what's desired.

I think going ahead and adding everything in the dependency chain
also makes sense. If I have 2 changes in libraries and a change in
a service and I want to test them all together I would expect to
be able to do that by using Depends-On and then for all 3 to be
installed from source in the job that runs.

> 
> For each type of project (service, lib, lib-group (eg oslo.messaging)),
> what do we want to test from git vs pypi?

We want to test changes to service projects with libraries from
PyPI so that we do not end up with services that rely on unreleased
features of libraries.

We want to test changes to libraries with some services installed
from git so that we know changes to the library do not break (current)
master of the service. The set of interesting services may vary,
but a default set that represents the tightly coupled services that
run in the integrated gate now is reasonable.

> How many jobs are needed to accomplish that?

Ideally 1? Or 2? That's what I'm trying to work out.

> What should happen with a change series with other
> projects in it?

I expect all of the patches in a series to be installed from source
somewhere in the chain. That works today if we have a library patch
that depends on a service patch because that patched version of the
service is used in the dsvm job run against the library change. If
we could make the reverse dependency work, too (where a patch to a
service depends on a library change), that would be grand.

I think your patch https://review.openstack.org/#/c/575801/ at least
lets us go in one direction (library->service) using a single job
definition, but I can't tell if it would work the other way around.

> 
> [1] http://logs.openstack.org/24/575324/3/check/tempest-full-py3/d183788/controller/logs/_.localrc_auto.txt
> 
> -Jim
>