Open Stack

Doug Hellmann <doug at doughellmann.com> writes:

> Excerpts from Ghanshyam's message of 2018-06-15 09:04:35 +0900:

>> Yes, It will not be set on LIBS_FROM_GIT as we did not set it
>> explicitly. But gate running on any repo does run job on current
>> change set of that repo which is nothing but "master + current patch
>> changes" . For example, any job running on oslo.config patch will
>> take oslo.config source code from that patch which is "master +
>> current change". You can see the results in this patch -
>> https://review.openstack.org/#/c/575324/ . Where I deleted a module
>> and gate jobs (including tempest-full-py3) fails as they run on
>> current change set of neutron-lib code not on pypi version(which
>> would pass the tests).
>
> The tempest-full-py3 job passed for that patch, though. Which seems to
> indicate that the neutron-lib repository was not used in the test job,
> even though it was checked out.

The automatic generation of LIBS_FROM_GIT only includes projects which
appear in required-projects.  So in this case neutron-lib does not
appear in LIBS_FROM_GIT[1], so the change is not actually tested by that
job.

Doug's approach of adding {{zuul.project}} to LIBS_FROM_GIT would work,
but anytime LIBS_FROM_GIT is set explicitly, it turns off the automatic
generation, so more complex jobs (which may want to inherit from that
job but need multiple libraries) would also have to override
LIBS_FROM_GIT and add the full set of projects.

The code that automatically sets LIBS_FROM_GIT is fairly simple and
could be modified to automatically add the project of the change under
test.  We could do that for all jobs, or we could add a flag which
toggles the behavior.  The question to answer here is: is there ever a
case where a devstack job should not install the change under test from
source?  I think the answer is no, and even though under Zuul v2
devstack-gate didn't automatically add the project under test to
LIBS_FROM_GIT, we probably had that behavior anyway due to some JJB
templating which did.

A further thing to consider is what the desired behavior is for a series
of changes.  If a change to neutron-lib depends on a change to
oslo.messaging, when the forward testing job runs on neutron-lib, should
it also add oslo.messaging to LIBS_FROM_GIT?  That's equally easy to
implement (but would certainly need a flag as it essentially would add
everything in the change series to LIBS_FROM_GIT which defeats the
purpose of the restriction for the jobs which need it), but I honestly
am not certain what's desired.

For each type of project (service, lib, lib-group (eg oslo.messaging)),
what do we want to test from git vs pypi?  How many jobs are needed to
accomplish that?  What should happen with a change series with other
projects in it?

[1] http://logs.openstack.org/24/575324/3/check/tempest-full-py3/d183788/controller/logs/_.localrc_auto.txt

-Jim