Open Stack

 ---- On Sat, 16 Jun 2018 04:01:45 +0900 Doug Hellmann <doug at doughellmann.com> wrote ---- 
 > Excerpts from corvus's message of 2018-06-15 08:46:40 -0700:
 > > Doug Hellmann <doug at doughellmann.com> writes:
 > > 
 > > > Excerpts from Ghanshyam's message of 2018-06-15 09:04:35 +0900:
 > > 
 > > >> Yes, It will not be set on LIBS_FROM_GIT as we did not set it
 > > >> explicitly. But gate running on any repo does run job on current
 > > >> change set of that repo which is nothing but "master + current patch
 > > >> changes" . For example, any job running on oslo.config patch will
 > > >> take oslo.config source code from that patch which is "master +
 > > >> current change". You can see the results in this patch -
 > > >> https://review.openstack.org/#/c/575324/ . Where I deleted a module
 > > >> and gate jobs (including tempest-full-py3) fails as they run on
 > > >> current change set of neutron-lib code not on pypi version(which
 > > >> would pass the tests).
 > > >
 > > > The tempest-full-py3 job passed for that patch, though. Which seems to
 > > > indicate that the neutron-lib repository was not used in the test job,
 > > > even though it was checked out.
 > > 
 > > The automatic generation of LIBS_FROM_GIT only includes projects which
 > > appear in required-projects.  So in this case neutron-lib does not
 > > appear in LIBS_FROM_GIT[1], so the change is not actually tested by that
 > > job.

Yes, this is now clear to me. I was in impressions of treating the lib same way as service for testing repo always from source but that's not the case. 

 > > 
 > > Doug's approach of adding {{zuul.project}} to LIBS_FROM_GIT would work,
 > > but anytime LIBS_FROM_GIT is set explicitly, it turns off the automatic
 > > generation, so more complex jobs (which may want to inherit from that
 > > job but need multiple libraries) would also have to override
 > > LIBS_FROM_GIT and add the full set of projects.
 > > 
 > > The code that automatically sets LIBS_FROM_GIT is fairly simple and
 > > could be modified to automatically add the project of the change under
 > > test.  We could do that for all jobs, or we could add a flag which
 > > toggles the behavior.  The question to answer here is: is there ever a
 > > case where a devstack job should not install the change under test from
 > > source?  I think the answer is no, and even though under Zuul v2
 > > devstack-gate didn't automatically add the project under test to
 > > LIBS_FROM_GIT, we probably had that behavior anyway due to some JJB
 > > templating which did.
 > 
 > Adding the project-under-test to LIBS_FROM_GIT unconditionally feels
 > like the behavior I would expect from the job.

++

 > 
 > > A further thing to consider is what the desired behavior is for a series
 > > of changes.  If a change to neutron-lib depends on a change to
 > > oslo.messaging, when the forward testing job runs on neutron-lib, should
 > > it also add oslo.messaging to LIBS_FROM_GIT?  That's equally easy to
 > > implement (but would certainly need a flag as it essentially would add
 > > everything in the change series to LIBS_FROM_GIT which defeats the
 > > purpose of the restriction for the jobs which need it), but I honestly
 > > am not certain what's desired.
 > 
 > I think going ahead and adding everything in the dependency chain
 > also makes sense. If I have 2 changes in libraries and a change in
 > a service and I want to test them all together I would expect to
 > be able to do that by using Depends-On and then for all 3 to be
 > installed from source in the job that runs.

Yes, I agree on testing all series(either alone repo or depends-on on lib or service) with installed from source. 

 > 
 > > 
 > > For each type of project (service, lib, lib-group (eg oslo.messaging)),
 > > what do we want to test from git vs pypi?
 > 
 > We want to test changes to service projects with libraries from
 > PyPI so that we do not end up with services that rely on unreleased
 > features of libraries.
 > 
 > We want to test changes to libraries with some services installed
 > from git so that we know changes to the library do not break (current)
 > master of the service. The set of interesting services may vary,
 > but a default set that represents the tightly coupled services that
 > run in the integrated gate now is reasonable.
 > 
 > > How many jobs are needed to accomplish that?
 > 
 > Ideally 1? Or 2? That's what I'm trying to work out.

Currently "tempest-full/-py3" job does not run the 'slow' marked scenario tests and they run in separate job "tempest-scenario-multinode-lvm-multibackend"(which i am working to make it more clean)

I think "tempest-full/-py3" will cover the most of the code/lib usage coverage.  

 > 
 > > What should happen with a change series with other
 > > projects in it?
 > 
 > I expect all of the patches in a series to be installed from source
 > somewhere in the chain. That works today if we have a library patch
 > that depends on a service patch because that patched version of the
 > service is used in the dsvm job run against the library change. If
 > we could make the reverse dependency work, too (where a patch to a
 > service depends on a library change), that would be grand.
 > 
 > I think your patch https://review.openstack.org/#/c/575801/ at least
 > lets us go in one direction (library->service) using a single job
 > definition, but I can't tell if it would work the other way around.
 > 
 > > 
 > > [1] http://logs.openstack.org/24/575324/3/check/tempest-full-py3/d183788/controller/logs/_.localrc_auto.txt
 > > 
 > > -Jim
 > > 
 > 
 > __________________________________________________________________________
 > OpenStack Development Mailing List (not for usage questions)
 > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
 > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 >