Open Stack

Excerpts from Ghanshyam Mann's message of 2018-06-18 20:38:00 +0900:
>  ---- On Sat, 16 Jun 2018 04:01:45 +0900 Doug Hellmann <doug at doughellmann.com> wrote ---- 
>  > Excerpts from corvus's message of 2018-06-15 08:46:40 -0700:
>  > > Doug Hellmann <doug at doughellmann.com> writes:
>  > > 
>  > > > Excerpts from Ghanshyam's message of 2018-06-15 09:04:35 +0900:
>  > > 
>  > > >> Yes, It will not be set on LIBS_FROM_GIT as we did not set it
>  > > >> explicitly. But gate running on any repo does run job on current
>  > > >> change set of that repo which is nothing but "master + current patch
>  > > >> changes" . For example, any job running on oslo.config patch will
>  > > >> take oslo.config source code from that patch which is "master +
>  > > >> current change". You can see the results in this patch -
>  > > >> https://review.openstack.org/#/c/575324/ . Where I deleted a module
>  > > >> and gate jobs (including tempest-full-py3) fails as they run on
>  > > >> current change set of neutron-lib code not on pypi version(which
>  > > >> would pass the tests).
>  > > >
>  > > > The tempest-full-py3 job passed for that patch, though. Which seems to
>  > > > indicate that the neutron-lib repository was not used in the test job,
>  > > > even though it was checked out.
>  > > 
>  > > The automatic generation of LIBS_FROM_GIT only includes projects which
>  > > appear in required-projects.  So in this case neutron-lib does not
>  > > appear in LIBS_FROM_GIT[1], so the change is not actually tested by that
>  > > job.
> 
> Yes, this is now clear to me. I was in impressions of treating the lib same way as service for testing repo always from source but that's not the case. 
> 
>  > > 
>  > > Doug's approach of adding {{zuul.project}} to LIBS_FROM_GIT would work,
>  > > but anytime LIBS_FROM_GIT is set explicitly, it turns off the automatic
>  > > generation, so more complex jobs (which may want to inherit from that
>  > > job but need multiple libraries) would also have to override
>  > > LIBS_FROM_GIT and add the full set of projects.
>  > > 
>  > > The code that automatically sets LIBS_FROM_GIT is fairly simple and
>  > > could be modified to automatically add the project of the change under
>  > > test.  We could do that for all jobs, or we could add a flag which
>  > > toggles the behavior.  The question to answer here is: is there ever a
>  > > case where a devstack job should not install the change under test from
>  > > source?  I think the answer is no, and even though under Zuul v2
>  > > devstack-gate didn't automatically add the project under test to
>  > > LIBS_FROM_GIT, we probably had that behavior anyway due to some JJB
>  > > templating which did.
>  > 
>  > Adding the project-under-test to LIBS_FROM_GIT unconditionally feels
>  > like the behavior I would expect from the job.
> 
> ++
> 
>  > 
>  > > A further thing to consider is what the desired behavior is for a series
>  > > of changes.  If a change to neutron-lib depends on a change to
>  > > oslo.messaging, when the forward testing job runs on neutron-lib, should
>  > > it also add oslo.messaging to LIBS_FROM_GIT?  That's equally easy to
>  > > implement (but would certainly need a flag as it essentially would add
>  > > everything in the change series to LIBS_FROM_GIT which defeats the
>  > > purpose of the restriction for the jobs which need it), but I honestly
>  > > am not certain what's desired.
>  > 
>  > I think going ahead and adding everything in the dependency chain
>  > also makes sense. If I have 2 changes in libraries and a change in
>  > a service and I want to test them all together I would expect to
>  > be able to do that by using Depends-On and then for all 3 to be
>  > installed from source in the job that runs.
> 
> Yes, I agree on testing all series(either alone repo or depends-on on lib or service) with installed from source. 
> 
>  > 
>  > > 
>  > > For each type of project (service, lib, lib-group (eg oslo.messaging)),
>  > > what do we want to test from git vs pypi?
>  > 
>  > We want to test changes to service projects with libraries from
>  > PyPI so that we do not end up with services that rely on unreleased
>  > features of libraries.
>  > 
>  > We want to test changes to libraries with some services installed
>  > from git so that we know changes to the library do not break (current)
>  > master of the service. The set of interesting services may vary,
>  > but a default set that represents the tightly coupled services that
>  > run in the integrated gate now is reasonable.
>  > 
>  > > How many jobs are needed to accomplish that?
>  > 
>  > Ideally 1? Or 2? That's what I'm trying to work out.
> 
> Currently "tempest-full/-py3" job does not run the 'slow' marked scenario tests and they run in separate job "tempest-scenario-multinode-lvm-multibackend"(which i am working to make it more clean)
> 
> I think "tempest-full/-py3" will cover the most of the code/lib usage coverage.  

It does seem like enough coverage to start out, and matches what
we are doing under python 2.

I have proposed https://review.openstack.org/#/c/575925/ to set up a
project template using tempest-full-py3. Andreas points out there that
we could put the project template in the tempest repo where the job is
defined. I don't know which we would prefer, so let me know what you
think.

I also set up a test patch on oslo.config
https://review.openstack.org/#/c/575927/ that depends on the
project-template patch above and Jim's devstack patch
https://review.openstack.org/#/c/575801/. The lots from the
tempest-full-py3 job are at
http://logs.openstack.org/27/575927/2/check/tempest-full-py3/16b8922/

> 
>  > 
>  > > What should happen with a change series with other
>  > > projects in it?
>  > 
>  > I expect all of the patches in a series to be installed from source
>  > somewhere in the chain. That works today if we have a library patch
>  > that depends on a service patch because that patched version of the
>  > service is used in the dsvm job run against the library change. If
>  > we could make the reverse dependency work, too (where a patch to a
>  > service depends on a library change), that would be grand.
>  > 
>  > I think your patch https://review.openstack.org/#/c/575801/ at least
>  > lets us go in one direction (library->service) using a single job
>  > definition, but I can't tell if it would work the other way around.
>  > 
>  > > 
>  > > [1] http://logs.openstack.org/24/575324/3/check/tempest-full-py3/d183788/controller/logs/_.localrc_auto.txt
>  > > 
>  > > -Jim
>  > > 
>  > 
>  > __________________________________________________________________________
>  > OpenStack Development Mailing List (not for usage questions)
>  > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>  > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>  > 
>