Open Stack

 ---- On Mon, 18 Jun 2018 22:04:38 +0900 Doug Hellmann <doug at doughellmann.com> wrote ---- 
 > Excerpts from Ghanshyam Mann's message of 2018-06-18 20:38:00 +0900:
 > >  ---- On Sat, 16 Jun 2018 04:01:45 +0900 Doug Hellmann <doug at doughellmann.com> wrote ---- 
 > >  > Excerpts from corvus's message of 2018-06-15 08:46:40 -0700:
 > >  > > Doug Hellmann <doug at doughellmann.com> writes:
 > >  > > 
 > >  > > > Excerpts from Ghanshyam's message of 2018-06-15 09:04:35 +0900:
 > >  > > 
 > >  > > >> Yes, It will not be set on LIBS_FROM_GIT as we did not set it
 > >  > > >> explicitly. But gate running on any repo does run job on current
 > >  > > >> change set of that repo which is nothing but "master + current patch
 > >  > > >> changes" . For example, any job running on oslo.config patch will
 > >  > > >> take oslo.config source code from that patch which is "master +
 > >  > > >> current change". You can see the results in this patch -
 > >  > > >> https://review.openstack.org/#/c/575324/ . Where I deleted a module
 > >  > > >> and gate jobs (including tempest-full-py3) fails as they run on
 > >  > > >> current change set of neutron-lib code not on pypi version(which
 > >  > > >> would pass the tests).
 > >  > > >
 > >  > > > The tempest-full-py3 job passed for that patch, though. Which seems to
 > >  > > > indicate that the neutron-lib repository was not used in the test job,
 > >  > > > even though it was checked out.
 > >  > > 
 > >  > > The automatic generation of LIBS_FROM_GIT only includes projects which
 > >  > > appear in required-projects.  So in this case neutron-lib does not
 > >  > > appear in LIBS_FROM_GIT[1], so the change is not actually tested by that
 > >  > > job.
 > > 
 > > Yes, this is now clear to me. I was in impressions of treating the lib same way as service for testing repo always from source but that's not the case. 
 > > 
 > >  > > 
 > >  > > Doug's approach of adding {{zuul.project}} to LIBS_FROM_GIT would work,
 > >  > > but anytime LIBS_FROM_GIT is set explicitly, it turns off the automatic
 > >  > > generation, so more complex jobs (which may want to inherit from that
 > >  > > job but need multiple libraries) would also have to override
 > >  > > LIBS_FROM_GIT and add the full set of projects.
 > >  > > 
 > >  > > The code that automatically sets LIBS_FROM_GIT is fairly simple and
 > >  > > could be modified to automatically add the project of the change under
 > >  > > test.  We could do that for all jobs, or we could add a flag which
 > >  > > toggles the behavior.  The question to answer here is: is there ever a
 > >  > > case where a devstack job should not install the change under test from
 > >  > > source?  I think the answer is no, and even though under Zuul v2
 > >  > > devstack-gate didn't automatically add the project under test to
 > >  > > LIBS_FROM_GIT, we probably had that behavior anyway due to some JJB
 > >  > > templating which did.
 > >  > 
 > >  > Adding the project-under-test to LIBS_FROM_GIT unconditionally feels
 > >  > like the behavior I would expect from the job.
 > > 
 > > ++
 > > 
 > >  > 
 > >  > > A further thing to consider is what the desired behavior is for a series
 > >  > > of changes.  If a change to neutron-lib depends on a change to
 > >  > > oslo.messaging, when the forward testing job runs on neutron-lib, should
 > >  > > it also add oslo.messaging to LIBS_FROM_GIT?  That's equally easy to
 > >  > > implement (but would certainly need a flag as it essentially would add
 > >  > > everything in the change series to LIBS_FROM_GIT which defeats the
 > >  > > purpose of the restriction for the jobs which need it), but I honestly
 > >  > > am not certain what's desired.
 > >  > 
 > >  > I think going ahead and adding everything in the dependency chain
 > >  > also makes sense. If I have 2 changes in libraries and a change in
 > >  > a service and I want to test them all together I would expect to
 > >  > be able to do that by using Depends-On and then for all 3 to be
 > >  > installed from source in the job that runs.
 > > 
 > > Yes, I agree on testing all series(either alone repo or depends-on on lib or service) with installed from source. 
 > > 
 > >  > 
 > >  > > 
 > >  > > For each type of project (service, lib, lib-group (eg oslo.messaging)),
 > >  > > what do we want to test from git vs pypi?
 > >  > 
 > >  > We want to test changes to service projects with libraries from
 > >  > PyPI so that we do not end up with services that rely on unreleased
 > >  > features of libraries.
 > >  > 
 > >  > We want to test changes to libraries with some services installed
 > >  > from git so that we know changes to the library do not break (current)
 > >  > master of the service. The set of interesting services may vary,
 > >  > but a default set that represents the tightly coupled services that
 > >  > run in the integrated gate now is reasonable.
 > >  > 
 > >  > > How many jobs are needed to accomplish that?
 > >  > 
 > >  > Ideally 1? Or 2? That's what I'm trying to work out.
 > > 
 > > Currently "tempest-full/-py3" job does not run the 'slow' marked scenario tests and they run in separate job "tempest-scenario-multinode-lvm-multibackend"(which i am working to make it more clean)
 > > 
 > > I think "tempest-full/-py3" will cover the most of the code/lib usage coverage.  
 > 
 > It does seem like enough coverage to start out, and matches what
 > we are doing under python 2.
 > 
 > I have proposed https://review.openstack.org/#/c/575925/ to set up a
 > project template using tempest-full-py3. Andreas points out there that
 > we could put the project template in the tempest repo where the job is
 > defined. I don't know which we would prefer, so let me know what you
 > think.
 > 
 > I also set up a test patch on oslo.config
 > https://review.openstack.org/#/c/575927/ that depends on the
 > project-template patch above and Jim's devstack patch
 > https://review.openstack.org/#/c/575801/. The lots from the
 > tempest-full-py3 job are at
 > http://logs.openstack.org/27/575927/2/check/tempest-full-py3/16b8922/

Thanks. lgtm.  I feel we can keep the template in 'openstack-zuul-jobs' like we have 'integrated-gate' template[1].

[1] https://github.com/openstack-infra/openstack-zuul-jobs/blob/22cf73ae5a3090a91eb3c81cc4c427b546b0254e/zuul.d/zuul-legacy-project-templates.yaml#L57 

-gmann 

 > 
 > > 
 > >  > 
 > >  > > What should happen with a change series with other
 > >  > > projects in it?
 > >  > 
 > >  > I expect all of the patches in a series to be installed from source
 > >  > somewhere in the chain. That works today if we have a library patch
 > >  > that depends on a service patch because that patched version of the
 > >  > service is used in the dsvm job run against the library change. If
 > >  > we could make the reverse dependency work, too (where a patch to a
 > >  > service depends on a library change), that would be grand.
 > >  > 
 > >  > I think your patch https://review.openstack.org/#/c/575801/ at least
 > >  > lets us go in one direction (library->service) using a single job
 > >  > definition, but I can't tell if it would work the other way around.
 > >  > 
 > >  > > 
 > >  > > [1] http://logs.openstack.org/24/575324/3/check/tempest-full-py3/d183788/controller/logs/_.localrc_auto.txt
 > >  > > 
 > >  > > -Jim
 > >  > > 
 > >  > 
 > >  > __________________________________________________________________________
 > >  > OpenStack Development Mailing List (not for usage questions)
 > >  > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
 > >  > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 > >  > 
 > > 
 > 
 > __________________________________________________________________________
 > OpenStack Development Mailing List (not for usage questions)
 > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
 > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 >