Open Stack

 ---- On Tue, 05 Feb 2019 23:35:20 +0900 Matt Riedemann <mriedemos at gmail.com> wrote ---- 
 > I'd like to propose some changes primarily to the CI jobs that run on  
 > nova changes, but also impact cinder and tempest. 
 >  
 > 1. Drop the nova-multiattach job and move test coverage to other jobs 
 >  
 > This is actually an old thread [1] and I had started the work but got  
 > hung up on a bug that was teased out of one of the tests when running in  
 > the multi-node tempest-slow job [2]. For now I've added a conditional  
 > skip on that test if running in a multi-node job. The open changes are  
 > here [3]. 

+1. The only question I commented on review - this test is skipped in all jobs now. 
If that all ok as of now then I am +2.

 >  
 > 2. Only run compute.api and scenario tests in nova-next job and run  
 > under python3 only 
 >  
 > The nova-next job is a place to test new or advanced nova features like  
 > placement and cells v2 when those were still optional in Newton. It  
 > currently runs with a few changes from the normal tempest-full job: 
 >  
 > * configures service user tokens 
 > * configures nova console proxy to use TLS 
 > * disables the resource provider association refresh interval 
 > * it runs the post_test_hook which runs some commands like  
 > archive_delete_rows, purge, and looks for leaked resource allocations [4] 
 >  
 > Like tempest-full, it runs the non-slow tempest API tests concurrently  
 > and then the scenario tests serially. I'm proposing that we: 
 >  
 > a) change that job to only run tempest compute API tests and scenario  
 > tests to cut down on the number of tests to run; since the job is really  
 > only about testing nova features, we don't need to spend time running  
 > glance/keystone/cinder/neutron tests which don't touch nova. 
 >  
 > b) run it with python3 [5] which is the direction all jobs are moving anyway

+1. It make sense to run only compute test in this job. 
 
 >  
 > 3. Drop the integrated-gate (py2) template jobs (from nova) 
 >  
 > Nova currently runs with both the integrated-gate and  
 > integrated-gate-py3 templates, which adds a set of tempest-full and  
 > grenade jobs each to the check and gate pipelines. I don't think we need  
 > to be gating on both py2 and py3 at this point when it comes to  
 > tempest/grenade changes. Tempest changes are still gating on both so we  
 > have coverage there against breaking changes, but I think anything  
 > that's py2 specific would be caught in unit and functional tests (which  
 > we're running on both py27 and py3*). 
 >  

IMO, we should keep running integrated-gate py2 templates on the project gate also
along with Tempest. Jobs in integrated-gate-* templates cover a large amount of code so
running that for both versions make sure we keep our code running on py2 also. Rest other
job like tempest-slow, nova-next etc are good to run only py3 on project side (Tempest gate
keep running py2 version also).

I am not sure if unit/functional jobs cover all code coverage and it is safe to ignore the py version
consideration from integration CI. As per TC resolution, python2 can be dropped during begning of
U cycle [1].

You have good point of having the integrated-gate py2 coverage on Tempest gate only is enough
but it has risk of merging the py2 breaking code on project side which will block the Tempest gate.
I agree that such chances are rare but still it can happen. 

Other point is that we need integrated-gate template running when Stein and Train become
stable branch (means on stable/stein and stable/train gate). Otherwise there are chance when
py2 broken code from U  (because we will test only py3 in U) is backported to stable/Train or
stable/stein. 

My opinion on this proposal is to wait till we officially drop py2 which is starting of U.

[1] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation-timeline.html

-gmann

 > Who's with me? 
 >  
 > [1]  
 > http://lists.openstack.org/pipermail/openstack-dev/2018-October/135299.html 
 > [2] https://bugs.launchpad.net/tempest/+bug/1807723 
 > [3]  
 > https://review.openstack.org/#/q/topic:drop-multiattach-job+(status:open+OR+status:merged) 
 > [4] https://github.com/openstack/nova/blob/5283b464b/gate/post_test_hook.sh 
 > [5] https://review.openstack.org/#/c/634739/ 
 >  
 > --  
 >  
 > Thanks, 
 >  
 > Matt 
 >  
 >