---- On Tue, 05 Feb 2019 23:35:20 +0900 Matt Riedemann <mriedemos@gmail.com> wrote ----
I'd like to propose some changes primarily to the CI jobs that run on nova changes, but also impact cinder and tempest.
1. Drop the nova-multiattach job and move test coverage to other jobs
This is actually an old thread [1] and I had started the work but got hung up on a bug that was teased out of one of the tests when running in the multi-node tempest-slow job [2]. For now I've added a conditional skip on that test if running in a multi-node job. The open changes are here [3].
+1. The only question I commented on review - this test is skipped in all jobs now. If that all ok as of now then I am +2.
2. Only run compute.api and scenario tests in nova-next job and run under python3 only
The nova-next job is a place to test new or advanced nova features like placement and cells v2 when those were still optional in Newton. It currently runs with a few changes from the normal tempest-full job:
* configures service user tokens * configures nova console proxy to use TLS * disables the resource provider association refresh interval * it runs the post_test_hook which runs some commands like archive_delete_rows, purge, and looks for leaked resource allocations [4]
Like tempest-full, it runs the non-slow tempest API tests concurrently and then the scenario tests serially. I'm proposing that we:
a) change that job to only run tempest compute API tests and scenario tests to cut down on the number of tests to run; since the job is really only about testing nova features, we don't need to spend time running glance/keystone/cinder/neutron tests which don't touch nova.
b) run it with python3 [5] which is the direction all jobs are moving anyway
+1. It make sense to run only compute test in this job.
3. Drop the integrated-gate (py2) template jobs (from nova)
Nova currently runs with both the integrated-gate and integrated-gate-py3 templates, which adds a set of tempest-full and grenade jobs each to the check and gate pipelines. I don't think we need to be gating on both py2 and py3 at this point when it comes to tempest/grenade changes. Tempest changes are still gating on both so we have coverage there against breaking changes, but I think anything that's py2 specific would be caught in unit and functional tests (which we're running on both py27 and py3*).
IMO, we should keep running integrated-gate py2 templates on the project gate also along with Tempest. Jobs in integrated-gate-* templates cover a large amount of code so running that for both versions make sure we keep our code running on py2 also. Rest other job like tempest-slow, nova-next etc are good to run only py3 on project side (Tempest gate keep running py2 version also). I am not sure if unit/functional jobs cover all code coverage and it is safe to ignore the py version consideration from integration CI. As per TC resolution, python2 can be dropped during begning of U cycle [1]. You have good point of having the integrated-gate py2 coverage on Tempest gate only is enough but it has risk of merging the py2 breaking code on project side which will block the Tempest gate. I agree that such chances are rare but still it can happen. Other point is that we need integrated-gate template running when Stein and Train become stable branch (means on stable/stein and stable/train gate). Otherwise there are chance when py2 broken code from U (because we will test only py3 in U) is backported to stable/Train or stable/stein. My opinion on this proposal is to wait till we officially drop py2 which is starting of U. [1] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation... -gmann
Who's with me?
[1] http://lists.openstack.org/pipermail/openstack-dev/2018-October/135299.html [2] https://bugs.launchpad.net/tempest/+bug/1807723 [3] https://review.openstack.org/#/q/topic:drop-multiattach-job+(status:open+OR+...) [4] https://github.com/openstack/nova/blob/5283b464b/gate/post_test_hook.sh [5] https://review.openstack.org/#/c/634739/
--
Thanks,
Matt