Open Stack

---- On Thu, 21 Mar 2019 22:16:59 -0500 Matt Riedemann <mriedemos at gmail.com> wrote ----
 > Nova runs the tempest-multinode-full job as non-voting for some reason: 
 >  
 > https://github.com/openstack/nova/blob/c993d4fe2ffaadea76d5ae76eedfddc3a4c71346/.zuul.yaml#L272 
 >  
 > I'm assuming we were running that job initially for live migration  
 > testing, but we get that with the voting nova-live-migration and  
 > nova-grenade-live-migration jobs. 
 >  
 > The tempest-multinode-full job is essentially just the tempest-full job  
 > with 2 nodes. It doesn't run slow tests. But we have the tempest-slow  
 > job which is multinode and runs slow tests only. 
 >  
 > So it seems we have a lot of overlap between the nova-*live-migration  
 > multinode jobs for live migration testing (and evacuate gets tested in  
 > the nova-live-migration job as well), the tempest-full single node job,  
 > the nova-next single-node job, the tempest-slow job and the non-voting  
 > tempest-multinode-slow job, so can we just drop that last one? 
 >  
 > If someone were really ambitious they could do a diff of the tests that  
 > actually get run between say  
 > tempest-full/nova-live-migration/tempest-slow and tempest-multinode-full  
 > to see if we'd be missing any test coverage. 

Not all multinode tests tun on tempest-slow and nova-live-migration jobs. 
I did diff of those jobs tests run and below are the coverage nova is going to lose with 
removing the tempest-multinode-full.

- test_server_multinode : https://github.com/openstack/tempest/blob/a955095f8369e2e15c97a536edc1a9c077208129/tempest/scenario/test_server_multinode.py

- test_cold_migration, test_revert_cold_migration: https://github.com/openstack/tempest/blob/16e4005dfcbd6b1aa68a1a1c0187b0af9cc70650/tempest/api/compute/admin/test_migrations.py

- test_servers_on_multinodes: https://github.com/openstack/tempest/blob/7a588ded216f74ddd0015c3065d4fae10de2161f/tempest/api/compute/admin/test_servers_on_multinodes.py

- Main concern of removing this job is each time we are going to make any mulitnode tests from slow->non-slow nova going to lose the coverage. 

I will suggest not to remove it. I have recently made this job running for stable branch also so that we do not regress there too. If we are going to cover most or full coverage of integrated tests, I will say we need to run these 3 tempest jobs 1. tempest-full-py3, 2. tempest-slow-py3 3. tempest-multinode-full 

When I made tempest-slow job I merged tempest-full-multinode also in that which means run  all multinode test there but later tempest-slow job modified to run only slow tests.

What we can improve is to make tempest-multinode-full to run only multinode tests (need to sort out the mechanish for that) but we need to check if there is no benefit of running single node test on multinode setup. 

-gmann

 >  
 > --  
 >  
 > Thanks, 
 >  
 > Matt 
 >  
 >