Open Stack

On 5/13/2018 9:06 PM, Ghanshyam Mann wrote:
>> +1 on idea. As of now slow marked tests are from nova, cinder and
>> neutron scenario tests and 2 API swift tests only [4]. I agree that
>> making a generic job in tempest is better for maintainability. We can
>> use existing job for that with below modification-
>> -  We can migrate
>> "legacy-tempest-dsvm-neutron-scenario-multinode-lvm-multibackend" job
>> zuulv3 in tempest repo
>> -  We can see if we can move migration tests out of it and use
>> "nova-live-migration" job (in tempest check pipeline ) which is much
>> better in live migration env setup and controlled by nova.
>> -  then it can be name something like
>> "tempest-scenario-multinode-lvm-multibackend".
>> -  run this job in nova, cinder, neutron check pipeline instead of experimental.
> Like this -https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:scenario-tests-job
> 
> That makes scenario job as generic with running all scenario tests
> including slow tests with concurrency 2. I made few cleanup and moved
> live migration tests out of it which is being run by
> 'nova-live-migration' job. Last patch making this job as voting on
> tempest side.
> 
> If looks good, we can use this to run on project side pipeline as voting.
> 
> -gmann
> 

I should have said something earlier, but I've said it on my original 
nova change now:

https://review.openstack.org/#/c/567697/

What was implemented in Tempest isn't really at all what I was going 
for, especially since it doesn't run the API tests marked 'slow'. All I 
want is a job like tempest-full (which excludes slow tests) to be 
tempest-full which *only* runs slow tests. They would run a mutually 
exclusive set of tests so we have that coverage. I don't care if the 
scenario tests are run in parallel or serial (it's probably best to 
start in serial like tempest-full today and then change to parallel 
later if that settles down).

But I think it's especially important given:

https://review.openstack.org/#/c/567697/2

That we have a job which only runs slow tests because we're going to be 
marking more tests as "slow" pretty soon and we don't need the overlap 
with the existing tests that are run in tempest-full.

--
Thanks,

Matt