Open Stack

I think this is a really great approach.  +1

Nate

On Sun, May 05, 2019 at 02:18:08AM -0500, Ghanshyam Mann wrote:
> Current integrated-gate jobs (tempest-full) is not so stable for various bugs specially timeout. We tried
> to improve it via filtering the slow tests in the separate tempest-slow job but the situation has not been improved much.
> 
> We talked about the Ideas to make it more stable and fast for projects especially when failure is not
> related to each project. We are planning to split the integrated-gate template (only tempest-full job as
> first step) per related services. 
> 
> Idea:
> - Run only dependent service tests on project gate.
> - Tempest gate will keep running all the services tests as the integrated gate at a centeralized  place without any change in the current job.
> - Each project can run the below mentioned template. 
> - All below template will be defined and maintained by QA team. 
> 
> I would like to know each 6 services which run integrated-gate jobs
> 
> 1."Integrated-gate-networking" (job to run on neutron gate)
>  Tests to run in this template: neutron APIs , nova APIs,  keystone APIs ? All scenario currently running in tempest-full in the same way ( means non-slow and in serial)
> Improvement for neutron gate: exlcude the cinder API tests,  glance API tests, swift API tests,
> 
> 2."Integrated-gate-storage" (job to run on cinder gate, glance gate)
> Tests to run in this template: Cinder APIs , Glance APIs, Swift APIs, Nova APIs and All scenario currently running in tempest-full in the same way ( means non-slow and in serial)
> Improvement for cinder, glance gate: excluded the neutron APIs tests, Keystone APIs tests
> 
> 3. "Integrated-gate-object-storage" (job to run on swift gate)
> Tests to run in this template: Cinder APIs , Glance APIs, Swift APIs and All scenario currently running in tempest-full in the same way ( means non-slow and in serial)
> Improvement for swift gate: excluded the neutron APIs tests, - Keystone APIs tests, - Nova APIs tests.
> Note: swift does not run integrated-gate as of now.
> 
> 4. "Integrated-gate-compute" (job to run on Nova gate)
> tests to run is : Nova APIs, Cinder APIs , Glance APIs ?, neutron APIs and All scenario currently running in tempest-full in same way ( means non-slow and in serial)
> Improvement for Nova gate: excluded the swift APIs tests(not running in current job but in future, it might), Keystone API tests. 
> 
> 5. "Integrated-gate-identity" (job to run on keystone gate)
> Tests to run is : all as all project use keystone, we might need to run all tests as it is running in integrated-gate.
> But does keystone is being unsed differently by all services? if no then, is it enough to run only single service tests say Nova or neutron ?
> 
> 6. "Integrated-gate-placement" (job to run on placement gate)
> Tests to run in this template: Nova APIs tests, Neutron APIs tests + scenario tests + any new service depends on placement APIs
>  Improvement for placement gate: excluded the  glance APIs tests, cinder APIs tests, swift APIs tests, keystone APIs tests
> 
> Thoughts on this approach?
> 
> The important point is we must not lose the coverage of integrated testing per project. So I would like to
> get each project view if we are missing any dependency (proposed tests removal) in above proposed templates.
> 
> - https://etherpad.openstack.org/p/qa-train-ptg 
> 
> -gmann
> 
>