On 3/26/2019 9:53 AM, Ghanshyam Mann wrote:
If we have the same coverage in functional test then question comes why we have such heavy tests as integration tests ? I think these were added for nova specific testing only.
My other concern is that after removing the this job, nova gate would not have integration testing running on multinode env even tests are multinode specific or single node one and any future non-slow multinode tests will not be running. I see the benefit of running single nodes operation on multinode env for nova multinode arch.
If the job is not stable then, we can work on making it stable but that's not the case seems. That job is voting on tempest side for long time.
My primary issue is the number of jobs and resources it takes to run through a nova change in the gate. I agree the job itself is probably stable, I'm not sure why it was non-voting but it was brought in that way with the aforementioned mechanical change. You've probably noticed a trend where I'm trying to reduce the amount of redundant overlapping tests in the jobs we run so we can trim down how many resources it takes in the gate to test a nova change and also cut back on how many different jobs with racy tests can prevent a nova change from landing. So if there are just 3 or 4 unique tests in that job which are now not covered by what nova does run, I would rather find a way to just run those tests rather than those tests plus the 98% other same tests we get in the tempest-full job. Maybe that's making a change such that single-node jobs no longer run those tests and the multinode job only runs tests that require multiple nodes, but that's quite a bit more work since it's going to affect all projects using the integrated-gate template. -- Thanks, Matt