Open Stack

On 06/22/15 at 04:23pm, Matt Riedemann wrote:
>The check-tempest-dsvm-cells job has been in nova's check queue since 
>January as non-voting and has been stable for a couple of weeks now, 
>so before it's regressed melwitt proposed a change to making it 
>voting and gating on nova changes [1].
>
>I raised a concern in that change that the tempest-dsvm-cells job is 
>not in the check queue for tempest or devstack changes, so if a 
>change is merged in tempest/devstack which breaks the cells job, it 
>will block nova changes from merging.
>
>mtreinish noted that tempest already has around 30 jobs running 
>against it right now in the check queue, so he'd prefer that another 
>one isn't added since the nova API shouldn't be different in the case 
>of cells, but we know there are quirks.  That can be seen from the 
>massive regex of excluded tests for the tempest-dvsm-cells job [2].
>
>If we could turn that regex list into tempest configurations, I think 
>that would make it possible to not have to run tempest changes 
>through the cells job in the check queue but also feel reasonably 
>confident that changes to tempest that use the config options 
>properly won't break the cells job (and block nova in the gate).
>
>This is actually something we should do regardless of voting or not 
>on nova since new tests get added which might not fall in the regex 
>and break the cells job.  So I'm going to propose some changes so 
>that the regex will be moved to devstack-gate (regex exodus (tm)) and 
>we'll work on whittling down the regex there (and run those d-g 
>changes against the tempest-dsvm-cells job in the experimental 
>queue).
>
>The question for the nova team is, shall we make the 
>tempest-dsvm-cells job voting on nova changes knowing that the gate 
>can be broken with a change to tempest that isn't caught in the 
>regex?  In my opinion I think we should make it voting so we don't 
>regress cells with changes to nova that go unnoticed with the 
>non-voting job today.  Cells v2 is a nova priority for Liberty so we 
>don't want setbacks now that it's stable.

I agree, though I'm pretty biased.

>
>If a change does land in tempest which breaks the cells job and 
>blocks nova, we (1) fix it or (2) modify the regex so it's excluded 
>until fixed as has been done up to this point.

During our efforts last cycle to reduce the number of failing tests it 
seemed that we had regressions based on Tempest changes 2 times.  Once 
around security group logic being added to tests and again when the 
networking setup changed during tests that created an instance.

The security group change led to exclusions and the networking change we 
were able to fix with a devstack change, and a Tempest flag iirc.


>
>I think we should probably mull this over in the ML and then vote on 
>it in this week's nova meeting.
>
>[1] https://review.openstack.org/#/c/190894/
>[2] http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n1004
>
>-- 
>
>Thanks,
>
>Matt Riedemann
>
>
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev