Open Stack

The check-tempest-dsvm-cells job has been in nova's check queue since 
January as non-voting and has been stable for a couple of weeks now, so 
before it's regressed melwitt proposed a change to making it voting and 
gating on nova changes [1].

I raised a concern in that change that the tempest-dsvm-cells job is not 
in the check queue for tempest or devstack changes, so if a change is 
merged in tempest/devstack which breaks the cells job, it will block 
nova changes from merging.

mtreinish noted that tempest already has around 30 jobs running against 
it right now in the check queue, so he'd prefer that another one isn't 
added since the nova API shouldn't be different in the case of cells, 
but we know there are quirks.  That can be seen from the massive regex 
of excluded tests for the tempest-dvsm-cells job [2].

If we could turn that regex list into tempest configurations, I think 
that would make it possible to not have to run tempest changes through 
the cells job in the check queue but also feel reasonably confident that 
changes to tempest that use the config options properly won't break the 
cells job (and block nova in the gate).

This is actually something we should do regardless of voting or not on 
nova since new tests get added which might not fall in the regex and 
break the cells job.  So I'm going to propose some changes so that the 
regex will be moved to devstack-gate (regex exodus (tm)) and we'll work 
on whittling down the regex there (and run those d-g changes against the 
tempest-dsvm-cells job in the experimental queue).

The question for the nova team is, shall we make the tempest-dsvm-cells 
job voting on nova changes knowing that the gate can be broken with a 
change to tempest that isn't caught in the regex?  In my opinion I think 
we should make it voting so we don't regress cells with changes to nova 
that go unnoticed with the non-voting job today.  Cells v2 is a nova 
priority for Liberty so we don't want setbacks now that it's stable.

If a change does land in tempest which breaks the cells job and blocks 
nova, we (1) fix it or (2) modify the regex so it's excluded until fixed 
as has been done up to this point.

I think we should probably mull this over in the ML and then vote on it 
in this week's nova meeting.

[1] https://review.openstack.org/#/c/190894/
[2] 
http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n1004

-- 

Thanks,

Matt Riedemann