Open Stack

On 4/17/2017 1:55 PM, Ihar Hrachyshka wrote:
> But since it's not executed anywhere in tempest gate, even as
> non-voting (?), it's effectively dead code that may be long broken
> without anyone knowing. Of course there are consumers of the tests
> downstream, but for those consumers it's a tough call to start
> depending on the tests if they are not sanity checked by tempest
> itself. Wouldn't it make sense to have some job in tempest gate that
> would execute those tests (maybe just them to speed up such a job?
> maybe non-voting? maybe even as periodic? but there should be
> something that keeps it green in long run).

I enabled the job that runs the slow scenario tests in the nova 
experimental queue:

https://review.openstack.org/#/c/458676/

Because we were changing some of the encrypted volume code in nova and 
that's only exercised by a scenario test that was marked slow, but as 
you said I need to run tests to have coverage on those changes. That 
works for me in this case, the problem is I had to (1) know those tests 
existed and (2) go out of my way looking for their results in a job run 
and find out they weren't there, then hunt them out. That's exceptional 
for most patches so yes, things are going to slip through and break most 
likely because we're not gating on them. I understand why the QA team 
did what they did though so I'm not pushing back on that.

-- 

Thanks,

Matt