Open Stack

On 22 June 2015 at 23:03, Matt Riedemann <mriedem at linux.vnet.ibm.com> wrote:
>
>
> On 6/22/2015 4:38 PM, Matt Riedemann wrote:
>>
>>
>>
>> On 6/22/2015 4:32 PM, Russell Bryant wrote:
>>>
>>> On 06/22/2015 05:23 PM, Matt Riedemann wrote:
>>>>
>>>> The check-tempest-dsvm-cells job has been in nova's check queue since
>>>> January as non-voting and has been stable for a couple of weeks now, so
>>>> before it's regressed melwitt proposed a change to making it voting and
>>>> gating on nova changes [1].
>>>>
>>>> I raised a concern in that change that the tempest-dsvm-cells job is not
>>>> in the check queue for tempest or devstack changes, so if a change is
>>>> merged in tempest/devstack which breaks the cells job, it will block
>>>> nova changes from merging.
>>>>
>>>> mtreinish noted that tempest already has around 30 jobs running against
>>>> it right now in the check queue, so he'd prefer that another one isn't
>>>> added since the nova API shouldn't be different in the case of cells,
>>>> but we know there are quirks.  That can be seen from the massive regex
>>>> of excluded tests for the tempest-dvsm-cells job [2].

That sounds like a good call.

For the record, cells v2 should fix this major issue, which is awesome.

>>>> If we could turn that regex list into tempest configurations, I think
>>>> that would make it possible to not have to run tempest changes through
>>>> the cells job in the check queue but also feel reasonably confident that
>>>> changes to tempest that use the config options properly won't break the
>>>> cells job (and block nova in the gate).
>>>>
>>>> This is actually something we should do regardless of voting or not on
>>>> nova since new tests get added which might not fall in the regex and
>>>> break the cells job.  So I'm going to propose some changes so that the
>>>> regex will be moved to devstack-gate (regex exodus (tm)) and we'll work
>>>> on whittling down the regex there (and run those d-g changes against the
>>>> tempest-dsvm-cells job in the experimental queue).
>>>>
>>>> The question for the nova team is, shall we make the tempest-dsvm-cells
>>>> job voting on nova changes knowing that the gate can be broken with a
>>>> change to tempest that isn't caught in the regex?  In my opinion I think
>>>> we should make it voting so we don't regress cells with changes to nova
>>>> that go unnoticed with the non-voting job today.  Cells v2 is a nova
>>>> priority for Liberty so we don't want setbacks now that it's stable.

I would be tempted to risk it, given the large gain, but I am a little
biased too.

But with us controlling the regex, that seems a much easier call to say yes.

>>>> If a change does land in tempest which breaks the cells job and blocks
>>>> nova, we (1) fix it or (2) modify the regex so it's excluded until fixed
>>>> as has been done up to this point.
>>>>
>>>> I think we should probably mull this over in the ML and then vote on it
>>>> in this week's nova meeting.
>>>>
>>>> [1] https://review.openstack.org/#/c/190894/
>>>> [2]
>>>>
>>>> http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n1004
>>>>
>>>>
>>>>
>>>
>>> Regarding your "regex exodus", I recently added something for this.  In
>>> another project, I'm setting the regex in a file I keep in the code repo
>>> instead of project-config.
>>>
>>> support for DEVSTACK_GATE_SETTINGS in devstack-gate:
>>> https://review.openstack.org/190321
>>>
>>> usage in a job definition: https://review.openstack.org/190325
>>>
>>> a DEVSTACK_GATE_SETTINGS file that sets DEVSTACK_GATE_TEMPEST_REGEX:
>>> https://review.openstack.org/186894
>>>
>>> It all seems to be working for me, except I still need to tweak my regex
>>> to get the job passing, but at least I can do that without updating
>>> project-config now.
>>>
>>
>> Awesome, that is way cleaner.  I'll go that route instead, thanks!
>>
>
> Here is the change that moves the regex into nova:
>
> https://review.openstack.org/#/c/194411/

This seems like a good best of both worlds. Awesome stuff.

Thanks,
John