Open Stack

On 06/12/2014 01:22 PM, Tim Bell wrote:
>> -----Original Message-----
>> From: Sean Dague [mailto:sean at dague.net]
>> Sent: 12 June 2014 17:37
>> To: OpenStack Development Mailing List (not for usage questions)
>> Subject: Re: [openstack-dev] Gate proposal - drop Postgresql configurations in
>> the gate
>>
> ...
>> But if we're talking about a devstack / tempest run, it's not really applicable.
>>
>> If someone can point me to a case where we've actually found this kind of bug
>> with tempest / devstack, that would be great. I've just *never* seen it. I was the
>> one that did most of the fixing for pg support in Nova, and have helped other
>> projects as well, so I'm relatively familiar with the kinds of fails we can discover.
>> The ones that Julien pointed really aren't likely to be exposed in our current
>> system.
>>
>> Which is why I think we're mostly just burning cycles on the existing approach
>> for no gain.
>>
> 
> In some cases, we've dropped support for drivers in OpenStack since they were not tested in the gate, on the grounds that if it is not tested, it is probably broken.
> 
> From my understanding, this change proposes to drop Postgres testing from the default gate. Yet, there does not seem to be a proposal to drop Postgres support.
> 
> Are these two positions consistent ?
> 
> (Just seeking clarification, I fully understand the difficulties involved in multiple parallel testing at our scale)

We're hitting a couple of inflection points.

1) We're basically at capacity for the unit work that we can do. Which
means it's time to start making decisions if we believe everything we
currently have running is more important than the things we aren't
currently testing.

Everyone wants multinode testing in the gate. It would be impossible to
support that given current resources.

2) We're far past the inflection point of people actually debugging jobs
when they go wrong.

The gate is backed up (currently to 24hrs) because there are bugs in
OpenStack. Those are popping up at a rate much faster than the number of
people who are willing to spend any time on them. And often they are
popping up in configurations that we're not all that familiar with.

Landing a gating job comes with maintenance. Maintenance in looking into
failures, and not just running recheck. So there is an overhead to
testing this many different configurations.

I think #2 is just as important to realize as #1. As such I think we
need to get to the point where there are a relatively small number of
configurations that Infra/QA support, and beyond that every job needs
sponsors. And if the job success or # of uncategorized fails go past
some thresholds, we demote them to non-voting, and if you are non-voting
for > 1 month, you get demoted to experimental (or some specific
timeline, details to be sorted).

	-Sean

-- 
Sean Dague
http://dague.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 478 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140612/647ca53f/attachment.pgp>