Open Stack

On 18/04/17 14:28 -0400, Emilien Macchi wrote:
>On Mon, Apr 17, 2017 at 3:52 PM, Justin Kilpatrick <jkilpatr at redhat.com> wrote:
>> Because CI jobs tend to max out about 5 nodes there's a whole class of
>> minor bugs that make it into releases.
>>
>> What happens is that they never show up in small clouds, then when
>> they do show up in larger testing clouds the people deploying those
>> simply work around the issue and get onto what they where supposed to
>> be testing. These workarounds do get documented/BZ'd but since they
>> don't block anyone and only show up in large environments they become
>> hard for developers to fix.
>>
>> So the issue gets stuck in limbo, with nowhere to test a patchset and
>> no one owning the issue.
>>
>> These issues pile up and pretty soon there is a significant difference
>> between the default documented workflow and the 'scale' workflow which
>> is filled with workarounds which may or may not be documented
>> upstream.
>>
>> I'd like to propose getting these issues more visibility to having a
>> periodic upstream job that uses 20-30 ovb instances to do a larger
>> deployment. Maybe at 3am on a Sunday or some other time where there's
>> idle execution capability to exploit. The goal being to make these
>> sorts of issues more visible and hopefully get better at fixing them.
>
>Wait no, I know some folks at 3am on a Saturday night who use TripleO
>CI (ok that was a joke).

Jokes apart, it really depends on the TZ and when you schedule it. 3:00 UTC on a
Sunday is Monday 13:00 in Sydney :) Saturdays might work better but remember
that some countries work on Sundays.

>> To be honest I'm not sure this is the best solution, but I'm seeing
>> this anti pattern across several issues and I think we should try and
>> come up with a solution.
>>
>
>Yes this proposal is really cool. There is an alternative to run this
>periodic scenario outside TripleO CI and send results via email maybe.
>But it is something we need to discuss with RDO Cloud people and see
>if we would have such resources to make it on a weekly frequency.
>
>Thanks for bringing this up, it's crucial for us to have this kind of
>feedback, now let's take actions.

+1

Flavio

-- 
@flaper87
Flavio Percoco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 862 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170419/83afd263/attachment.sig>