Open Stack

On 10/02/16 18:48, "Clint Byrum" <clint at fewbar.com> wrote:

>Excerpts from Sean Dague's message of 2016-02-10 04:33:44 -0800:
>> The largeops tests at this point are mostly finding out that some of our
>> new cloud providers are slow - http://tinyurl.com/j5u4nf5
>> 
>> This is fundamentally a performance test, with timings having been tuned
>> to pass 98% of the time on two clouds that were very predictable in
>> performance. We're now running on 4 clouds, and the variance between
>> them all, and between every run on each can be as much as a factor of 2.
>> 
>> We could just bump all the timeouts again, but that's basically the same
>> thing as dropping them.
>> 
>> These tests are not instrumented in a way that any real solution can be
>> addressed in most cases. Tests without a path forward, that are failing
>> good patches a lot, are very much the kind of thing we should remove
>> from the system.
>> 
>
>I think we need to replace this with something that measures work
>counters, and not clock time. As you say, some of the other test suites
>out there already pick up a lot of this slack too. Also, I'm working at
>this as well with the counter-inspection spec, so hopefully dropping
>this now won't leave too much of a gap in coverage while we ramp up
>counter-inspection.
>
>+1 to getting rid of it now, as instability in the test suites, which
>slows down development velocity, is worse than missing a few performance
>regressions in the corners.

I believe there is now a performance working group as part of the large deployments team (https://wiki.openstack.org/wiki/Performance_Team). Have they been contacted to
determine the alternative scenarios and how to strengthen the testing ?

Tim

>
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2792 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160210/077f3c98/attachment.bin>