Open Stack

Hi,

I will try to be short.

- Voting unit test coverage job is ready, and you can just use it as is
from rally source code:
   you need this file
https://github.com/openstack/rally/blob/master/tests/ci/cover.sh
   and this change in tox:
https://github.com/openstack/rally/blob/master/tox.ini#L51-L52

- Rally is in gates, and it's easy to add jobs in any project. If you have
any problems with this
  just ping me or someone from Rally team (or just write comment in
openstack-rally IRC)

- Rally was a performance tool, however that change  and now we are more
like common testing
  framework, that allows to do various kinds of testing (perf, volume,
stress, functional, ...)

- In Rally we were testing all plugins with relative small concurrency
(already for more then 1.5 year),
  and I can say that we faced a lot of issues with concurrency (and we are
still facing).
  However I can't give guarantee that we are facing 100% of cases
  (however facing most of issues is better then nothing)



Best regards,
Boris Pavlovic

On Wed, Mar 2, 2016 at 7:30 AM, Michał Dulko <michal.dulko at intel.com> wrote:

> On 03/02/2016 04:11 PM, Gorka Eguileor wrote:
> > On 02/03, Ivan Kolodyazhny wrote:
> >> Eric,
> >>
> >> There are Gorka's patches [10] to remove API Races
> >>
> >>
> >> [10]
> >>
> https://review.openstack.org/#/q/project:openstack/cinder+branch:master+topic:fix/api-races-simplified
> >>
> > I looked at Rally a long time ago so apologies if I'm totally off base
> > here, but it looked like it was a performance evaluation tool, which
> > means that it probably won't help to check for API Races (at least I
> > didn't see how when I looked).
> >
> > Many of the API races only happen if you simultaneously try the same
> > operation multiple times against the same resource or if there are
> > different operations that are trying to operate on the same resource.
> >
> > On the first case if Rally allowed it we could test it because we know
> > only 1 of the operations should succeed, but on the second case when we
> > are talking about preventing races from different operations there is no
> > way to know what the result should be, since the order in which those
> > operations are executed on each test run will determine which one will
> > fail and which one will succeed.
> >
> > I'm not trying to go against the general idea of adding rally tests, I
> > just think that they won't help in the case of the API races.
>
> You're probably right - Rally would need to cache API responses to
> parallel runs, predict the result of accepted requests (these which
> haven't received VolumeIsBusy) and then verify it. In case of API race
> conditions things explode inside the stack, and not on the API response
> level. The issue is that two requests, that shouldn't ever be accepted
> together, get positive API response.
>
> I cannot say it's impossible to implement a situation like that as Rally
> resource, but definitely it seems non-trivial to verify if result is
> correct.
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160302/b6ab6206/attachment.html>