Open Stack

On 03/02/2016 04:11 PM, Gorka Eguileor wrote:
> On 02/03, Ivan Kolodyazhny wrote:
>> Eric,
>>
>> There are Gorka's patches [10] to remove API Races
>>
>>
>> [10]
>> https://review.openstack.org/#/q/project:openstack/cinder+branch:master+topic:fix/api-races-simplified
>>
> I looked at Rally a long time ago so apologies if I'm totally off base
> here, but it looked like it was a performance evaluation tool, which
> means that it probably won't help to check for API Races (at least I
> didn't see how when I looked).
>
> Many of the API races only happen if you simultaneously try the same
> operation multiple times against the same resource or if there are
> different operations that are trying to operate on the same resource.
>
> On the first case if Rally allowed it we could test it because we know
> only 1 of the operations should succeed, but on the second case when we
> are talking about preventing races from different operations there is no
> way to know what the result should be, since the order in which those
> operations are executed on each test run will determine which one will
> fail and which one will succeed.
>
> I'm not trying to go against the general idea of adding rally tests, I
> just think that they won't help in the case of the API races.

You're probably right - Rally would need to cache API responses to
parallel runs, predict the result of accepted requests (these which
haven't received VolumeIsBusy) and then verify it. In case of API race
conditions things explode inside the stack, and not on the API response
level. The issue is that two requests, that shouldn't ever be accepted
together, get positive API response.

I cannot say it's impossible to implement a situation like that as Rally
resource, but definitely it seems non-trivial to verify if result is
correct.