Open Stack

2016-03-17 3:22 GMT-07:00 Jordan Pittier <jordan.pittier at scality.com>:
>>
>> If wanting to add negative tests, it is a nice option to implement
>> these tests on each component repo with Tempest plugin interface. We
>> can avoid operating negative tests on different component gates and
>> each component team can decide what negative tests are valuable on the
>> gate.
>>
>> In long term, all negative tests will be migrated into each component
>> repo with Tempest plugin interface. We will be able to operate
>> valuable negative tests only on each gate.
>>
>> Any thoughts?
>
>
> I am not sure we should remove negative tests from Tempest. Agreed that we
> should reject most new negative tests, but some negative
> tests do test useful things imo. Also I ran all the negative tests today:
> "Ran: 452 tests in 144.0000 sec." They just account for 2 minutes and 20sec
> in the gate. That's very little, removing them won't bring a lot. And the
> code for negative tests is quite contain, not a big maintenance burden.

This is a nice point. The merit of removal is less from the operation time.

My concern was the existing negative tests attract new ones like "ah,
current Tempest doesn't cover these (corner) cases. We should add
them".
I agree with keeping the existing negative tests if we reject new
corner negative tests.

Thanks
Ken Ohmichi