Open Stack

On Wed, Mar 16, 2016 at 10:41 PM, Jim Rollenhagen
<jim at jimrollenhagen.com> wrote:
> On Wed, Mar 16, 2016 at 06:20:11PM -0700, Ken'ichi Ohmichi wrote:
>> Hi
>>
>> I have one proposal[1] related to negative tests in Tempest, and
>> hoping opinions before doing that.
>>
>> Now Tempest contains negative tests and sometimes patches are being
>> posted for adding more negative tests, but I'd like to propose
>> removing them from Tempest instead.
>>
>> Negative tests verify surfaces of REST APIs for each component without
>> any integrations between components. That doesn't seem integration
>> tests which are scope of Tempest.
>> In addition, we need to spend the test operating time on different
>> component's gate if adding negative tests into Tempest. For example,
>> we are operating negative tests of Keystone and more
>> components on the gate of Nova. That is meaningless, so we need to
>> avoid more negative tests into Tempest now.
>>
>> If wanting to add negative tests, it is a nice option to implement
>> these tests on each component repo with Tempest plugin interface. We
>> can avoid operating negative tests on different component gates and
>> each component team can decide what negative tests are valuable on the
>> gate.
>>
>> In long term, all negative tests will be migrated into each component
>> repo with Tempest plugin interface. We will be able to operate
>> valuable negative tests only on each gate.
>
> So, positive tests in tempest, negative tests as a plugin.
>
> Is there any longer term goal to have all tests for all projects in a
> plugin for that project? Seems odd to separate them.

I'd love to see this idea explored further. What happens if Tempest
ends up without tests, as a library for shared code as well as a
centralized place to run tests from via plugins?

>
> // jim
>
>>
>> Any thoughts?
>>
>> Thanks
>> Ken Ohmichi
>>
>> ---
>> [1]: https://review.openstack.org/#/c/293197/
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev