Open Stack

On 03/16/2016 11:01 PM, Ken'ichi Ohmichi wrote:
> 2016-03-16 19:29 GMT-07:00 Adam Young <ayoung at redhat.com>:
>> On 03/16/2016 09:20 PM, Ken'ichi Ohmichi wrote:
>>> Hi
>>>
>>> I have one proposal[1] related to negative tests in Tempest, and
>>> hoping opinions before doing that.
>>>
>>> Now Tempest contains negative tests and sometimes patches are being
>>> posted for adding more negative tests, but I'd like to propose
>>> removing them from Tempest instead.
>>>
>>> Negative tests verify surfaces of REST APIs for each component without
>>> any integrations between components. That doesn't seem integration
>>> tests which are scope of Tempest.
>>> In addition, we need to spend the test operating time on different
>>> component's gate if adding negative tests into Tempest. For example,
>>> we are operating negative tests of Keystone and more
>>> components on the gate of Nova. That is meaningless, so we need to
>>> avoid more negative tests into Tempest now.
>>>
>>> If wanting to add negative tests, it is a nice option to implement
>>> these tests on each component repo with Tempest plugin interface. We
>>> can avoid operating negative tests on different component gates and
>>> each component team can decide what negative tests are valuable on the
>>> gate.
>>
>> Hear hear!  For Keystone, please put them in the Keystone Client Functional
>> tests.
> They are negative tests of Keystone API, not keystoneclient tests.
> These tests are implemented with Tempest original rest clients without
> official clients because it is nice to test variable cases in Tempest.
> So it is nice to keep them in Keystone repo, I feel.

That works, too, and there is a Functional test section in Keystone Proper.

Most of the Keystone APIs are fairly well covered by the Unit tests, 
which make a full HTTP call through a liteish server, so the functional 
tests are really to cover the live database (soon to include LDAP, 
hopefully) integrations.
>
> Thanks
> Ken Ohmichi
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev