[openstack-dev] [QA][all] Propose to remove negative tests from Tempest
ayoung at redhat.com
Thu Mar 17 12:32:02 UTC 2016
On 03/16/2016 11:01 PM, Ken'ichi Ohmichi wrote:
> 2016-03-16 19:29 GMT-07:00 Adam Young <ayoung at redhat.com>:
>> On 03/16/2016 09:20 PM, Ken'ichi Ohmichi wrote:
>>> I have one proposal related to negative tests in Tempest, and
>>> hoping opinions before doing that.
>>> Now Tempest contains negative tests and sometimes patches are being
>>> posted for adding more negative tests, but I'd like to propose
>>> removing them from Tempest instead.
>>> Negative tests verify surfaces of REST APIs for each component without
>>> any integrations between components. That doesn't seem integration
>>> tests which are scope of Tempest.
>>> In addition, we need to spend the test operating time on different
>>> component's gate if adding negative tests into Tempest. For example,
>>> we are operating negative tests of Keystone and more
>>> components on the gate of Nova. That is meaningless, so we need to
>>> avoid more negative tests into Tempest now.
>>> If wanting to add negative tests, it is a nice option to implement
>>> these tests on each component repo with Tempest plugin interface. We
>>> can avoid operating negative tests on different component gates and
>>> each component team can decide what negative tests are valuable on the
>> Hear hear! For Keystone, please put them in the Keystone Client Functional
> They are negative tests of Keystone API, not keystoneclient tests.
> These tests are implemented with Tempest original rest clients without
> official clients because it is nice to test variable cases in Tempest.
> So it is nice to keep them in Keystone repo, I feel.
That works, too, and there is a Functional test section in Keystone Proper.
Most of the Keystone APIs are fairly well covered by the Unit tests,
which make a full HTTP call through a liteish server, so the functional
tests are really to cover the live database (soon to include LDAP,
> Ken Ohmichi
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
More information about the OpenStack-dev