[openstack-dev] [QA][ALL] What tempest tests will go under tempest plugin for a Project?

Chandan kumar chkumar246 at gmail.com
Tue Jun 27 06:52:59 UTC 2017

Hello Ghanshyam,

On Sat, Jun 24, 2017 at 3:48 PM, Ghanshyam Mann <ghanshyammann at gmail.com> wrote:
> On Fri, Jun 23, 2017 at 4:51 PM, Chandan kumar <chkumar246 at gmail.com> wrote:
>> Hello,
>> In Queen OpenStack release, We have a community goal to split In-Tree
>> tempest plugin to a separate repo[1.].
>> I have a couple question regarding the tempest tests movement within
>> tempest plugins.
>> [1.] Since some of the core OpenStack projects like Nova, Glance and
>> Swift does have tempest plugin currently.
>>      Their Tempest tests reside under tempest project repo.
>>      are we going to create tempest plugin for the same?
>>      If yes, what are the tempest tests (API/Scenario) tests moving
>> under tempest plugins?
>> [2.] And, like other core projects like neutron and cinder have their
>> in-tree tempest plugins also.
>>      And those are also moving to a separate repo and currently, their
>> tests also resides under tempest repo.
>>      How can we avoid the duplication of the tempest tests?
> Its same answer for 1 and 2. Tempest is a place to have integration
> tests and future tests also falls in same scope.
> Yes, we do have API tests negative as well as positive which are there
> because of defcore. Defcore need those for interop certification.
> Those will reside in Tempest as of now and so new tests can be added
> in Tempest if defcore require them.
> New or existing Tempest plugin for 6 core projects whose tests are
> present in Tempest, will target their functional/API/negative testing
> etc which are/should be out of scope of Tempest.
> Regarding the duplication of tests, we do take care of those while
> review of new tests addition. If there is any new tests proposed in
> Tempest, reviewers need to check whether same coverage is there on
> project side or not (either functional tests or in tempest plugin).
> Also if those are more appropriate to reside on project side. We will
> be continuing with the same process to avoid duplicate tests.

Thanks got it, So basically if any tests needed by DefCore as well as
integration tests
needed for Core  Projects will go under Tempest.

>> [3.] For other projects while moving tests to a separate repo how we
>> are going to collaborate together to avoid
>>      duplication and move common tests to Tempest?
> You mean tests in projects tree as functional tests etc and their
> tempest plugin ?

Yes, For example, Swift have lots of functional tests with in tests
folder of swift project tree.
Does it go under tempest plugin?
This part i am confused.

>  If so, then same process as done in Tempest. When
> adding new tests in tempest plugin, reviewers need to check the
> duplicate coverage of those tests.


Chandan Kumar

More information about the OpenStack-dev mailing list