Open Stack

Hey folks,

This is important info for DefCore (and Refstack to some extent) to track, as tests will be moving, but we will also get a much clearer picture of what the Neutron project considers "core" functionality, and what is considered advanced functionality.  

Worth following and worth commenting on.

--Rocky

-----Original Message-----
From: Assaf Muller [mailto:assaf at redhat.com] 
Sent: Friday, May 13, 2016 3:54 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Neutron][QA] Call to action - Neutron/Tempest API tests dedup

TL;DR: I'm looking for volunteers for tasks 1, 2 and 3 listed below.
Help would be hugely appreciated and %(local_drink)s will be bought in
Barcelona. I've posted example patches that demonstrate the idea.
Needless to say I'm here to provide reviews and to answer questions.
Additionally, most of the discussions have been within the Neutron
community and I'm looking for feedback from Tempest folks.


The context:
The Neutron community has been engaged in a long running effort to
move some of the networking API tests to the Neutron tree. We started
by copying the api/network directory tree, later keeping only the
tests, importing the test infrastructure itself from Tempest. We
continued by minimizing the imports we do from Tempest (Excluding
tempest.lib), and introduced a Neutron Tempest plugin.

One issue that remains is that some of the tests are still found in
both repositories. This confuses contributors and wastes compute
resources. Since the tests run against stable/{liberty|mitaka} and
master, it should be safe to dedup. I proposed a line in the sand so
that 'core resources' remain to be tested in Tempest and more
'advanced' APIs are tested in Neutron. The concept was agreed upon by
the Neutron and (Then) Tempest PTLs, and the specifics were discussed
and a consensus was found in patch [2]. Here is the resulting doc for
your viewing pleasure [5].


The work:
After I removed the API tests for core resources from the Neutron
tree, there remain three tasks to finish the de-dup:

1*) Remove tests for advanced APIs from Tempest. The full list of
tests that I propose be removed from Tempest is tracked here [1] (With
the rationale found at [2]), and an example patch may be found here
[4].
2) Push tests for Neutron core resources that were added after the
fork from Tempest, then delete these from Neutron. This is also
tracked in [1], with example patches found here [6]. This is not a
strict cut/paste as the way Tempest and Neutron interact with clients
is slightly different. Fun!
3) Sync tests for Neutron core resources that were updated after the
fork from Tempest. Test modifications include: Bug fixes for raceful
tests, py3 fixes, doc string typos and more. This is also tracked in
[1], with example patches found here [3].

* I believe that as far as the Tempest test removal criteria found at
[7], this case falls under the first exception: 'The class of testing
has been decided to be outside the scope of tempest' and we may skip
the three prong rule for removal. Input welcome.

[1] https://etherpad.openstack.org/p/neutron-tempest-defork
[2] https://review.openstack.org/#/c/280427/
[3] https://review.openstack.org/#/c/316280/ +
https://review.openstack.org/#/c/316283/
[4] https://review.openstack.org/#/c/316183/
[5] docs.openstack.org/developer/neutron/devref/development.environment.html#api-tests
[6] https://review.openstack.org/#/c/316265/ +
https://review.openstack.org/#/c/316269/
[7] https://wiki.openstack.org/wiki/QA/Tempest-test-removal

The work is tracked via:
* https://review.openstack.org/#/q/topic:bug/1552960
* https://bugs.launchpad.net/neutron/+bug/1552960
* https://etherpad.openstack.org/p/neutron-tempest-defork
* My head

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev