Open Stack

On 02/12/2014 01:48 PM, Maru Newby wrote:
> At the last 2 summits, I've suggested that API tests could be maintained in the Neutron tree and reused by Tempest.  I've finally submitted some patches that demonstrate this concept:
> 
> https://review.openstack.org/#/c/72585/  (implements a unit test for the lifecycle of the network resource)
> https://review.openstack.org/#/c/72588/  (runs the test with tempest rest clients)
> 
> My hope is to make API test maintenance a responsibility of the Neutron team.  The API compatibility of each Neutron plugin has to be validated by Neutron tests anyway, and if the tests are structured as I am proposing, Tempest can reuse those efforts rather than duplicating them.
> 
> I've added this topic to this week's agenda, and I would really appreciate it interested parties would take a look at the patches in question to prepare themselves to participate in the discussion.
> 
> 
> m.
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

Realistically, having API tests duplicated in the Tempest tree is a
feature, not a bug.

tempest/api is there for double book keep accounting, and it has been
really effective at preventing accidental breakage of our APIs (which
used to happen all the time), so I don't think putting API testing in
neutron obviates that.

Today most projects (excepting swift... which I'll get to in a second)
think about testing in 2 ways. Unit tests driven by tox in a venv, and
tempest tests in a devstack environment.

Because of this dualism, people have put increasingly more awkward live
environments in the tox unit tests jobs. For instance, IIRC, neutron
actually starts a full wsgi stack to tests every single in tree plugin,
instead of just testing the driver call down path.

Swift did something a little different. They have 3 classes of things.
Unit tests, Tempest Tests, and Swift Functional tests. The Swift
functional tests run in a devstack, but not with Tempest. Instead they
run their own suite.

This test suite only runs on the swift project, not on other projects,
and it's something they can use to test functional scenarios that
wouldn't fit in tempest, and extend to their heart's content, not having
to worry about the interaction with other components, because it's only
pushing on swift.

Going down this third path with neutron testing is I think very
valuable. Honestly, I'd like to encourage more projects to do this as
well. It would give them a greater component assuredness before entering
the integrated gate.

	-Sean

-- 
Sean Dague
Samsung Research America
sean at dague.net / sean.dague at samsung.com
http://dague.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 547 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140212/8cb18692/attachment.pgp>