Open Stack

Hi,

I am in the process of distilling all our conversations around the
suitability of the current tests for interoperability and I am putting
all the facts together in this email as the starting point to writing a
draft of the DefCore spec.

*The midcycle conversation*
We all seem to remember that we decided to write a DefCore test spec
back in the midcycle[0]. Just wanted to remind everyone the straw poll
questions that we took to foster this discussion[1].

The only thing that seems to be clear from that discussion is that we
needed a spec for interop/portability that would help frame the
interoperability discussion/testing.

We didn't agree on a way of owning tests because the resolution of this
discussion was that we would write interop specs and the discussion
about who owns what and how would hopefully be easier to answer after we
have a spec.

*Test analysis*
On a different session on the second day we agreed that Darrel and his
team of testers would do an analysis of the current tests and come back
with results of this analysis as input for the spec/test ownership
discussion. We have two links to results of this analysis[2]:

 -
https://github.com/dwalleck/defcore-tools/blob/master/manual_test_analysis.txt
 -
https://github.com/dwalleck/defcore-tools/blob/master/generated_test_analysis.txt

I will be working with him on getting more up to date information on
this front. And also figure out if he has already gone through the data
and made a list of potential test refactorings.

*The spec*

After a bit of thinking I have decided to work on this document in the
open on an etherpad so that we can collaborate on it and come up with a
draft that we can then propose and discuss in gerrit:

https://etherpad.openstack.org/p/DefCoreSpec-draft

This is just a boiler plate spec to fill out. Anybody with ideas can
just dump them in "other thoughts" and we will be writing the doc in a
collaborative manner. If you have sections to add that you think are
necessary, by all means do so even if you don't have content for them yet.

Cheers,
Gema


[2] https://etherpad.openstack.org/p/DefCoreRing.16
[1] Summary of straw poll during the sprint:

*Question #1*: Who is in favor of dedicated defcore interoperability
tests, maintained by DefCore and separate from developer-maintained tests?

Yes: +10
	- as a tag to upstream tests
	- if DefCore does  not have to maintain the test
	- need stable tests that match the spec

No: +1
	- prefer to augment  Tempest or extend Rally

*Question #2*: Does defcore needs ownership of test suite (to match the
spec mentioned in Q3, below)?

Yes: +8
No: +5

*Question #3*: Should DefCore define an explicit spec (for
interop/portability)?

Yes: +12
No: 0

[0]https://etherpad.openstack.org/p/DefCoreSpring2016MidCycle

-- 
Gema Gomez-Solano        <gema.gomez-solano at canonical.com>
STS, QE                  https://launchpad.net/~gema
Canonical Ltd.           http://www.canonical.com