<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
That's great. We are in the process of creating a Jenkins slave to
run on a collection of servers we have. I think this would be good
for some black box tests but I have been having some problems with
nova failures. At the design summit I thought I heard that there
were such tests but they were not currently working. Is there an
existing black box test for a nova cluster that passes, using diablo
and/or trunk, that I could try running?<br>
<br>
David<br>
<br>
On 10/10/2011 11:52 AM, Gabe Westmaas wrote:
<blockquote cite="mid:CAB88C97.2FC1%25gabe.westmaas@rackspace.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<div>I'd like to try to summarize and propose at least one next
step for the content of the openstack-integration-tests git
repository. Note that this is only about the actual tests
themselves, and says absolutely nothing about any gating
decisions made in other sessions. </div>
<div><br>
</div>
<div>First, there was widespread agreement that in order for an
integration suite to be run in the openstack jenkins, it should
be included in the community github repository.</div>
<div><br>
</div>
<div>Second, it was agreed that there is value in having tests in
multiple languages, especially in the case where those tests add
value beyond the base language. Examples of this may include
testing using another set of bindings, and therefore testing the
API. Using a testing framework that just takes a different
approach to testing. Invalid examples include implanting the
exact same test in another language simply because you don't
like python.</div>
<div><br>
</div>
<div>Third, it was agreed that there is value in testing using
novaclient as well as httplib2. Similarly that there is value
in testing both XML and JSON.</div>
<div><br>
</div>
<div>Fourth, for black box tests, any fixture setup that a suite
of tests requires should be done via script that is close to but
not within that suite – we want tests to be as agnostic to an
implementation of openstack as possible, and anything you cannot
do from the the API should not be inside the tests.</div>
<div><br>
</div>
<div>Fifth, there are suites of white box tests – we understand
there can be value here, but we aren't sure how to approach that
in this project, definitely more discussion needed here. Maybe
we have a separate directory for holding white and black box
tests?</div>
<div><br>
</div>
<div>Sixth, no matter what else changes, we must maintain the
ability to run a subset of tests through a common runner. This
can be done via command line or configuration, whichever makes
the most sense. I'd personally lean towards configuration with
the ability to override on the command line.</div>
<div><br>
</div>
<div>If you feel I mischaracterized any of the agreements, please
feel free to say so.</div>
<div><br>
</div>
<div>Next, we want to start moving away from the multiple entry
points to write additional tests. That means taking inventory
of the tests that are there now, and figuring out what they are
testing, and how we run them, and then working to combine what
makes sense, into a directory structure that makes sense. As
often as possible, we should make sure the tests can be run in
the same way. I started a little wiki to start collecting
information. I think a short description of the general
strategy of each suite and then details about the specific tests
in that suite would be useful.</div>
<div><br>
</div>
<div><a moz-do-not-send="true"
href="http://wiki.openstack.org/openstack-integration-test-suites">http://wiki.openstack.org/openstack-integration-test-suites</a></div>
<div><br>
</div>
<div>Hopefully this can make things a little easier to start
contributing.</div>
<div><br>
</div>
<div>Gabe</div>
<font face="monospace">This email may include confidential
information. If you received it in error, please delete it.</font>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Mailing list: <a class="moz-txt-link-freetext" href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a>
Post to : <a class="moz-txt-link-abbreviated" href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a>
Unsubscribe : <a class="moz-txt-link-freetext" href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a>
More help : <a class="moz-txt-link-freetext" href="https://help.launchpad.net/ListHelp">https://help.launchpad.net/ListHelp</a>
</pre>
</blockquote>
<br>
</body>
</html>