<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 21/02/13 21:00, David Kranz wrote:<br>
</div>
<blockquote cite="mid:51266ECB.3070602@qrclab.com" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<div class="moz-cite-prefix">Joe, that's great. This is part of
the wonderful explosion of tests being contributed to Tempest. I
am concerned about the performance of the gate. Do we really
want to run a complete test of novaclient every time some one
checks something into swift? Or any project other than nova and
novaclient? I am not trying to single out this change but the
current situation is that a test either gates all projects, or
none. I don't see how this is sustainable. I have submitted a
summit session about this <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://summit.openstack.org/cfp/details/23">http://summit.openstack.org/cfp/details/23</a>.<br>
<br>
-David<br>
</div>
</blockquote>
<br>
Depends on the client and the project, I guess. <br>
AFAIK (client:users):<br>
<br>
keystoneclient:<br>
- Everything, servers and clients<br>
<br>
novaclient:<br>
- Horizon<br>
<br>
cinderclient:<br>
- Horizon<br>
- Nova<br>
<br>
quantumclient:<br>
- Horizon<br>
- Nova<br>
<br>
glanceclient:<br>
- Horizon<br>
- Nova<br>
<br>
Y.<br>
<br>
<blockquote cite="mid:51266ECB.3070602@qrclab.com" type="cite">
<div class="moz-cite-prefix"> <br>
On 2/21/2013 1:47 PM, Joe Gordon wrote:<br>
</div>
<blockquote
cite="mid:CAKe5d-QjLism1M8EwQ7ACKSKZXZDyaA1OnQ2=RSnyCScLQNE+g@mail.gmail.com"
type="cite">Hi All,
<div><br>
</div>
<div>I am pleased to say that the CLI tests are now gating (<a
moz-do-not-send="true"
href="https://review.openstack.org/#/c/22096/">https://review.openstack.org/#/c/22096/</a>).
They run at the end of the standard tempest test (example: <a
moz-do-not-send="true"
href="http://logs.openstack.org/22546/5/check/gate-tempest-devstack-vm-full/4878/console.html.gz">http://logs.openstack.org/22546/5/check/gate-tempest-devstack-vm-full/4878/console.html.gz</a>).
<div> <br>
</div>
<div>The current set of tests are very basic and need a lot of
work. But now that the CLI tests are self-gating it should
be easy for anyone to contribute to them. Currently we
have:</div>
<div>* uses tempest.conf</div>
<div>* in unittest framework</div>
<div>* admin user only</div>
<div>* python-novaclient only</div>
<div>* read-only tests</div>
<div><br>
</div>
<div>But we would like to have:</div>
<div>* Tests for all python-*clients</div>
<div> * Tests for all *-manage binaries</div>
<div>* Tests for both admin and regular users</div>
<div>* Read/Write tests (spin up a VM with client do something
etc)</div>
<div><br>
</div>
<div>
<div>
<div><br>
</div>
<div>Daryl, as for your WIP patch, I will take a look at
it hopefully there are some things we can use from it.</div>
<div><br>
</div>
<div>best,</div>
<div>Joe Gordon</div>
<div><br>
<div class="gmail_quote">On Thu, Feb 21, 2013 at 8:18
AM, Daryl Walleck <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:daryl.walleck@rackspace.com"
target="_blank">daryl.walleck@rackspace.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Ahh,
I didn't realize we an agreed upon solution for
command line testing already. If there's nothing of
use that can be pulled from what I submitted, I'll
withdraw the branch.<br>
<br>
Daryl<br>
________________________________________<br>
From: Jay Pipes [<a moz-do-not-send="true"
href="mailto:jaypipes@gmail.com">jaypipes@gmail.com</a>]<br>
Sent: Thursday, February 21, 2013 9:44 AM<br>
To: <a moz-do-not-send="true"
href="mailto:openstack-qa@lists.openstack.org">openstack-qa@lists.openstack.org</a><br>
<div class="HOEnZb">
<div class="h5">Subject: Re: [openstack-qa] New
Client Test Suite<br>
<br>
On 02/21/2013 10:24 AM, Daryl Walleck wrote:<br>
> It took me a bit longer than I hoped, but I
but a basic outline of some of the command line
tests we have out in a WIP branch. If there's
enough interest, I can get this merged into
Tempest properly.<br>
><br>
> <a moz-do-not-send="true"
href="https://review.openstack.org/#/c/22552/"
target="_blank">https://review.openstack.org/#/c/22552/</a><br>
<br>
Hi Daryl!<br>
<br>
We now have a CLI test module in Tempest:<br>
<br>
<a moz-do-not-send="true"
href="https://github.com/openstack/tempest/tree/master/cli"
target="_blank">https://github.com/openstack/tempest/tree/master/cli</a><br>
<br>
If we bring in your CLI tests, I think I'd
prefer to:<br>
<br>
a) Keep the code style consistent with the rest
of Tempest<br>
b) Use the simple framework that Joe put
together for the cli tests<br>
currently in Tempest<br>
<br>
Best,<br>
-jay<br>
<br>
> Daryl<br>
> ________________________________________<br>
> From: Sam Danes [<a moz-do-not-send="true"
href="mailto:sam.danes@RACKSPACE.COM">sam.danes@RACKSPACE.COM</a>]<br>
> Sent: Thursday, February 14, 2013 11:15 AM<br>
> To: All Things QA.<br>
> Subject: Re: [openstack-qa] New Client Test
Suite<br>
><br>
> I've given Daryl a pointer to the prototype
code we wrote. He'll work on getting it out to
the community.<br>
><br>
> -----Original Message-----<br>
> From: Attila Fazekas [mailto:<a
moz-do-not-send="true"
href="mailto:afazekas@redhat.com">afazekas@redhat.com</a>]<br>
> Sent: Thursday, February 14, 2013 7:33 AM<br>
> To: All Things QA.<br>
> Subject: Re: [openstack-qa] New Client Test
Suite<br>
><br>
> Very good idea!<br>
><br>
> I think it can fit into tempest repository,
but in a different directory.<br>
><br>
> I would extend a scope too.<br>
><br>
> At the first Round:<br>
> - *mange command tests.<br>
> - remote execution (with ssh connection
pooling) The whitebox test type (not the
existing test case), partially should be
converted to utilities, partially the
functionality should move to the new folder.<br>
> <a moz-do-not-send="true"
href="https://github.com/openstack/tempest/blob/master/tempest/whitebox.py#L138"
target="_blank">https://github.com/openstack/tempest/blob/master/tempest/whitebox.py#L138</a><br>
><br>
><br>
> ----- Original Message -----<br>
>> From: "Daryl Walleck" <<a
moz-do-not-send="true"
href="mailto:daryl.walleck@RACKSPACE.COM">daryl.walleck@RACKSPACE.COM</a>><br>
>> To: "All Things QA." <<a
moz-do-not-send="true"
href="mailto:openstack-qa@lists.openstack.org">openstack-qa@lists.openstack.org</a>><br>
>> Sent: Thursday, February 14, 2013
3:33:20 AM<br>
>> Subject: Re: [openstack-qa] New Client
Test Suite<br>
>><br>
>><br>
>><br>
>><br>
>><br>
>> We've bumped into some Novaclient only
issues before in the past as<br>
>> well. Sam and some of his team did
prototype some tests that execute<br>
>> using Novaclient. If there's any
interest I can see about getting it<br>
>> into a WIP branch to get eyes on it.<br>
>><br>
>><br>
>><br>
>> Daryl<br>
>><br>
>><br>
>> From: Joe Gordon [<a
moz-do-not-send="true"
href="mailto:jogo@cloudscaling.com">jogo@cloudscaling.com</a>]<br>
>> Sent: Tuesday, February 12, 2013 4:39
PM<br>
>> To: <a moz-do-not-send="true"
href="mailto:openstack-qa@lists.openstack.org">openstack-qa@lists.openstack.org</a><br>
>> Subject: [openstack-qa] New Client Test
Suite<br>
>><br>
>><br>
>><br>
>><br>
>> Hi All,<br>
>><br>
>> We currently have thorough API tests
(Tempest) that do an excellent<br>
>> job of testing nova, glance, keystone,
etc. But to date we don't have<br>
>> anything that tests the assorted python
clients.<br>
>><br>
>><br>
>> Extensive python client tests, which
would consist of mostly shell<br>
>> commands, would prevent many bugs from
arising such as:<br>
>> <a moz-do-not-send="true"
href="https://bugs.launchpad.net/python-novaclient/+bug/1122958"
target="_blank">https://bugs.launchpad.net/python-novaclient/+bug/1122958</a>
.<br>
>><br>
>><br>
>> The question is where should theses
tests live?<br>
>><br>
>><br>
>> * Devstack/smoketests - these are just
smoketests and are not designed<br>
>> to be exhaustive<br>
>> * Tempest - currently explicitly
doesn't use the python clients, but<br>
>> we can add a separate folder that tests
the clients<br>
>> * In respective client repos - easier
to require new tests when adding<br>
>> to client, but will result in divergent
test suites<br>
>> * Separate Repo - perhaps this is
premature?<br>
>><br>
>><br>
>> I propose adding a separate folder in
tempest to run the python client<br>
>> tests, and taking it from there.<br>
>><br>
>><br>
>><br>
>><br>
>> best,<br>
>> Joe Gordon<br>
>>
_______________________________________________<br>
>> openstack-qa mailing list<br>
>> <a moz-do-not-send="true"
href="mailto:openstack-qa@lists.openstack.org">openstack-qa@lists.openstack.org</a><br>
>> <a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa</a><br>
>><br>
><br>
>
_______________________________________________<br>
> openstack-qa mailing list<br>
> <a moz-do-not-send="true"
href="mailto:openstack-qa@lists.openstack.org">openstack-qa@lists.openstack.org</a><br>
> <a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa</a><br>
><br>
>
_______________________________________________<br>
> openstack-qa mailing list<br>
> <a moz-do-not-send="true"
href="mailto:openstack-qa@lists.openstack.org">openstack-qa@lists.openstack.org</a><br>
> <a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa</a><br>
><br>
>
_______________________________________________<br>
> openstack-qa mailing list<br>
> <a moz-do-not-send="true"
href="mailto:openstack-qa@lists.openstack.org">openstack-qa@lists.openstack.org</a><br>
> <a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa</a><br>
><br>
<br>
_______________________________________________<br>
openstack-qa mailing list<br>
<a moz-do-not-send="true"
href="mailto:openstack-qa@lists.openstack.org">openstack-qa@lists.openstack.org</a><br>
<a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa</a><br>
<br>
_______________________________________________<br>
openstack-qa mailing list<br>
<a moz-do-not-send="true"
href="mailto:openstack-qa@lists.openstack.org">openstack-qa@lists.openstack.org</a><br>
<a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa</a><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
openstack-qa mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:openstack-qa@lists.openstack.org">openstack-qa@lists.openstack.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa</a>
</pre>
</blockquote>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
openstack-qa mailing list
<a class="moz-txt-link-abbreviated" href="mailto:openstack-qa@lists.openstack.org">openstack-qa@lists.openstack.org</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa</a>
</pre>
</blockquote>
<br>
</body>
</html>