<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"> Tue, Dec 10, 2013 at 12:43 PM, David Kranz <span dir="ltr"><<a href="mailto:dkranz@redhat.com" target="_blank">dkranz@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><div><div class="h5">
<div>On 12/09/2013 01:37 PM, Devananda van
der Veen wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Fri, Dec 6, 2013 at 2:13 PM, Clark
Boylan <span dir="ltr"><<a href="mailto:clark.boylan@gmail.com" target="_blank">clark.boylan@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div>
<div>On Fri, Dec 6, 2013 at 1:53 PM, David
Kranz <<a href="mailto:dkranz@redhat.com" target="_blank">dkranz@redhat.com</a>>
wrote:<br>
> It's great that tempest tests for ironic have
been submitted! I was<br>
> reviewing <a href="https://review.openstack.org/#/c/48109/" target="_blank">https://review.openstack.org/#/c/48109/</a>
and noticed that the tests<br>
> do not actually run. They are skipped because
baremetal is not enabled. This<br>
> is not terribly surprising but we have had a
policy in tempest to only merge<br>
> code that has demonstrated that it works. For
services that cannot run in<br>
> the single-vm environment of the upstream gate we
said there could be a<br>
> system running somewhere that would run them and
report a result to gerrit.<br>
> Is there a plan for this, or to make an exception
for ironic?<br>
><br>
> -David<br>
><br>
> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</div>
</div>
There is a change[0] to openstack-infra/config to add
experimental<br>
tempest jobs to test ironic. I think that change is close
to being<br>
ready, but I need to give it time for a proper review.
Once in that<br>
will allow you to test 48109 (in theory, not sure if all
the bits will<br>
just work). I don't think these tests fall under the
cannot run in a<br>
single vm environment umbrella, we should be able to test
the<br>
baremetal code via the pxe booting of VMs within the
single VM<br>
environment.<br>
<br>
[0] <a href="https://review.openstack.org/#/c/53917/" target="_blank">https://review.openstack.org/#/c/53917/</a><br>
<span><font color="#888888"><br>
<br>
Clark<br>
</font></span>
<div>
<div><br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>We can test the ironic services, database, and the
driver interfaces by using our "fake" driver within a
single devstack VM today (I'm not sure the exercises for
all of this have been written yet, but it's practical to
test it). OTOH, I don't believe we can test a PXE deploy
within a single VM today, and need to resume discussions
with infra about this.</div>
<div><br>
</div>
<div>There are some other aspects of Ironic (IPMI, SOL
access, any vendor-specific drivers) which we'll need real
hardware to test because they can't effectively be
virtualized. TripleO should cover some (much?) of those
needs, once they are able to switch to using Ironic
instead of nova-baremetal.</div>
<div><br>
</div>
<div>-Devananda</div>
</div>
</div>
</div>
</blockquote></div></div>
So it seems that the code in the submitted tempest tests can run in
a regular job if devstack is configured to enable ironic, but that
this cannot be the default. So I propose that we create a regular
devstack+ironic job that will run in the ironic and tempest gates,
and run just the ironic tests. When third-party bare-metal results
can be reported for ironic, tempest can then accept tests that
require bare-metal. Does any one have a problem with this approach?<span class=""><font color="#888888"><br>
<br>
-David<br>
</font></span></div>
<br></blockquote><div><br></div><div>As I understand it, the infra/config patch which Clark already linked (<a href="https://review.openstack.org/#/c/53917">https://review.openstack.org/#/c/53917</a>), which has gone through several iterations, should be enabling Ironic within devstack -- and thus causing tempest to run the relevant tests -- within the Ironic and Tempest check and gate pipelines. This will exercise Ironic's API by performing CRUD actions on resources. It doesn't do any more than that yet.</div>
<div><br></div><div><div>David, I'm not sure what you mean by "when third-party bare-metal results can be reported for ironic" -- I don't see any reason why we couldn't accept third-party smoke tests right now, except that none of the tempest tests are written... Am I missing something?<br>
</div></div><div><br></div><div>In the longer term, we are planning to enable tempest testing of deployment by ironic within devstack-gate as all the pieces come together. This will take a fair bit more work / time, but I'm going to start nudging resources in this direction very soon. In fact, we just talked about this in #infra for a bit. Here's an attempt to summarize what came of it w.r.t. Ironic's testing plans. We will need:</div>
<div><br></div><div>- some changes in devstack-gate to prepare a new environment by...</div><div>-- install sshd + firewall it to only allow connections from localhost<br></div><div>-- create a bunch of tiny qemu VMs (of configurable size and number)</div>
<div>- some changes in devstack to...</div><div>-- suck up a list of those VM's MAC addresses and enroll them in Ironic</div><div>-- configure nova to use ironic</div><div>-- configure ironic to use the pxe+ssh driver</div>
<div>- a new test job that turns all this on, thus allowing tempest to do all its usual work against a "virtual baremetal" cloud</div><div><br></div><div>Also, it's worth mentioning, the above-described plan won't exercise the CRUD of Ironic resources -- I think they need to be pre-enrolled with Ironic before tempest starts (maybe not? maybe tempest can enroll them, instead of devstack?). This is one reason why we have proposed separate tempest tests for exercising Ironic's API. Another reason is, testing our API is a valuable thing all by itself, and has a much lower development cost than all the changes above.</div>
<div><br></div><div>-Devananda</div></div></div></div>