<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 12/10/2013 08:41 PM, Devananda van
der Veen wrote:<br>
</div>
<blockquote
cite="mid:CAExZKEr4WLW_2+BhT3wo=d1r-Q6A5r=MAfjJ-3DXmwDwZkwCXw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote"> Tue, Dec 10, 2013 at 12:43 PM, David
Kranz <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:dkranz@redhat.com" target="_blank">dkranz@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>
<div class="h5">
<div>On 12/09/2013 01:37 PM, Devananda van der Veen
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Fri, Dec 6, 2013
at 2:13 PM, Clark Boylan <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:clark.boylan@gmail.com"
target="_blank">clark.boylan@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div>
<div>On Fri, Dec 6, 2013 at 1:53 PM,
David Kranz <<a
moz-do-not-send="true"
href="mailto:dkranz@redhat.com"
target="_blank">dkranz@redhat.com</a>>
wrote:<br>
> It's great that tempest tests for
ironic have been submitted! I was<br>
> reviewing <a
moz-do-not-send="true"
href="https://review.openstack.org/#/c/48109/"
target="_blank">https://review.openstack.org/#/c/48109/</a>
and noticed that the tests<br>
> do not actually run. They are
skipped because baremetal is not
enabled. This<br>
> is not terribly surprising but we
have had a policy in tempest to only
merge<br>
> code that has demonstrated that
it works. For services that cannot run
in<br>
> the single-vm environment of the
upstream gate we said there could be a<br>
> system running somewhere that
would run them and report a result to
gerrit.<br>
> Is there a plan for this, or to
make an exception for ironic?<br>
><br>
> -David<br>
><br>
>
_______________________________________________<br>
> OpenStack-dev mailing list<br>
> <a moz-do-not-send="true"
href="mailto:OpenStack-dev@lists.openstack.org"
target="_blank">OpenStack-dev@lists.openstack.org</a><br>
> <a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</div>
</div>
There is a change[0] to
openstack-infra/config to add experimental<br>
tempest jobs to test ironic. I think that
change is close to being<br>
ready, but I need to give it time for a
proper review. Once in that<br>
will allow you to test 48109 (in theory,
not sure if all the bits will<br>
just work). I don't think these tests fall
under the cannot run in a<br>
single vm environment umbrella, we should
be able to test the<br>
baremetal code via the pxe booting of VMs
within the single VM<br>
environment.<br>
<br>
[0] <a moz-do-not-send="true"
href="https://review.openstack.org/#/c/53917/"
target="_blank">https://review.openstack.org/#/c/53917/</a><br>
<span><font color="#888888"><br>
<br>
Clark<br>
</font></span>
<div>
<div><br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>We can test the ironic services,
database, and the driver interfaces by
using our "fake" driver within a single
devstack VM today (I'm not sure the
exercises for all of this have been
written yet, but it's practical to test
it). OTOH, I don't believe we can test a
PXE deploy within a single VM today, and
need to resume discussions with infra
about this.</div>
<div><br>
</div>
<div>There are some other aspects of Ironic
(IPMI, SOL access, any vendor-specific
drivers) which we'll need real hardware to
test because they can't effectively be
virtualized. TripleO should cover some
(much?) of those needs, once they are able
to switch to using Ironic instead of
nova-baremetal.</div>
<div><br>
</div>
<div>-Devananda</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
So it seems that the code in the submitted tempest tests
can run in a regular job if devstack is configured to
enable ironic, but that this cannot be the default. So I
propose that we create a regular devstack+ironic job
that will run in the ironic and tempest gates, and run
just the ironic tests. When third-party bare-metal
results can be reported for ironic, tempest can then
accept tests that require bare-metal. Does any one have
a problem with this approach?<span class=""><font
color="#888888"><br>
<br>
-David<br>
</font></span></div>
<br>
</blockquote>
<div><br>
</div>
<div>As I understand it, the infra/config patch which Clark
already linked (<a moz-do-not-send="true"
href="https://review.openstack.org/#/c/53917">https://review.openstack.org/#/c/53917</a>),
which has gone through several iterations, should be
enabling Ironic within devstack -- and thus causing
tempest to run the relevant tests -- within the Ironic and
Tempest check and gate pipelines. This will exercise
Ironic's API by performing CRUD actions on resources. It
doesn't do any more than that yet.</div>
</div>
</div>
</div>
</blockquote>
It looks like that patch is adding ironic jobs to the experimental
queue but I think we want them on check/gate.<br>
<blockquote
cite="mid:CAExZKEr4WLW_2+BhT3wo=d1r-Q6A5r=MAfjJ-3DXmwDwZkwCXw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>
<div>David, I'm not sure what you mean by "when
third-party bare-metal results can be reported for
ironic" -- I don't see any reason why we couldn't accept
third-party smoke tests right now, except that none of
the tempest tests are written... Am I missing something?<br>
</div>
</div>
</div>
</div>
</div>
</blockquote>
I was assuming there were some ironic tests that actually need bare
metal resources to run. Perhaps there are not. Either way, we just
want to make sure that when tests are submitted to tempest we have
evidence that they have successfully run. Sounds like the CRUD tests
will just work the same way as our existing tests once ironic is
enabled in devstack.<br>
<blockquote
cite="mid:CAExZKEr4WLW_2+BhT3wo=d1r-Q6A5r=MAfjJ-3DXmwDwZkwCXw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div>
</div>
</div>
<div><br>
</div>
<div>In the longer term, we are planning to enable tempest
testing of deployment by ironic within devstack-gate as
all the pieces come together. This will take a fair bit
more work / time, but I'm going to start nudging resources
in this direction very soon. In fact, we just talked about
this in #infra for a bit. Here's an attempt to summarize
what came of it w.r.t. Ironic's testing plans. We will
need:</div>
<div><br>
</div>
<div>- some changes in devstack-gate to prepare a new
environment by...</div>
<div>-- install sshd + firewall it to only allow connections
from localhost<br>
</div>
<div>-- create a bunch of tiny qemu VMs (of configurable
size and number)</div>
<div>- some changes in devstack to...</div>
<div>-- suck up a list of those VM's MAC addresses and
enroll them in Ironic</div>
<div>-- configure nova to use ironic</div>
<div>-- configure ironic to use the pxe+ssh driver</div>
<div>- a new test job that turns all this on, thus allowing
tempest to do all its usual work against a "virtual
baremetal" cloud</div>
<div><br>
</div>
<div>Also, it's worth mentioning, the above-described plan
won't exercise the CRUD of Ironic resources -- I think
they need to be pre-enrolled with Ironic before tempest
starts (maybe not? maybe tempest can enroll them, instead
of devstack?). This is one reason why we have proposed
separate tempest tests for exercising Ironic's API.
Another reason is, testing our API is a valuable thing all
by itself, and has a much lower development cost than all
the changes above.</div>
<div><br>
</div>
<div>-Devananda</div>
</div>
</div>
</div>
<br>
</blockquote>
Looking forward to it!<br>
<br>
-David<br>
<br>
</body>
</html>