[openstack-dev] [ironic][qa] How will ironic tests run in tempest?

Devananda van der Veen devananda.vdv at gmail.com
Wed Dec 11 01:41:58 UTC 2013

 Tue, Dec 10, 2013 at 12:43 PM, David Kranz <dkranz at redhat.com> wrote:

>  On 12/09/2013 01:37 PM, Devananda van der Veen wrote:
>  On Fri, Dec 6, 2013 at 2:13 PM, Clark Boylan <clark.boylan at gmail.com>wrote:
>>  On Fri, Dec 6, 2013 at 1:53 PM, David Kranz <dkranz at redhat.com> wrote:
>> > It's great that tempest tests for ironic have been submitted! I was
>> > reviewing https://review.openstack.org/#/c/48109/ and noticed that the
>> tests
>> > do not actually run. They are skipped because baremetal is not enabled.
>> This
>> > is not terribly surprising but we have had a policy in tempest to only
>> merge
>> > code that has demonstrated that it works. For services that cannot run
>> in
>> > the single-vm environment of the upstream gate we said there could be a
>> > system running somewhere that would run them and report a result to
>> gerrit.
>> > Is there a plan for this, or to make an exception for ironic?
>> >
>> >  -David
>> >
>> > _______________________________________________
>> > OpenStack-dev mailing list
>> > OpenStack-dev at lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>  There is a change[0] to openstack-infra/config to add experimental
>> tempest jobs to test ironic. I think that change is close to being
>> ready, but I need to give it time for a proper review. Once in that
>> will allow you to test 48109 (in theory, not sure if all the bits will
>> just work). I don't think these tests fall under the cannot run in a
>> single vm environment umbrella, we should be able to test the
>> baremetal code via the pxe booting of VMs within the single VM
>> environment.
>> [0] https://review.openstack.org/#/c/53917/
>> Clark
>  We can test the ironic services, database, and the driver interfaces by
> using our "fake" driver within a single devstack VM today (I'm not sure the
> exercises for all of this have been written yet, but it's practical to test
> it). OTOH, I don't believe we can test a PXE deploy within a single VM
> today, and need to resume discussions with infra about this.
>  There are some other aspects of Ironic (IPMI, SOL access, any
> vendor-specific drivers) which we'll need real hardware to test because
> they can't effectively be virtualized. TripleO should cover some (much?) of
> those needs, once they are able to switch to using Ironic instead of
> nova-baremetal.
>  -Devananda
> So it seems that the code in the submitted tempest tests can run in a
> regular job if devstack is configured to enable ironic, but that this
> cannot be the default. So I propose that we create a regular
> devstack+ironic job that will run in the ironic and tempest gates, and run
> just the ironic tests. When third-party bare-metal results can be reported
> for ironic, tempest can then accept tests that require bare-metal.  Does
> any one have a problem with this approach?
>  -David
As I understand it, the infra/config patch which Clark already linked (
https://review.openstack.org/#/c/53917), which has gone through several
iterations, should be enabling Ironic within devstack -- and thus causing
tempest to run the relevant tests -- within the Ironic and Tempest check
and gate pipelines. This will exercise Ironic's API by performing CRUD
actions on resources. It doesn't do any more than that yet.

David, I'm not sure what you mean by "when third-party bare-metal results
can be reported for ironic" -- I don't see any reason why we couldn't
accept third-party smoke tests right now, except that none of the tempest
tests are written... Am I missing something?

In the longer term, we are planning to enable tempest testing of deployment
by ironic within devstack-gate as all the pieces come together. This will
take a fair bit more work / time, but I'm going to start nudging resources
in this direction very soon. In fact, we just talked about this in #infra
for a bit. Here's an attempt to summarize what came of it w.r.t. Ironic's
testing plans. We will need:

- some changes in devstack-gate to prepare a new environment by...
-- install sshd + firewall it to only allow connections from localhost
-- create a bunch of tiny qemu VMs (of configurable size and number)
- some changes in devstack to...
-- suck up a list of those VM's MAC addresses and enroll them in Ironic
-- configure nova to use ironic
-- configure ironic to use the pxe+ssh driver
- a new test job that turns all this on, thus allowing tempest to do all
its usual work against a "virtual baremetal" cloud

Also, it's worth mentioning, the above-described plan won't exercise the
CRUD of Ironic resources -- I think they need to be pre-enrolled with
Ironic before tempest starts (maybe not? maybe tempest can enroll them,
instead of devstack?). This is one reason why we have proposed separate
tempest tests for exercising Ironic's API. Another reason is, testing our
API is a valuable thing all by itself, and has a much lower development
cost than all the changes above.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131210/3ab84f5e/attachment.html>

More information about the OpenStack-dev mailing list