Open Stack

On 12/10/2013 08:41 PM, Devananda van der Veen wrote:
>  Tue, Dec 10, 2013 at 12:43 PM, David Kranz <dkranz at redhat.com 
> <mailto:dkranz at redhat.com>> wrote:
>
>     On 12/09/2013 01:37 PM, Devananda van der Veen wrote:
>>     On Fri, Dec 6, 2013 at 2:13 PM, Clark Boylan
>>     <clark.boylan at gmail.com <mailto:clark.boylan at gmail.com>> wrote:
>>
>>         On Fri, Dec 6, 2013 at 1:53 PM, David Kranz
>>         <dkranz at redhat.com <mailto:dkranz at redhat.com>> wrote:
>>         > It's great that tempest tests for ironic have been
>>         submitted! I was
>>         > reviewing https://review.openstack.org/#/c/48109/ and
>>         noticed that the tests
>>         > do not actually run. They are skipped because baremetal is
>>         not enabled. This
>>         > is not terribly surprising but we have had a policy in
>>         tempest to only merge
>>         > code that has demonstrated that it works. For services that
>>         cannot run in
>>         > the single-vm environment of the upstream gate we said
>>         there could be a
>>         > system running somewhere that would run them and report a
>>         result to gerrit.
>>         > Is there a plan for this, or to make an exception for ironic?
>>         >
>>         >  -David
>>         >
>>         > _______________________________________________
>>         > OpenStack-dev mailing list
>>         > OpenStack-dev at lists.openstack.org
>>         <mailto:OpenStack-dev at lists.openstack.org>
>>         >
>>         http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>         There is a change[0] to openstack-infra/config to add
>>         experimental
>>         tempest jobs to test ironic. I think that change is close to
>>         being
>>         ready, but I need to give it time for a proper review. Once
>>         in that
>>         will allow you to test 48109 (in theory, not sure if all the
>>         bits will
>>         just work). I don't think these tests fall under the cannot
>>         run in a
>>         single vm environment umbrella, we should be able to test the
>>         baremetal code via the pxe booting of VMs within the single VM
>>         environment.
>>
>>         [0] https://review.openstack.org/#/c/53917/
>>
>>
>>         Clark
>>
>>
>>     We can test the ironic services, database, and the driver
>>     interfaces by using our "fake" driver within a single devstack VM
>>     today (I'm not sure the exercises for all of this have been
>>     written yet, but it's practical to test it). OTOH, I don't
>>     believe we can test a PXE deploy within a single VM today, and
>>     need to resume discussions with infra about this.
>>
>>     There are some other aspects of Ironic (IPMI, SOL access, any
>>     vendor-specific drivers) which we'll need real hardware to test
>>     because they can't effectively be virtualized. TripleO should
>>     cover some (much?) of those needs, once they are able to switch
>>     to using Ironic instead of nova-baremetal.
>>
>>     -Devananda
>     So it seems that the code in the submitted tempest tests can run
>     in a regular job if devstack is configured to enable ironic, but
>     that this cannot be the default. So I propose that we create a
>     regular devstack+ironic job that will run in the ironic and
>     tempest gates, and run just the ironic tests. When third-party
>     bare-metal results can be reported for ironic, tempest can then
>     accept tests that require bare-metal.  Does any one have a problem
>     with this approach?
>
>      -David
>
>
> As I understand it, the infra/config patch which Clark already linked 
> (https://review.openstack.org/#/c/53917), which has gone through 
> several iterations, should be enabling Ironic within devstack -- and 
> thus causing tempest to run the relevant tests -- within the Ironic 
> and Tempest check and gate pipelines. This will exercise Ironic's API 
> by performing CRUD actions on resources. It doesn't do any more than 
> that yet.
It looks like that patch is adding ironic jobs to the experimental queue 
but I think we want them on check/gate.
>
> David, I'm not sure what you mean by "when third-party bare-metal 
> results can be reported for ironic" -- I don't see any reason why we 
> couldn't accept third-party smoke tests right now, except that none of 
> the tempest tests are written... Am I missing something?
I was assuming there were some ironic tests that actually need bare 
metal resources to run. Perhaps there are not. Either way, we just want 
to make sure that when tests are submitted to tempest we have evidence 
that they have successfully run. Sounds like the CRUD tests will just 
work the same way as our existing tests once ironic is enabled in devstack.
>
> In the longer term, we are planning to enable tempest testing of 
> deployment by ironic within devstack-gate as all the pieces come 
> together. This will take a fair bit more work / time, but I'm going to 
> start nudging resources in this direction very soon. In fact, we just 
> talked about this in #infra for a bit. Here's an attempt to summarize 
> what came of it w.r.t. Ironic's testing plans. We will need:
>
> - some changes in devstack-gate to prepare a new environment by...
> -- install sshd + firewall it to only allow connections from localhost
> -- create a bunch of tiny qemu VMs (of configurable size and number)
> - some changes in devstack to...
> -- suck up a list of those VM's MAC addresses and enroll them in Ironic
> -- configure nova to use ironic
> -- configure ironic to use the pxe+ssh driver
> - a new test job that turns all this on, thus allowing tempest to do 
> all its usual work against a "virtual baremetal" cloud
>
> Also, it's worth mentioning, the above-described plan won't exercise 
> the CRUD of Ironic resources -- I think they need to be pre-enrolled 
> with Ironic before tempest starts (maybe not? maybe tempest can enroll 
> them, instead of devstack?). This is one reason why we have proposed 
> separate tempest tests for exercising Ironic's API. Another reason is, 
> testing our API is a valuable thing all by itself, and has a much 
> lower development cost than all the changes above.
>
> -Devananda
>
Looking forward to it!

  -David

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131211/561dfc1f/attachment.html>