Open Stack

Dan Smith <dms at danplanet.com> wrote on 10/10/2013 08:26:14 PM:

> From: Dan Smith <dms at danplanet.com>
> To: OpenStack Development Mailing List 
<openstack-dev at lists.openstack.org>, 
> Date: 10/10/2013 08:31 PM
> Subject: Re: [openstack-dev] [nova][powervm] my notes from the 
> meeting on powervm CI
> 
> > 4. What is the max amount of time for us to report test results?  Dan
> > didn't seem to think 48 hours would fly. :)
> 
> Honestly, I think that 12 hours during peak times is the upper limit of
> what could be considered useful. If it's longer than that, many patches
> could go into the tree without a vote, which defeats the point.

Yeah, I was just joking about the 48 hour thing, 12 hours seems excessive
but I guess that has happened when things are super backed up with gate
issues and rechecks.

Right now things take about 4 hours, with Tempest being around 1.5 hours
of that. The rest of the time is setup and install, which includes heat
and ceilometer. So I guess that raises another question, if we're really
setting this up right now because of nova, do we need to have heat and
ceilometer installed and configured in the initial delivery of this if
we're not going to run tempest tests against them (we don't right now)?

I think some aspect of the slow setup time is related to DB2 and how
the migrations perform with some of that, but the overall time is not
considerably different from when we were running this with MySQL so
I'm reluctant to blame it all on DB2.  I think some of our topology
could have something to do with it too since the IVM hypervisor is running
on a separate system and we are gated on how it's performing at any
given time.  I think that will be our biggest challenge for the scale
issues with community CI.

> 
> > 5. What are the minimum tests that need to run (excluding APIs that 
the
> > powervm driver doesn't currently support)?
> >         - smoke/gate/negative/whitebox/scenario/cli?  Right now we 
have
> > 1152 tempest tests running, those are only within api/scenario/cli and
> > we don't run everything.
> 
> I think that "a full run of tempest" should be required. That said, if
> there are things that the driver legitimately doesn't support, it makes
> sense to exclude those from the tempest run, otherwise it's not useful.
> 
> I think you should publish the tempest config (or config script, or
> patch, or whatever) that you're using so that we can see what it means
> in terms of the coverage you're providing.

Just to clarify, do you mean publish what we are using now or publish
once it's all working?  I can certainly attach our nose.cfg and
latest x-unit results xml file.

> 
> > 6. Network service? We're running with openvswitch 1.10 today so we
> > probably want to continue with that if possible.
> 
> Hmm, so that means neutron? AFAIK, not much of tempest runs with
> Nova/Neutron.
> 
> I kinda think that since nova-network is our default right now (for
> better or worse) that the run should include that mode, especially if
> using neutron excludes a large portion of the tests.
> 
> I think you said you're actually running a bunch of tempest right now,
> which conflicts with my understanding of neutron workiness. Can you 
clarify?

Correct, we're running with neutron using the ovs plugin. We basically 
have
the same issues that the neutron gate jobs have, which is related to 
concurrency
issues and tenant isolation (we're doing the same as devstack with neutron
in that we don't run tempest with tenant isolation).  We are running most
of the nova and most of the neutron API tests though (we don't have all
of the neutron-dependent scenario tests working though, probably more due
to incompetence in setting up neutron than anything else).

> 
> > 7. Cinder backend? We're running with the storwize driver but we do we
> > do about the remote v7000?
> 
> Is there any reason not to just run with a local LVM setup like we do in
> the real gate? I mean, additional coverage for the v7000 driver is
> great, but if it breaks and causes you to not have any coverage at all,
> that seems, like, bad to me :)

Yeah, I think we'd just run with a local LVM setup, that's what we do for
x86_64 and s390x tempest runs. For whatever reason we thought we'd do
storwize for our ppc64 runs, probably just to have a matrix of coverage.

> 
> > Again, just getting some thoughts out there to help us figure out our
> > goals for this, especially around 4 and 5.
> 
> Yeah, thanks for starting this discussion!
> 
> --Dan
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131010/c23b0483/attachment.html>