Open Stack

On Mon, Mar 21, 2016, at 11:49 AM, Clark Boylan wrote:
> On Mon, Mar 21, 2016, at 11:08 AM, Clark Boylan wrote:
> > On Mon, Mar 21, 2016, at 09:32 AM, Armando M. wrote: 
> > > Do you have an a better insight of job runtimes vs jobs in other
> > > projects?
> > > Most of the time in the job runtime is actually spent setting the
> > > infrastructure up, and I am not sure we can do anything about it, unless
> > > we
> > > take this with Infra.
> > 
> > I haven't done a comparison yet buts lets break down the runtime of a
> > recent successful neutron full run against neutron master [0].
> 
> And now for some comparative data from the gate-tempest-dsvm-full job
> [0]. This job also ran against a master change that merged and ran in
> the same cloud and region as the neutron job.
> 
snip
> Generally each step of this job was quicker. There were big differences
> in devstack and tempest run time though. Is devstack much slower to
> setup neutron when compared to nova net? For tempest it looks like we
> run ~1510 tests against neutron and only ~1269 against nova net. This
> may account for the large difference there. I also recall that we run
> ipv6 tempest tests against neutron deployments that were inefficient and
> booted 2 qemu VMs per test (not sure if that is still the case but
> illustrates that the tests themselves may not be very quick in the
> neutron case).

Looking at the tempest slowest tests output for each of these jobs
(neutron and nova net) some tests line up really well across jobs and
others do not. In order to get a better handle on the runtime for
individual tests I have pushed https://review.openstack.org/295487 which
will run tempest serially reducing the competition for resources between
tests.

Hopefully the subunit logs generated by this change can provide more
insight into where we are losing time during the tempest test runs.

Clark