Open Stack

On Wed, Aug 13, 2014 at 03:23:55PM +1000, Ian Wienand wrote:
> Hi,
> 
> I'm looking at Sean's change to run more of tempest on F20 [1].  To
> test it I'm using [2] -- it has shown is that tempest on F20 is *way*
> slower than on Ubuntu, to the point it times out.
> 
> I compared logs from [3] & [4] to come up with a comparison [5],
> e.g. SecurityGroupsTestXML.test_server_security_groups goes from 28
> seconds to 315 seconds (interestingly, it also showed up a bug that
> Ubuntu isn't preparing the images for boto tests [6]).
> 
> I feel this has got to be due to some combination of nested virt v
> binary translation.  Before I waste too much time, I thought I'd check
> with the experts for any thoughts?  Is this a known issue?

I'm not sure I'd agree with the idea that the nested virt / binary
translation is relevant. Presumably the Fedora and Ubuntu VMs are
running on equivalent cloud infrastructure, so should exhibit the
same approximate performance from the hypervisor. I'd be more
inclined to look at differences in the software stack or setup
between the two. As an example, Fedora is possibly exercising the
libguestfs codepath for file injection while Ubuntu probably does
the nbd/losetup codepath for file injection. The first run of
libguestfs is always slow due to the need to build its appliance.
This is cached thereafter in normal use, but since VMs for the
gate are throw-away, the first VM spawn will always be slow with
libguestfs.

I'd suggest analysing the nova compute log file to try and identify
which stage of the test run is showing the delay. It might let you
narrow it down to a specific part of VM spawn process for example.

> [1] https://review.openstack.org/112670
> [2] https://review.openstack.org/112739
> [3] http://logs.openstack.org/39/112739/1/check/check-tempest-dsvm-f20/36f13b6/
> [4] http://logs.openstack.org/39/112739/1/check/check-tempest-dsvm-full/6a81223/
> [5] http://people.redhat.com/~iwienand/f-v-u-tempest.txt
> [6] https://review.openstack.org/113767

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|