[Openstack] New instances booting time

Ivan Derbenev ivan.derbenev at tech-corps.com
Fri Aug 14 20:29:43 UTC 2015


Well, thanks. 
Actually, wanted to know, if this behavior is ok for every installation, or I has misconfigured smthing

Regards, 
IT engineer
Farheap, Russia
Ivan Derbenev

-----Original Message-----
From: Daniel P. Berrange [mailto:berrange at redhat.com] 
Sent: Friday, August 14, 2015 5:56 PM
To: Jay Pipes <jaypipes at gmail.com>
Cc: openstack at lists.openstack.org
Subject: Re: [Openstack] New instances booting time

On Fri, Aug 14, 2015 at 03:34:09PM +0100, Daniel P. Berrange wrote:
> On Fri, Aug 14, 2015 at 03:10:13PM +0100, Daniel P. Berrange wrote:
> > On Fri, Aug 14, 2015 at 09:55:42AM -0400, Jay Pipes wrote:
> > > On 08/13/2015 11:37 PM, Ivan Derbenev wrote:
> > > >*From:*Ivan Derbenev [mailto:ivan.derbenev at tech-corps.com]
> > > >*Sent:* Wednesday, August 5, 2015 1:21 PM
> > > >*To:* openstack at lists.openstack.org
> > > >*Subject:* [Openstack] New instances booting time
> > > >
> > > >Hello, guys, I have a question
> > > >
> > > >We now have OS Kilo + KVM+ Ubuntu 14.04
> > > >
> > > >Nova-compute.conf:
> > > >
> > > >[libvirt]
> > > >
> > > >virt_type=kvm
> > > >
> > > >images_type = lvm
> > > >
> > > >images_volume_group =openstack-controller01-ky01-vg
> > > >
> > > >volume_clear = none
> > > >
> > > >the problem is when nova boots a new instance it’s SUPER slow
> > > >
> > > >exactly this step:
> > > >
> > > >Running cmd (subprocess): sudo nova-rootwrap 
> > > >/etc/nova/rootwrap.conf qemu-img convert -O raw 
> > > >/var/lib/nova/instances/_base/999f7fff2521e4a7243c9e1d21599fd64a1
> > > >9b42e 
> > > >/dev/openstack-controller01-ky01-vg/5f831046-435c-4636-8b71-a6623
> > > >27b608c_disk
> > > >
> > > >Well, I understand what this step is doing – it’s copying raw image to lvm.
> > > >
> > > >How can we speed it up?
> > > >
> > > >I don’t wanna have instance from 100GB image booted for 4 hours
> > > 
> > > Don't use base images that are 100G in size. Quite simple, really.
> > 
> > Presumably the actual image is only a few 100 MB in size, but the 
> > virtual qcow2 size is 100 GB ?
> > 
> > I guess qemu-img probably has to still write out 100 GB of zeros as 
> > it can't assume that the LVM target is pre-zeroed. I do wonder 
> > though if there's a way to optimize this so that qemu-img only has 
> > to write out the 100 MB of actual data, and not al the zeros.
> > 
> > Perhaps there's scope for a flag to qemu-img to tell it to seek in 
> > the target when it sees holes in the source image. If OpenStack then 
> > used sparse LVM images, there would be no need to rwite out 100's of 
> > GB of zeros.
> 
> Speaking with the KVM block layer maintainers, the behaviour seen is 
> not entirely expected. qemu-img wants to ensure no existing data is 
> left behind as that could cause data corruption if the original qcow2 
> file had dropped empty sectors. So it has to make sure it can write 
> out the 100 GB even if that is mostly zeros.
> 
> That said, it has support for using discard, and it will use that in 
> preference to writing zeros, which should be really fast. For it to 
> work with block devices though, it seems that we need to disable the 
> I/O cache by passing '-t none' to qemu-img when doing this conversion. 
> This also assumes the underlying storage that the user has configured 
> behind their LVM volume supports discard which may not always be the 
> case.
> 
> So it would be interesting to know if changing
> 
> $ qemu-img convert -O raw \
>     /var/lib/nova/instances/_base/999f7fff2521e4a7243c9e1d21599fd64a19b42e \
>     
> /dev/openstack-controller01-ky01-vg/5f831046-435c-4636-8b71-a662327b60
> 8c_disk
> 
> to
> 
> $ qemu-img convert -O raw -t none \
>     /var/lib/nova/instances/_base/999f7fff2521e4a7243c9e1d21599fd64a19b42e \
>     
> /dev/openstack-controller01-ky01-vg/5f831046-435c-4636-8b71-a662327b60
> 8c_disk
> 
> makes any significant difference to the speed.

I filed a bug to track this possible enhancement

  https://bugs.launchpad.net/nova/+bug/1484992

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to     : openstack at lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack


More information about the Openstack mailing list