Open Stack

Hi Daniel,

On Fri, 2015-10-23 at 12:05 +0100, Daniel P. Berrange wrote:
> On Thu, Oct 22, 2015 at 06:01:42PM +0000, Marc Heckmann wrote:
> > Hi,
> > 
> > On Thu, 2015-10-22 at 08:17 -0700, Abel Lopez wrote:
> > > I've actually looked for this for our RBD backed ephemeral instances,
> > > but found the options lacking. I last looked in Juno. 
> > > 
> > > On Thursday, October 22, 2015, Tim Bell <Tim.Bell at cern.ch> wrote:
> > >          
> > >         
> > >         Has anyone had experience with setting up Nova with KVM so it
> > >         has raw ephemeral disks but qcow2 images for the VMs ? We’ve
> > >         got very large ephemeral disks and could benefit from the
> > >         performance of raw volumes for this.
> > 
> > We looked into this for the very same reasons and it doesn't seem to be
> > supported.
> > 
> > That being said, I'm fearful of the boot time performance impact of
> > using RAW for ephemeral.
> 
> There should be no performance impact of using fully pre-allocated
> raw images. Any decent modern filesystem (ext4, xfs) supports fallocate
> which allows you to pre-allocate an arbitrary sizes plain file in
> constant time.

I was actually more afraid of the time it would take mkfs.<ext4|ntfs> to
create the filesystem. I was thinking that it might be non-negligible on
larger ephemeral sizes (approaching 1TB) going up to a minute or more.

I just tested this and it doesn't seem to be an issue at all (obviously
--fast is used for NTFS).

This would seem to make a setting to use raw only for ephemeral slightly
more attractive.

That being said, if Nova gets fixed to pre-initialize qcow2 internally,
then the raison-d'être for raw might become less important. I'm assuming
the Nova fix for qcow2 would be easier to implement? Any downsides to
pre-initialized qcow2?

In any case, thanks for the useful info!

> 
> By default Nova does *not* preallocate images - so both raw & qcow2
> will grow-on-demand as guest writes to sectors.
> 
> If the "preallocate_images" nova.conf option is set to "space", then
> Nova will call fallocate for both raw & qcow2 images to fully allocate
> the maximum space they require. There's no appreciable time overhead
> for this - it just prevents you overcommitting disk obviously.
> 
> If you fully preallocate a qcow2 image its performance should pretty
> much match raw images (modulo the l2-cache-size item mentioned
> below), unfortunately, the way Nova is preallocating qcow2 images is
> wrong - it preallocates the space on disk, but does not pre-initialize
> the internal qcow2 data structures to match :-( So we need to fix
> that for qcow2 in Nova.
> 
> > I suggest you check out the following presentation about qcow2
> > performance if you haven't already done so:
> > 
> > http://events.linuxfoundation.org/sites/events/files/slides/p0.pp_.pdf
> > 
> > I think it would be worthwhile for Openstack (and libvirt if required)
> > to support the "l2-cache-size" option for qcow2.
> 
> Yep, we should look at supporting that.
> 
> 
> Regards,
> Daniel