Open Stack

Always having ephemeral disks as raw might make sense given that there is going to be little overlap with others (and therefore the qcow2 benefits are not significant)

I guess this could also be a simple nova.conf flag per hypervisor with low risk (a default ephemeral format to qcow2 or raw).

The l2 cache size option would still interesting for images.

Tim



On 23/10/15 19:36, "Marc Heckmann" <marc.heckmann at ubisoft.com> wrote:

>Hi Daniel,
>
>On Fri, 2015-10-23 at 12:05 +0100, Daniel P. Berrange wrote:
>> On Thu, Oct 22, 2015 at 06:01:42PM +0000, Marc Heckmann wrote:
>> > Hi,
>> > 
>> > On Thu, 2015-10-22 at 08:17 -0700, Abel Lopez wrote:
>> > > I've actually looked for this for our RBD backed ephemeral instances,
>> > > but found the options lacking. I last looked in Juno. 
>> > > 
>> > > On Thursday, October 22, 2015, Tim Bell <Tim.Bell at cern.ch> wrote:
>> > >          
>> > >         
>> > >         Has anyone had experience with setting up Nova with KVM so it
>> > >         has raw ephemeral disks but qcow2 images for the VMs ? We’ve
>> > >         got very large ephemeral disks and could benefit from the
>> > >         performance of raw volumes for this.
>> > 
>> > We looked into this for the very same reasons and it doesn't seem to be
>> > supported.
>> > 
>> > That being said, I'm fearful of the boot time performance impact of
>> > using RAW for ephemeral.
>> 
>> There should be no performance impact of using fully pre-allocated
>> raw images. Any decent modern filesystem (ext4, xfs) supports fallocate
>> which allows you to pre-allocate an arbitrary sizes plain file in
>> constant time.
>
>I was actually more afraid of the time it would take mkfs.<ext4|ntfs> to
>create the filesystem. I was thinking that it might be non-negligible on
>larger ephemeral sizes (approaching 1TB) going up to a minute or more.
>
>I just tested this and it doesn't seem to be an issue at all (obviously
>--fast is used for NTFS).
>
>This would seem to make a setting to use raw only for ephemeral slightly
>more attractive.
>
>That being said, if Nova gets fixed to pre-initialize qcow2 internally,
>then the raison-d'être for raw might become less important. I'm assuming
>the Nova fix for qcow2 would be easier to implement? Any downsides to
>pre-initialized qcow2?
>
>In any case, thanks for the useful info!
>
>> 
>> By default Nova does *not* preallocate images - so both raw & qcow2
>> will grow-on-demand as guest writes to sectors.
>> 
>> If the "preallocate_images" nova.conf option is set to "space", then
>> Nova will call fallocate for both raw & qcow2 images to fully allocate
>> the maximum space they require. There's no appreciable time overhead
>> for this - it just prevents you overcommitting disk obviously.
>> 
>> If you fully preallocate a qcow2 image its performance should pretty
>> much match raw images (modulo the l2-cache-size item mentioned
>> below), unfortunately, the way Nova is preallocating qcow2 images is
>> wrong - it preallocates the space on disk, but does not pre-initialize
>> the internal qcow2 data structures to match :-( So we need to fix
>> that for qcow2 in Nova.
>> 
>> > I suggest you check out the following presentation about qcow2
>> > performance if you haven't already done so:
>> > 
>> > http://events.linuxfoundation.org/sites/events/files/slides/p0.pp_.pdf
>> > 
>> > I think it would be worthwhile for Openstack (and libvirt if required)
>> > to support the "l2-cache-size" option for qcow2.
>> 
>> Yep, we should look at supporting that.
>> 
>> 
>> Regards,
>> Daniel
>
>
>_______________________________________________
>OpenStack-operators mailing list
>OpenStack-operators at lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators