Open Stack

On Wed, 5 Jun 2013, Day, Phil wrote:

> I guess the other way to look at this is that users would have a choice between a possibly faster start-up time by keeping to a few well know root disc sizes in the images or a fully custom root disk size but with less chance of a faster start-up time.

I dont understand why thats the choice.  Why does ephemeral disk size have
to be linked to root disk size?

>
> From the host perspective it has to clean up base files that are no longer in use anyway, so I don't see a big penalty in having a lot of different base files, just a lack of optimization in the user experience - or am I missing something ?
>

The time and IO it takes to create a filesystem on a "device" (file or
block device, doens't really matter) is very real.  I think the caching of
that is not an insignificant performance improvement.

Heres an example:
$ truncate --size=20G my.img
$ ls -s my.img
0 my.img
$ time mkfs.ext3 -F my.img >/dev/null 2>&1
real  0m15.279s
user  0m0.020s
sys   0m0.980s
$ ls -s my.img
464476 my.img

So it looks to me that it did ~400M of IO in order to put a filesystem on
a 20G image.  If you push that off to the guest, its only going to perform
worse (as IO will be slower).

I understand its a very remedial benchmark.

deciding to not create a filesystem on the instance is really not an
option in my opinion.  You'll break *loads* of workloads that expect to
come up and have /mnt mounted with whatever ephemeral disk is available.

Generally, I would argue on the side of "the cloud platform shouldn't deal
with software things like filesystems".  But here, there is such a large
expectation of ephemeral disk to have a filesystem already on it, that I
think its worth continuing.

openstack needs to be very careful about breaking an instances' expectation.

> Phil
>
> -----Original Message-----
> From: Day, Phil
> Sent: 05 June 2013 17:43
> To: OpenStack Development Mailing List
> Subject: Re: [openstack-dev] Feedback wanted please on proposal to make root disk size variable
>
> Hi Scott,
>
> I take your point about the number of backing files - which could in the worst case end up as one for every instance on the host.
>
> I use the term worst case loosely though, as there are times where
> having a filesystem in the base file can be a pain - for example if you
> have two VMs on the same system that are expecting the uuids of their
> filesystems to always be unique then they'll be in for a shock, and also

I dont think thats a real issue, honestly.  Any one working with disk
images realizes that clones are ... well they're clones.  They've seen the
same UUIDs on different disks before.

> on migration to a host that already has a backing file of that size then
> you either have to squash the COW layer of the VM you’re migrating ,or
> re-base it and accept that the filesystem uuid is going to change.

You most certainly can't change the contents of the ephemeral image during
a live migration.  That'd quite likely crash a kernel and best case would
lose customer data.
>
> I have been wondering if the common backing files are really that much of a gain ?

I suspect the're not really much of a gain, as any usage in one instance
is almost certainly different than another instance.  However, as shown
above, there probably is 400M of IO gained by the cache and re-use.