Open Stack

Sent from my iPad

On Mar 16, 2012, at 12:26, "Pádraig Brady" <P at draigBrady.com> wrote:

> On 03/16/2012 04:11 PM, Jay Pipes wrote:
>> Hi Stackers,
>> 
>> So, in diagnosing a few things on TryStack yesterday, I ran into an interesting problem with snapshotting that I'm hoping to get some advice on.
>> 
>> == The Problem ==
>> 
> 
>> QEMU was unhelpfully returning a vague error message of "error while writing".
> 
> That could be improved.
> As an aside, since qemu-img is mainly dealing with large files,
> it would be a prime candidate to call fallocate() from
> to get good layout for the files and immediate feedback
> if there isn't enough space.
> 
> On a related note, I've a patch pending for after RC1
> that should auto clean any of these partially written files:
> https://review.openstack.org/#change,5442
> 
>> As it turns out, the base operating system we install on our compute nodes in TryStack has a (very) small root partition
> 
>> == Possible Solutions ==
>> 
>> So, there are a number of solutions that we can work on here, and I'm wondering what the preference would be. Here are the solutions I have come up with, along with a no-brainer improvement to Nova that would help in diagnosing this problem:
>> 
>> The no-brainer: Detect before attempting a snapshot that there is enough space on a device to perform the operation, and if not, throw a useful error message up the stack
> 
> The space can change while writing, so you could still get the same error above.
> 
>> 
>> Solutions to the disk space problem:
>> 
>> (1) Silly Jay, change the damn size of the root partition in your PXE base OS install!
>> 
>> Now, I'm no expert in creating customized base disk images, but from looking at the build_pxe_env.sh script in devstack [1], it seems pretty trivial to change the ramdisk_size parameter in the startup options to something larger than 2109600. We could do this and reimage the compute nodes one by one.
>> 
>> (2) Make the location in which the snapshot is made configurable.
>> 
>> Right now, as mentioned above, tempfile.mkdtemp() is used, which creates a directory in the user's TMPDIR (typically /tmp, which is usually on the root partition).
>> 
>> We could add an option (--libvirt-snapshot-dir?) that would allow nova-compute to override where that snapshot is built.
>> 
>> (3) Change the user (running nova-compute) TMPDIR setting to something different than /tmp on the root partition).
> 
> I'd lean towards (3).
> That's something that depends on the environment (as you've nicely demonstrated),
> and also for security reasons the admin should be able to set TMPDIR.
> That's the standard way to do it, and it works already (hopefully).

Actually I would argue that the best way to accomplish this would be option #2. That way an admin/operator has control over the location. Not manipulating this by messing around with a users environment variable.

> 
> cheers,
> Pádraig.
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack at lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp