I had a couple different ideas on how to approach this given the constraints:<div><br></div><div>1. Keep the snapshots around until the next operation which restarts the VM process. This code would run prior to (re)starting a Qemu/KVM process such as hard reboot, resume on host boot, another snapshot, etc. It would look at all existing snapshots in a given qcow2 file and remove them prior to executing said operation. This would limit the number of snapshots in the image itself to 1, which I would take as a fair trade off to make snapshots only minimally disruptive.</div>
<div><br></div><div>2. Suspend the VM twice during the snapshot process. Once to take the snapshot and a second time to delete it. This is a simple change and I already have the code done but think #1 would be a cleaner approach.<span></span></div>
<div><br></div><div>Or, we could support both via configuration options. I looked at the possibility of executing snapshots via libvirt but realized it only recently became available and the documentation around the API hook is a bit sparse. So it is probably not a feasible solution at this time, mainly due do the former.</div>
<div><span class="Apple-style-span" style><br></span></div><div>If <span class="Apple-style-span" style>anyone else has another idea, that would be great to hear as well.</span><br></div><div></div><div><br></div><div>Rafi<br>
<br><br>On Thursday, August 23, 2012, Vishvananda Ishaya wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">We discussed this in the mailing list in the past<br>
<br>
quoting daniel:<br>
<br>
> a) is it safe to use qemu-img to create/delete a snapshot in a disk file that libvirt is writing to.<br>
> if not:<br>
> b) is it safe to use qemu-img to delete a snapshot in a disk file that libvirt is writing to but not actively using.<br>
> if not:<br>
> c) is it safe to use qemu-img to create/delete a snapshot in a disk file that libvirt has an open file handle to.<br>
<br>
Sadly, the answer is no to all those questions. For Qcow2 files, using<br>
internal snapshots, you cannot make *any* changes to the qcow2 file,<br>
while QEMU has it open. The reasons are that QEMU may have metadata<br>
changes pending to the file which have not yet flushed to disk, and<br>
second, creating/deleteing the snapshot with qemu-img may cause<br>
metadat changes that QEMU won't be aware of. Either way you will likely<br>
cause corruption of the qcow2 file.<br>
<br>
For these reasons, QEMU provides monitor commands for snapshotting,<br>
that libvirt uses whenever the guest is running. Libvirt will only<br>
use qemu-img, if the the guest is offline.<br>
<br>
Regards,<br>
Daniel<br>
<br>
<br>
So we unfortunately cannot delete the snapshot while the domain is running. Unless we are willing to leave a bunch of old internal snapshots in the file then we have to deal with this performance hit.<br>
Vish<br>
<br>
On Aug 23, 2012, at 5:27 PM, Rafi Khardalian <<a>rafi@metacloud.com</a>> wrote:<br>
<br>
> Assuming there are reasons for keeping suspend part of the snapshot<br>
> process, the flow can be optimized to reduce the impact to running VMs.<br>
> This is done by resuming immediately after the "qemu-img snapshot"<br>
> operation (libvirt_utils.create_snapshot), rather than waiting until the<br>
> "qemu-img convert" process (libvirt_utils.extract_snapshot) also<br>
> completes. I've been unable to find a reason for waiting until the<br>
> convert is done.<br>
><br>
> Modified snippet snapshot() snippet from the libvirt driver, representing<br>
> the change I'm proposing:<br>
><br>
> # Make the snapshot<br>
> try:<br>
> libvirt_utils.create_snapshot(disk_path, snapshot_name)<br>
> finally:<br>
> if state == power_state.RUNNING:<br>
> self._create_new_domain(xml_desc)<br>
><br>
> # Export the snapshot to a raw image<br>
> with utils.tempdir() as tmpdir:<br>
> try:<br>
> out_path = os.path.join(tmpdir, snapshot_name)<br>
> libvirt_utils.extract_snapshot(disk_path, source_format,<br>
> snapshot_name, out_path,<br>
> image_format)<br>
> finally:<br>
> libvirt_utils.delete_snapshot(disk_path, snapshot_name)<br>
><br>
> I agree it would be ideal if we could find a way to guarantee a consistent<br>
> state in the guest VM, though I'm concerned about how users would respond<br>
> to a full shutdown being forced upon them to take a snapshot.<br>
><br>
> -----Original Message-----<br>
> From: Joshua Harlow [mailto:<a>harlowja@yahoo-inc.com</a>]<br>
> Sent: Thursday, August 23, 2012 5:13 PM<br>
> To: OpenStack Development Mailing List; Rafi Khardalian<br>
> Cc: openstack-dev<br>
> Subject: Re: [openstack-dev] Libvirt snapshot process optimization<br>
><br>
> I'd almost like to see the VM be shutdown before snapshot, but that零 just<br>
> me.<br>
><br>
> In fact just looking at the libvirt docs, 'suspend does not save a<br>
> persistent image of the guest's memory. For this, save is used.' So that<br>
> could leave guests in some weird state, so that sort of sucks. A shutdown<br>
> could at least trigger ACPI shutdown to occur in the VM and would<br>
> hopefully leave it in a ok state (emphasis on hopefully). I just think<br>
> that reducing the amount of time is going to be hard without<br>
> hypervisor<->vm communication (ie signaling all the apps in the vm to<br>
> stop) or libvirt (+others) needs to persist the memory image.<br>
><br>
> My guess is suspend is trying to do what it can, which won't be 100% right<br>
> without memory state saving or some other communication happening...<br>
> Perhaps a 'save' call (or shutdown sequence) should be used, but this<br>
> probably isn't any faster, but at least it would be 'correct' (shared<br>
> storage state not included). There is also the question of uploading<br>
> snapshots (but that零 a different question).<br>
><br>
> On 8/23/12 2:00 PM, "Rafi Khardalian" <<a>rafi@metacloud.com</a>> wrote:<br>
><br>
>> Hi all,<br>
>><br>
>> I'm looking at the libvirt snapshot code and was wondering about the<br>
> order<br>
>> and purpose of several operations. At a high level, it looks like the VM<br>
>> being snapshotted is first suspended (managedSave), actual qcow2 snapshot<br>
>> is taken, then extraction is done (qemu-img convert) before returning the<br>
>> instance to its prior state.<br>
>><br>
>> My question is, with snapshots being atomic, why suspend the VM?<br>
> Assuming<br>
>> there's a reason for this, why not do the qemu-img convert call after the<br>
>> VM </blockquote></div><br><br>-- <br><div>---</div><div>Rafi Khardalian</div><div>Vice President, Operations | Metacloud, Inc.</div><div>Email: <a href="mailto:rafi@metacloud.com" target="_blank">rafi@metacloud.com</a> | Tel: 855-638-2256, Ext. 2662</div>
<br>