<div class="gmail_quote">On Fri, Nov 9, 2012 at 9:45 PM, Vishvananda Ishaya <span dir="ltr"><<a href="mailto:vishvananda@gmail.com" target="_blank">vishvananda@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">The libvirt driver has actually gotten quite good at rebuilding all of the data for instances. This only thing it can't do right now is redownload base images from glance. With current state if you simply back up the instances directory (usually /var/lib/nova/instances) then you can recover by bringing back the whole directory and doing a nova reboot <uuid> for each instance.</div>
</blockquote><div> </div><div>What about image corruption, If I start backing up '/var/lib/instances' when the instances are running? Should they be paused or suspended while doing that?</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word"><div><br></div><div>You could just stick the whole thing on an lvm and snaphot it regularly for dr. The _base directory can be regenerated with images from glance so you could also write a script to regenerate it and not have to worry about backing it up. The code to add to nova to make it automatically re-download the image from glance if it isn't there shouldn't be too bad either, which would mean you could safely ignore the _base directory for backups. Additionally using qcow images in glance and the config option `force_raw_images=False` will keep this directory much smaller.</div>
</div></blockquote><div>If I put everything on LVM, I will not be able to regular snapshots anymore. Is there a workaround to this. I want to get some understanding and confidence before I start rebuild my current setup.</div>
<div><br></div><div>With image regeneration you meant image download from glance ('glance image-download')?</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word"><div><br></div><div>Vish</div><div><br></div><div><br></div><div><div><div><div><div class="h5"><div>On Nov 9, 2012, at 2:51 AM, Jānis Ģeņģeris <<a href="mailto:janis.gengeris@gmail.com" target="_blank">janis.gengeris@gmail.com</a>> wrote:</div>
<br></div></div><blockquote type="cite"><div><div class="h5">Hello all,<div><br></div><div>I would like to know the available solutions that are used regarding to backing up and/or snapshotting running </div><div>instances on compute nodes. Documentation does not mention anything related to this. With snapshots I don't mean </div>
<div>the current snapshot mechanism, that imports image of the running VM into glance. I'm using KVM, but this is </div><div>significant for any hypervisor.</div><div><br></div><div>Why is this important?</div><div>Consider simple scenario when hardware on compute node fails and the node goes down immediately and is not recoverable </div>
<div>in reasonable time. The images of the running instances are also lost. Shared file system is not considered here as it </div><div>may cause IO bottlenecks and adds another layer of complexity.</div><div><br></div><div>
There have been a few discussions on the the list about this problem, but none have really answered the question.</div><div><br></div><div>The documentation speaks of disaster recovery when power loss have happened and failed compute node recovery from </div>
<div>shared file system. But don't cover the case without shared file system.</div><div><br></div><div>I can think of few solutions currently (for KVM):</div><div>a) using LVM images for VMs, and making LVM logical volume snapshots, but then the current nova snapshot mechanism</div>
<div>will not work (from the docs - '<span style="font-family:Verdana,Geneva,sans-serif;font-size:13px">current snapshot mechanism in OpenStack Compute works only with instances backed </span></div><div><span style="font-family:Verdana,Geneva,sans-serif;font-size:13px">with Qcow2 images');</span></div>
<div><span style="font-family:Verdana,Geneva,sans-serif;font-size:13px">b) snapshot machines with OpenStack snapshotting mechanism, but this doesn't fit somehow, because it has</span></div><div><span style="font-family:Verdana,Geneva,sans-serif;font-size:13px">other goal than creating backups, will be slow and pollute the glance image space;</span></div>
<div><br></div><div>Regards<br>--janis<br>
</div></div></div>
_______________________________________________<br>Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
</blockquote></div><br></div></div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>--janis<br>