[Cinder][nova] queens backup
Hi All, Please, I' d like to know if cinder backup and/or cinder snapshot call qemu guest agent for fsfreezing. If Yes, does it freeze file systems in the volume? Regards Ignazio
On Tue, Jan 22, 2019 at 07:04:05PM +0100, Ignazio Cassano wrote:
Hi All, Please, I' d like to know if cinder backup and/or cinder snapshot call qemu guest agent for fsfreezing. If Yes, does it freeze file systems in the volume? Regards Ignazio
Unfortunately no, initiating a snapshot or backup from Cinder does not call out to Nova to do any guest quiescing. There would need to be something else coordinating the calls between the guest OS and initiating the Cinder snapshot to do that. Sean
Thanks for the info. Ignazio Il giorno Mar 22 Gen 2019 19:45 Sean McGinnis <sean.mcginnis@gmx.com> ha scritto:
On Tue, Jan 22, 2019 at 07:04:05PM +0100, Ignazio Cassano wrote:
Hi All, Please, I' d like to know if cinder backup and/or cinder snapshot call qemu guest agent for fsfreezing. If Yes, does it freeze file systems in the volume? Regards Ignazio
Unfortunately no, initiating a snapshot or backup from Cinder does not call out to Nova to do any guest quiescing. There would need to be something else coordinating the calls between the guest OS and initiating the Cinder snapshot to do that.
Sean
On 1/22/2019 12:45 PM, Sean McGinnis wrote:
On Tue, Jan 22, 2019 at 07:04:05PM +0100, Ignazio Cassano wrote:
Hi All, Please, I' d like to know if cinder backup and/or cinder snapshot call qemu guest agent for fsfreezing. If Yes, does it freeze file systems in the volume? Regards Ignazio Unfortunately no, initiating a snapshot or backup from Cinder does not call out to Nova to do any guest quiescing. There would need to be something else coordinating the calls between the guest OS and initiating the Cinder snapshot to do that.
Sean
If you snapshot a volume-backed server in nova the compute API will attempt to quiesce the guest before creating the snapshot of the volume: https://github.com/openstack/nova/blob/31956108e6e785407bdcc31dbc8ba99e6a28c... As for backups, the compute createBackup API does not support volume-backed servers: https://github.com/openstack/nova/blob/31956108e6e785407bdcc31dbc8ba99e6a28c... -- Thanks, Matt
On Tue, 22 Jan 2019 at 18:51, Sean McGinnis <sean.mcginnis@gmx.com> wrote:
On Tue, Jan 22, 2019 at 07:04:05PM +0100, Ignazio Cassano wrote:
Hi All, Please, I' d like to know if cinder backup and/or cinder snapshot call qemu guest agent for fsfreezing. If Yes, does it freeze file systems in the volume? Regards Ignazio
Unfortunately no, initiating a snapshot or backup from Cinder does not call out to Nova to do any guest quiescing. There would need to be something else coordinating the calls between the guest OS and initiating the Cinder snapshot to do that.
Although it wouldn't help making a consistent snapshot of an instance with multiple disks, for a single disk it shouldn't matter if the guest is quiesced as long as the backend can make an instantaneous snapshot. I'm pretty sure many (most?) backends would support that; certainly plain old LVM does. Does cinder use this functionality where it's available, and would that solve the problem you're trying to address? Matt
Sean
-- Matthew Booth Red Hat OpenStack Engineer, Compute DFG Phone: +442070094448 (UK)
Hello Matthew, our backend is netapp fas8040 via nfs and our instances have often more than one disk. Thanks Ignazio Il giorno mer 23 gen 2019 alle ore 15:19 Matthew Booth <mbooth@redhat.com> ha scritto:
On Tue, 22 Jan 2019 at 18:51, Sean McGinnis <sean.mcginnis@gmx.com> wrote:
On Tue, Jan 22, 2019 at 07:04:05PM +0100, Ignazio Cassano wrote:
Hi All, Please, I' d like to know if cinder backup and/or cinder snapshot
call
qemu guest agent for fsfreezing. If Yes, does it freeze file systems in the volume? Regards Ignazio
Unfortunately no, initiating a snapshot or backup from Cinder does not call out to Nova to do any guest quiescing. There would need to be something else coordinating the calls between the guest OS and initiating the Cinder snapshot to do that.
Although it wouldn't help making a consistent snapshot of an instance with multiple disks, for a single disk it shouldn't matter if the guest is quiesced as long as the backend can make an instantaneous snapshot. I'm pretty sure many (most?) backends would support that; certainly plain old LVM does. Does cinder use this functionality where it's available, and would that solve the problem you're trying to address?
Matt
Sean
-- Matthew Booth Red Hat OpenStack Engineer, Compute DFG
Phone: +442070094448 (UK)
Although it wouldn't help making a consistent snapshot of an instance with multiple disks, for a single disk it shouldn't matter if the guest is quiesced as long as the backend can make an instantaneous snapshot. I'm pretty sure many (most?) backends would support that; certainly plain old LVM does. Does cinder use this functionality where it's available, and would that solve the problem you're trying to address?
Matt
That is a good point. If you snap all of the volumes used, it may not be quiesced and have all IO flushed, but it would at least be a crash consistent set of data. All of the volumes used by a VM would need to be added to a group: https://docs.openstack.org/cinder/latest/admin/blockstorage-groups.html You would then be able to create a group snapshot to have all volumes snapped together.
Hello, I did not understand if you mean cinder snapshot pr netapp snapshot. Any case, why, we do not need to quiesce the instance ? Regards Ignazio Il giorno mer 23 gen 2019 alle ore 16:49 Sean McGinnis < sean.mcginnis@gmx.com> ha scritto:
Although it wouldn't help making a consistent snapshot of an instance with multiple disks, for a single disk it shouldn't matter if the guest is quiesced as long as the backend can make an instantaneous snapshot. I'm pretty sure many (most?) backends would support that; certainly plain old LVM does. Does cinder use this functionality where it's available, and would that solve the problem you're trying to address?
Matt
That is a good point. If you snap all of the volumes used, it may not be quiesced and have all IO flushed, but it would at least be a crash consistent set of data.
All of the volumes used by a VM would need to be added to a group:
https://docs.openstack.org/cinder/latest/admin/blockstorage-groups.html
You would then be able to create a group snapshot to have all volumes snapped together.
On Wed, Jan 23, 2019 at 04:56:24PM +0100, Ignazio Cassano wrote:
Hello, I did not understand if you mean cinder snapshot pr netapp snapshot. Any case, why, we do not need to quiesce the instance ? Regards Ignazio
If being crash consistent is good enough for your needs, then you don't. I know some do prefer the coordinated quiescing of IO in the instance to make sure any in-flight transactions are flushed out and application data is more likely to be in a good consistent state. Depending on your application running in the instance, things like databases are pretty good at rolling back incomplete transactions, so it's just a matter of whether you can allow the possibility that something that was successful in the milliseconds before the snap was created to now be rolled back when the application restarts.
Manu thanks. I read a blueprint for providing quiesce function to nova api but I cannot find it. Must I talk directly with libvirt api? Ignazio Il giorno Mer 23 Gen 2019 17:01 Sean McGinnis <sean.mcginnis@gmx.com> ha scritto:
On Wed, Jan 23, 2019 at 04:56:24PM +0100, Ignazio Cassano wrote:
Hello, I did not understand if you mean cinder snapshot pr netapp snapshot. Any case, why, we do not need to quiesce the instance ? Regards Ignazio
If being crash consistent is good enough for your needs, then you don't. I know some do prefer the coordinated quiescing of IO in the instance to make sure any in-flight transactions are flushed out and application data is more likely to be in a good consistent state.
Depending on your application running in the instance, things like databases are pretty good at rolling back incomplete transactions, so it's just a matter of whether you can allow the possibility that something that was successful in the milliseconds before the snap was created to now be rolled back when the application restarts.
On Wed, 23 Jan 2019 18:02:56 +0100, Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Manu thanks. I read a blueprint for providing quiesce function to nova api but I cannot find it. Must I talk directly with libvirt api?
Quiesce was never added to the nova API as a separate function and a spec proposal to add it was last reviewed in Newton [1]. At the time of review, only one virt driver, libvirt, supported quiesce and the justification to add a new REST API that all but one driver could not support, was not compelling enough. AFAIK the libvirt driver is still the only one that supports quiesce. There were other concerns beyond that though, and they are detailed in the review. As Matt Riedemann mentioned in his earlier reply on this thread [2], a quiesce step is integrated into the nova snapshot API, if the driver supports it (only libvirt). This is the only way you can quiesce an instance today. Cheers, -melanie [1] https://review.openstack.org/295595 [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-January/001984.h...
Il giorno Mer 23 Gen 2019 17:01 Sean McGinnis <sean.mcginnis@gmx.com <mailto:sean.mcginnis@gmx.com>> ha scritto:
On Wed, Jan 23, 2019 at 04:56:24PM +0100, Ignazio Cassano wrote: > Hello, I did not understand if you mean cinder snapshot pr netapp snapshot. > Any case, why, we do not need to quiesce the instance ? > Regards > Ignazio >
If being crash consistent is good enough for your needs, then you don't. I know some do prefer the coordinated quiescing of IO in the instance to make sure any in-flight transactions are flushed out and application data is more likely to be in a good consistent state.
Depending on your application running in the instance, things like databases are pretty good at rolling back incomplete transactions, so it's just a matter of whether you can allow the possibility that something that was successful in the milliseconds before the snap was created to now be rolled back when the application restarts.
Thanks for the info. Ignazio Il giorno Mer 23 Gen 2019 19:29 melanie witt <melwittt@gmail.com> ha scritto:
On Wed, 23 Jan 2019 18:02:56 +0100, Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Manu thanks. I read a blueprint for providing quiesce function to nova api but I cannot find it. Must I talk directly with libvirt api?
Quiesce was never added to the nova API as a separate function and a spec proposal to add it was last reviewed in Newton [1]. At the time of review, only one virt driver, libvirt, supported quiesce and the justification to add a new REST API that all but one driver could not support, was not compelling enough. AFAIK the libvirt driver is still the only one that supports quiesce. There were other concerns beyond that though, and they are detailed in the review.
As Matt Riedemann mentioned in his earlier reply on this thread [2], a quiesce step is integrated into the nova snapshot API, if the driver supports it (only libvirt). This is the only way you can quiesce an instance today.
Cheers, -melanie
[1] https://review.openstack.org/295595 [2]
http://lists.openstack.org/pipermail/openstack-discuss/2019-January/001984.h...
Il giorno Mer 23 Gen 2019 17:01 Sean McGinnis <sean.mcginnis@gmx.com <mailto:sean.mcginnis@gmx.com>> ha scritto:
On Wed, Jan 23, 2019 at 04:56:24PM +0100, Ignazio Cassano wrote: > Hello, I did not understand if you mean cinder snapshot pr netapp snapshot. > Any case, why, we do not need to quiesce the instance ? > Regards > Ignazio >
If being crash consistent is good enough for your needs, then you don't. I know some do prefer the coordinated quiescing of IO in the instance to make sure any in-flight transactions are flushed out and application data is more likely to be in a good consistent state.
Depending on your application running in the instance, things like databases are pretty good at rolling back incomplete transactions, so it's just a matter of whether you can allow the possibility that something that was successful in the milliseconds before the snap was created to now be rolled back when the application restarts.
Thanks for the info. Ignazio
Il giorno Mer 23 Gen 2019 19:29 melanie witt <melwittt@gmail.com> ha scritto:
On Wed, 23 Jan 2019 18:02:56 +0100, Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Manu thanks. I read a blueprint for providing quiesce function to nova api but I cannot find it. Must I talk directly with libvirt api?
Quiesce was never added to the nova API as a separate function and a spec proposal to add it was last reviewed in Newton [1]. At the time of review, only one virt driver, libvirt, supported quiesce and the justification to add a new REST API that all but one driver could not support, was not compelling enough. AFAIK the libvirt driver is still the only one that supports quiesce. There were other concerns beyond that though, and they are detailed in the review.
As Matt Riedemann mentioned in his earlier reply on this thread [2], a quiesce step is integrated into the nova snapshot API, if the driver supports it (only libvirt). This is the only way you can quiesce an instance today.
On Wed, 2019-01-23 at 19:34 +0100, Ignazio Cassano wrote: the closest semi portable api call is pause, but unlike quiesce, pause will also stop the execution of the vm. i say its semi portable as drivers are not required to implement it. it does have more broad support https://docs.openstack.org/nova/latest/user/support-matrix.html#operation_pa... vmware,powervm and ironic being the main virt drivers missing support. calling pause however will be a distruptive backup and would not be suitable in many cases. it is overkill in most cases and it also wont guarentte that the io buffers are flushed just that no new data is written to the disks but the paused instnace while the backup is done.
Cheers, -melanie
[1] https://review.openstack.org/295595 [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-January/001984.h...
Il giorno Mer 23 Gen 2019 17:01 Sean McGinnis <sean.mcginnis@gmx.com <mailto:sean.mcginnis@gmx.com>> ha scritto:
On Wed, Jan 23, 2019 at 04:56:24PM +0100, Ignazio Cassano wrote: > Hello, I did not understand if you mean cinder snapshot pr netapp snapshot. > Any case, why, we do not need to quiesce the instance ? > Regards > Ignazio >
If being crash consistent is good enough for your needs, then you don't. I know some do prefer the coordinated quiescing of IO in the instance to make sure any in-flight transactions are flushed out and application data is more likely to be in a good consistent state.
Depending on your application running in the instance, things like databases are pretty good at rolling back incomplete transactions, so it's just a matter of whether you can allow the possibility that something that was successful in the milliseconds before the snap was created to now be rolled back when the application restarts.
participants (6)
-
Ignazio Cassano
-
Matt Riedemann
-
Matthew Booth
-
melanie witt
-
Sean McGinnis
-
Sean Mooney