[Openstack] nova backup - instances unreachable

Eugen Block eblock at nde.ag
Thu Jan 12 12:47:51 UTC 2017


That's strange, I also searched for this message, but nothing there. I  
have debug logs enabled on compute node but I don't see anything  
regarding ceph. No matter, what I do, my instance is always shutdown  
before a snapshot is taken. What else can I try?


Zitat von John Petrini <jpetrini at coredial.com>:

> Mohammed,
>
> It looks like you may be right. Just found the permissions issue in the
> nova log on the compute node.
>
> 4-e8f52e4fbcfb 691caf1c10354efab3e3c8ed61b7d89a
> 49bc5e5bf2684bd0948d9f94c7875027 - - -] Performing standard snapshot
> because direct snapshot failed: no write permission on storage pool images
>
> I'm going to test the change and will send an update you all with the
> results.
>
> Thank You,
>
> ___
>
> John Petrini
>
>
>
>>>
>> Yes, we are also running Mitaka and I also read Sebastien Han's blogs ;-)
>>
>> our snapshots are not happening at the RBD level,
>>> they are being copied and uploaded to glance which takes up a lot of space
>>> and is very slow.
>>>
>>
>> Unfortunately, that's what we are experiencing, too. I don't know if
>> there's something I missed in the nova configs or somewhere else, but I'm
>> relieved that I'm not the only one :-)
>>
>> While writing this email I searched again and found something:
>>
>> https://specs.openstack.org/openstack/nova-specs/specs/mitak
>> a/implemented/rbd-instance-snapshots.html
>>
>> https://review.openstack.org/#/c/205282/
>>
>> It seems to be implemented already, I'm looking for the config options to
>> set. If you manage to get nova to make rbd snapshots, please let me know ;-)
>>
>> Regards,
>> Eugen
>>
>>
>>
>> Zitat von John Petrini <jpetrini at coredial.com>:
>>
>> Hi Eugen,
>>>
>>> Thanks for the response! That makes a lost of sense and is what I figured
>>> was going on but I missed it in the documentation. We use Ceph as well and
>>> I had considered doing the snapshots at the RBD level but I was hoping
>>> there was someway to accomplish this via nova. I came across this
>>> Sebastien
>>> Han write-up that claims this functionality was added to Mitaka:
>>> http://www.sebastien-han.fr/blog/2015/10/05/openstack-nova-
>>> snapshots-on-ceph-rbd/
>>>
>>> We are running Mitaka but our snapshots are not happening at the RBD
>>> level,
>>> they are being copied and uploaded to glance which takes up a lot of space
>>> and is very slow.
>>>
>>> Have you or anyone else implemented this in Mitaka? Other than Sebastian's
>>> blog I haven't found any documentation on this.
>>>
>>> Thank You,
>>>
>>> ___
>>>
>>> John Petrini
>>>
>>> On Wed, Jan 11, 2017 at 3:32 AM, Eugen Block <eblock at nde.ag> wrote:
>>>
>>> Hi,
>>>>
>>>> this seems to be exptected, the docs say:
>>>>
>>>> "Shut down the source VM before you take the snapshot to ensure that all
>>>> data is flushed to disk."
>>>>
>>>> So if the VM is not shut down, it's freezed to prevent data loss (I
>>>> guess). Depending on your storage backend, there are other ways to
>>>> perform
>>>> backups of your VMs.
>>>> We use Ceph as backend for nova, glance and cinder. Ceph stores the
>>>> disks,
>>>> images and volumes as Rados block device objects. We have a backup script
>>>> that creates snapshots of these RBDs, which are exported to our backup
>>>> drive. This way the running VM is not stopped or freezed, the user
>>>> doesn't
>>>> notice any issues. Unlike a nova snapshot, the rbd snapshot is created
>>>> immediately within a few seconds. After a successful backup the snapshots
>>>> are removed.
>>>>
>>>> Hope this helps! If you are interested in Ceph, visit [1].
>>>>
>>>> Regards,
>>>> Eugen
>>>>
>>>> [1] http://docs.ceph.com/docs/giant/start/intro/
>>>>
>>>>
>>>> Zitat von John Petrini <jpetrini at coredial.com>:
>>>>
>>>>
>>>> Hello,
>>>>
>>>>>
>>>>> I've just started experimenting with nova backup and discovered that
>>>>> there
>>>>> is a period of time during the snapshot where the instance becomes
>>>>> unreachable. Is this behavior expected during a live snapshot? Is there
>>>>> any
>>>>> way to prevent this?
>>>>>
>>>>> ___
>>>>>
>>>>> John Petrini
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Eugen Block                             voice   : +49-40-559 51 75
>>>> NDE Netzdesign und -entwicklung AG      fax     : +49-40-559 51 77
>>>> Postfach 61 03 15
>>>> D-22423 Hamburg                         e-mail  : eblock at nde.ag
>>>>
>>>>         Vorsitzende des Aufsichtsrates: Angelika Mozdzen
>>>>           Sitz und Registergericht: Hamburg, HRB 90934
>>>>                   Vorstand: Jens-U. Mozdzen
>>>>                    USt-IdNr. DE 814 013 983
>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: http://lists.openstack.org/cgi
>>>> -bin/mailman/listinfo/openstac
>>>> k
>>>> Post to     : openstack at lists.openstack.org
>>>> Unsubscribe : http://lists.openstack.org/cgi
>>>> -bin/mailman/listinfo/openstac
>>>> k
>>>>
>>>>
>>
>>
>> --
>> Eugen Block                             voice   : +49-40-559 51 75
>> NDE Netzdesign und -entwicklung AG      fax     : +49-40-559 51 77
>> Postfach 61 03 15
>> D-22423 Hamburg                         e-mail  : eblock at nde.ag
>>
>>         Vorsitzende des Aufsichtsrates: Angelika Mozdzen
>>           Sitz und Registergericht: Hamburg, HRB 90934
>>                   Vorstand: Jens-U. Mozdzen
>>                    USt-IdNr. DE 814 013 983
>>
>>



-- 
Eugen Block                             voice   : +49-40-559 51 75
NDE Netzdesign und -entwicklung AG      fax     : +49-40-559 51 77
Postfach 61 03 15
D-22423 Hamburg                         e-mail  : eblock at nde.ag

         Vorsitzende des Aufsichtsrates: Angelika Mozdzen
           Sitz und Registergericht: Hamburg, HRB 90934
                   Vorstand: Jens-U. Mozdzen
                    USt-IdNr. DE 814 013 983





More information about the Openstack mailing list