Open Stack

On Fri, Feb 5, 2021 at 10:00 AM Adam Zheng <adam.zheng at colorado.edu> wrote:

> Hello,
>
>
>
> I’ve been trying to get availability zones defined for volumes.
>
> Everything works fine if I leave the zone at “nova”, all volume types work
> and backups/snapshots also work.
>
>
>
> ie:
>
>
> +------------------+----------------------------+------+---------+-------+----------------------------+
>
> | Binary           | Host                       | Zone | Status  | State |
> Updated At                 |
>
>
> +------------------+----------------------------+------+---------+-------+----------------------------+
>
> | cinder-scheduler | cs-os-ctl-001              | nova | enabled | up    |
> 2021-02-05T17:22:51.000000 |
>
> | cinder-scheduler | cs-os-ctl-003              | nova | enabled | up    |
> 2021-02-05T17:22:54.000000 |
>
> | cinder-scheduler | cs-os-ctl-002              | nova | enabled | up    |
> 2021-02-05T17:22:56.000000 |
>
> | cinder-volume    | cs-os-ctl-001 at rbd-ceph-gp2 | nova | enabled | up
> | 2021-02-05T17:22:56.000000 |
>
> | cinder-volume    | cs-os-ctl-001 at rbd-ceph-st1 | nova | enabled | up
> | 2021-02-05T17:22:54.000000 |
>
> | cinder-volume    | cs-os-ctl-002 at rbd-ceph-gp2 | nova | enabled | up
> | 2021-02-05T17:22:50.000000 |
>
> | cinder-volume    | cs-os-ctl-003 at rbd-ceph-gp2 | nova | enabled | up
> | 2021-02-05T17:22:55.000000 |
>
> | cinder-volume    | cs-os-ctl-002 at rbd-ceph-st1 | nova | enabled | up
> | 2021-02-05T17:22:57.000000 |
>
> | cinder-volume    | cs-os-ctl-003 at rbd-ceph-st1 | nova | enabled | up
> | 2021-02-05T17:22:54.000000 |
>
> | cinder-backup    | cs-os-ctl-002              | nova | enabled | up    |
> 2021-02-05T17:22:56.000000 |
>
> | cinder-backup    | cs-os-ctl-001              | nova | enabled | up    |
> 2021-02-05T17:22:53.000000 |
>
> | cinder-backup    | cs-os-ctl-003              | nova | enabled | up    |
> 2021-02-05T17:22:58.000000 |
>
>
> +------------------+----------------------------+------+---------+-------+----------------------------+
>
>
>
> However, if I apply the following changes:
>
>
>
> cinder-api.conf
>
> [DEFAULT]
>
> default_availability_zone = not-nova
>
> default_volume_type = ceph-gp2
>
> allow_availability_zone_fallback=True
>
>
>
> cinder-volume.conf
>
> [rbd-ceph-gp2]
>
> <…>
>
> backend_availability_zone = not-nova
>
> <…>
>
>
>
> I’ll get the following
>
>
> +------------------+----------------------------+----------+---------+-------+----------------------------+
>
> | Binary           | Host                       | Zone     | Status  |
> State | Updated At                 |
>
>
> +------------------+----------------------------+----------+---------+-------+----------------------------+
>
> | cinder-scheduler | cs-os-ctl-001              | nova     | enabled |
> up    | 2021-02-05T17:22:51.000000 |
>
> | cinder-scheduler | cs-os-ctl-003              | nova     | enabled |
> up    | 2021-02-05T17:22:54.000000 |
>
> | cinder-scheduler | cs-os-ctl-002              | nova     | enabled |
> up    | 2021-02-05T17:22:56.000000 |
>
> | cinder-volume    | cs-os-ctl-001 at rbd-ceph-gp2 | not-nova | enabled |
> up    | 2021-02-05T17:22:56.000000 |
>
> | cinder-volume    | cs-os-ctl-001 at rbd-ceph-st1 | nova     | enabled |
> up    | 2021-02-05T17:22:54.000000 |
>
> | cinder-volume    | cs-os-ctl-002 at rbd-ceph-gp2 | not-nova | enabled |
> up    | 2021-02-05T17:22:50.000000 |
>
> | cinder-volume    | cs-os-ctl-003 at rbd-ceph-gp2 | not-nova | enabled |
> up    | 2021-02-05T17:22:55.000000 |
>
> | cinder-volume    | cs-os-ctl-002 at rbd-ceph-st1 | nova     | enabled |
> up    | 2021-02-05T17:22:57.000000 |
>
> | cinder-volume    | cs-os-ctl-003 at rbd-ceph-st1 | nova     | enabled |
> up    | 2021-02-05T17:22:54.000000 |
>
> | cinder-backup    | cs-os-ctl-002              | nova     | enabled |
> up    | 2021-02-05T17:22:56.000000 |
>
> | cinder-backup    | cs-os-ctl-001              | nova     | enabled |
> up    | 2021-02-05T17:22:53.000000 |
>
> | cinder-backup    | cs-os-ctl-003              | nova     | enabled |
> up    | 2021-02-05T17:22:58.000000 |
>
>
> +------------------+----------------------------+----------+---------+-------+----------------------------+
>
>
>
> At this point, creating new volumes still work and go into the expected
> ceph pools.
>
> However, backups no longer work for the cinder-volume that is not nova.
>
> In the above example, it still works fine for volumes that that were
> created with type “ceph-gp2” in az “nova”.
>
> Does not work for volumes that were created with type “ceph-st1” in az
> “not-nova”. It fails immediately and goes into error state with reason
> “Service not found for creating backup.”
>
Hi Adam,

Cinder's backup service has the ability to create backups of volumes in
another AZ. The 'cinder' CLI supports this feature as of microversion 3.51.
(bear in mind the 'openstack' client doesn't support microversions for the
cinder (volume) service, so you'll need to use the 'cinder' command.

Rather than repeat what I've written previously, I refer you to [1] for
additional details.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1649845#c4

One other thing to note is the corresponding "cinder backup-restore"
command currently does not support restoring to a volume in another AZ, but
there is a workaround. You can pre-create a new volume in the destination
AZ, and use the ability to restore a backup to a specific volume (which
just happens to be in your desired AZ).

There's also a patch [2] under review to enhance the cinder shell so that
both backup and restore shell commands work the same way.

[2] https://review.opendev.org/c/openstack/python-cinderclient/+/762020

Alan


> I suspect I need to try to get another set of “cinder-backup” services
> running in the Zone “not-nova”, but cannot seem to figure out how.
>
>
>
> I’ve scoured the docs on cinder.conf, and if I set default zones in
> cinder-backup (I’ve tried backend_availability_zone,
> default_availability_zone, and storage_availability_zone) I cannot seem to
> get backups working if the disk it’s backing up is not in az “nova”. The
> cinder-backup service in volume service list will always show “nova” no
> matter what I put there.
>
>
>
> Any advice would be appreciated.
>
> OpenStack Victoria deployed via kolla-ansible
>
>
>
> Thanks!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210205/11144765/attachment.html>