Open Stack

I'm not familiar with the setup created by openstack-ansible but let me
leave a few comments
from cinder's perspective.

- cinder api does not require access to backend storage. Only cinder-volume
communicates with
  backend storage. Thus missing nfs mount/tools in cinder-api is what we
definitely expect.

- Usually you need to deploy cinder-volume as an act/sby process. Your
cinder-volume would  run
  on only one of the controller nodes managed by a cluster technology like
pacemaker.
  You can split cinder-volume per backend but in such case you'd run each
c-vol in act/sby mode.

- Recent cinder supports deploying cinder-volume in active/active mode with
lock management by
  tooz. I'm not sure whether this is supported by openstack-ansible as well
as it is well tested with
  NFS backend.

- When you deploy cinder-volume in active/active mode, cinder-volume should
run in all controller nodes
  and all processes should have access to the same storage backends. If you
use different backend setting
  for each controller then you'd lost control about some of your volumes
when the associated controller node
  goes down.

I'd recommend checking whether you expect deploying cinder-volume in
active/standby or in
active/active. In case you want active/active then you should have multiple
cinder-volume processes
with the same backend definition.

On Sun, Aug 7, 2022 at 11:55 PM Dmitriy Rabotyagov <noonedeadpunk at gmail.com>
wrote:

> Hi,
>
> 1. When you remove definition from openstack_user_config, items from
> inventory won't be dropped automatically. For that you would need to use
> scripts/inventory-manage.py -r <container_name>. You should indeed destroy
> containers first.
>
> 2. In order to remove cider-volume definitions from API, you would need to
> run cinder-manage service remove <service> <host>
>
>
> https://docs.openstack.org/cinder/rocky/man/cinder-manage.html#cinder-service
>
> cinder-manage binary can be found on cinder-api containers inside
> virtualenv, ie /openstack/venvs/cinder-<venv_tag>/bin/cinder-manage
>
>
> вс, 7 авг. 2022 г., 16:22 Dave Hall <kdhall at binghamton.edu>:
>
>> Hello,
>>
>> Please pardon the repost - I noticed this morning that I didn't finish
>> the subject line.
>>
>> Problem summary:  I have a bunch of lingering non-functional cinder
>> definitions and I'm looking for guidance on how to clean them up.
>>
>> Thanks.
>>
>> -Dave
>>
>> --
>> Dave Hall
>> Binghamton University
>> kdhall at binghamton.edu
>>
>>
>>
>> On Sat, Aug 6, 2022 at 2:52 PM Dave Hall <kdhall at binghamton.edu> wrote:
>>
>>> Hello,
>>>
>>> I seem to have gotten myself in a bit of a mess trying to set up Cinder
>>> with an NFS back-end.  After working with Glance and NFS, I started on
>>> Cinder.  I noticed immediately that there weren't any NFS mounts in the
>>> Cinder-API containers like there were in the Glance-API containers.  Also
>>> that there were no NFS packages in the Cinder-API containers.
>>>
>>> In reading some Cinder documentation, I also got the impression that
>>> each Cinder host/container needs to have its own NFS store.
>>>
>>> Pawing through the playbooks and documentation I saw that unlike Glance,
>>> Cinder is split into two pieces - Cinder-API and Cinder-Volume.  I found
>>> cinder-volume.yml.example in env.d, activated it, and created Cinder-Volume
>>> containers on my 3 infra hosts.  I also created 3 separate NFS shares and
>>> changed the storage-hosts section of my openstack_user_config.yml
>>> accordingly.
>>>
>>> After this I found that while I was able to create volumes, the
>>> prep_volume part of launching an instance was failing.
>>>
>>> Digging in, I found:
>>>
>>> # openstack volume service list
>>>
>>> +------------------+-------------------------------------------------------+------+---------+-------+----------------------------+
>>> | Binary           | Host
>>>    | Zone | Status  | State | Updated At                 |
>>>
>>> +------------------+-------------------------------------------------------+------+---------+-------+----------------------------+
>>> | cinder-volume    | C6220-9 at nfs_volume
>>>    | nova | enabled | down  | 2022-07-23T02:46:13.000000 |
>>> | cinder-volume    | C6220-10 at nfs_volume
>>>     | nova | enabled | down  | 2022-07-23T02:46:14.000000 |
>>> | cinder-volume    | C6220-11 at nfs_volume
>>>     | nova | enabled | down  | 2022-07-23T02:46:14.000000 |
>>> | cinder-scheduler | infra36-cinder-api-container-da8e100f
>>>   | nova | enabled | up    | 2022-08-06T13:29:10.000000 |
>>> | cinder-scheduler | infra38-cinder-api-container-27219f93
>>>   | nova | enabled | up    | 2022-08-06T13:29:10.000000 |
>>> | cinder-scheduler | infra37-cinder-api-container-ea7f847b
>>>   | nova | enabled | up    | 2022-08-06T13:29:10.000000 |
>>> | cinder-volume    | C6220-9 at nfs_volume1
>>>     | nova | enabled | up    | 2022-08-06T13:29:10.000000 |
>>> | cinder-volume    | infra37-cinder-volumes-container-5b9635ad at nfs_volume
>>>  | nova | enabled | down  | 2022-08-04T18:32:53.000000 |
>>> | cinder-volume    |
>>> infra36-cinder-volumes-container-77190057 at nfs_volume1 | nova | enabled
>>> | down  | 2022-08-06T13:03:03.000000 |
>>> | cinder-volume    | infra38-cinder-volumes-container-a7bcfc9b at nfs_volume
>>>  | nova | enabled | down  | 2022-08-04T18:32:53.000000 |
>>> | cinder-volume    |
>>> infra37-cinder-volumes-container-5b9635ad at nfs_volume2 | nova | enabled
>>> | down  | 2022-08-06T13:03:05.000000 |
>>> | cinder-volume    | C6220-10 at nfs_volume2
>>>    | nova | enabled | up    | 2022-08-06T13:29:10.000000 |
>>> | cinder-volume    | C6220-11 at nfs_volume3
>>>    | nova | enabled | up    | 2022-08-06T13:29:10.000000 |
>>> | cinder-volume    |
>>> infra38-cinder-volumes-container-a7bcfc9b at nfs_volume3 | nova | enabled
>>> | down  | 2022-08-06T13:03:03.000000 |
>>>
>>> +------------------+-------------------------------------------------------+------+---------+-------+----------------------------+
>>>
>>> Thinking I could save this, I used containers-lxc-destroy.yml to destroy
>>> my cinder-volumes containers and deactivated cinder-volume.yml.example.
>>> Then I ran setup-hosts.yml, which has restored the cinder-volumes
>>> containers even though is_metal: false has been removed.
>>>
>>> Clearly a stronger intervention will be required.  I would like to fully
>>> get rid of the cinder-volumes containers and go back to an is_metal: true
>>> scenario.  I also need to get rid of  the unnumbered nfs_volume referenes,
>>> which I assume are in some cinder config file somewhere.
>>>
>>> Below is a clip from my openstack_user_config.yml:
>>>
>>> storage_hosts:
>>>   infra36:
>>>     ip: 172.29.236.36
>>>     container_vars:
>>>       cinder_backends:
>>>         nfs_volume1:
>>>           volume_backend_name: NFS_VOLUME1
>>>           volume_driver: cinder.volume.drivers.nfs.NfsDriver
>>>           nfs_mount_options:
>>> "rsize=65535,wsize=65535,timeo=1200,actimeo=120"
>>>           nfs_shares_config: /etc/cinder/nfs_shares_volume1
>>>           shares:
>>>           - { ip: "172.29.244.27", share: "/NFS_VOLUME1" }
>>>   infra37:
>>>     ip: 172.29.236.37
>>>     container_vars:
>>>       cinder_backends:
>>>         nfs_volume2:
>>>           volume_backend_name: NFS_VOLUME2
>>>           volume_driver: cinder.volume.drivers.nfs.NfsDriver
>>>           nfs_mount_options:
>>> "rsize=65535,wsize=65535,timeo=1200,actimeo=120"
>>>           nfs_shares_config: /etc/cinder/nfs_shares_volume2
>>>           shares:
>>>           - { ip: "172.29.244.27", share: "/NFS_VOLUME2" }
>>>   infra38:
>>>     ip: 172.29.236.38
>>>     container_vars:
>>>       cinder_backends:
>>>         nfs_volume3:
>>>           volume_backend_name: NFS_VOLUME3
>>>           volume_driver: cinder.volume.drivers.nfs.NfsDriver
>>>           nfs_mount_options:
>>> "rsize=65535,wsize=65535,timeo=1200,actimeo=120"
>>>           nfs_shares_config: /etc/cinder/nfs_shares_volume3
>>>           shares:
>>>           - { ip: "172.29.244.27", share: "/NFS_VOLUME3" }
>>>
>>> Any advice would be greatly appreciated.
>>>
>>> Thanks.
>>>
>>> -Dave
>>>
>>> --
>>> Dave Hall
>>> Binghamton University
>>> kdhall at binghamton.edu
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20220808/4d8fc28a/attachment-0001.htm>