Open Stack

Thank you both for your responses and insight.  It is very much appreciated.

Dmitriy,

Is there any sort of document/webpage about OpenStack-Ansible Internals?  I
didn't know anything about the scripts/inventory-manage.py script.  I will
study it further and get a full understanding.  I'd also be interested to
know how OpenStack-Ansible is put together - how the inventory is generated
and managed, how the playbooks in /etc/ansible are managed and interpreted,
etc.

I'm also going to be wondering how to clean up the cinder-volume processes
running on metal - until I get this active/active vs. active/standby sorted
out I may need to drop to a single cinder-volume process.  What's important
for me right now is to be able to deploy instances ASAP and sort out the
resilience issues later.

Takashi,

Thank you for clearing up my misconceptions and helping me to understand
cinder's logical structure.  In the long run I will definitely aim for a
3-way active/active deployment.

-Dave

--
Dave Hall
Binghamton University
kdhall at binghamton.edu

On Sun, Aug 7, 2022 at 11:10 AM Takashi Kajinami <tkajinam at redhat.com>
wrote:

> I'm not familiar with the setup created by openstack-ansible but let me
> leave a few comments
> from cinder's perspective.
>
> - cinder api does not require access to backend storage. Only
> cinder-volume communicates with
>   backend storage. Thus missing nfs mount/tools in cinder-api is what we
> definitely expect.
>
> - Usually you need to deploy cinder-volume as an act/sby process. Your
> cinder-volume would  run
>   on only one of the controller nodes managed by a cluster technology like
> pacemaker.
>   You can split cinder-volume per backend but in such case you'd run each
> c-vol in act/sby mode.
>
> - Recent cinder supports deploying cinder-volume in active/active mode
> with lock management by
>   tooz. I'm not sure whether this is supported by openstack-ansible as
> well as it is well tested with
>   NFS backend.
>
> - When you deploy cinder-volume in active/active mode, cinder-volume
> should run in all controller nodes
>   and all processes should have access to the same storage backends. If
> you use different backend setting
>   for each controller then you'd lost control about some of your volumes
> when the associated controller node
>   goes down.
>
> I'd recommend checking whether you expect deploying cinder-volume in
> active/standby or in
> active/active. In case you want active/active then you should have
> multiple cinder-volume processes
> with the same backend definition.
>
> On Sun, Aug 7, 2022 at 11:55 PM Dmitriy Rabotyagov <
> noonedeadpunk at gmail.com> wrote:
>
>> Hi,
>>
>> 1. When you remove definition from openstack_user_config, items from
>> inventory won't be dropped automatically. For that you would need to use
>> scripts/inventory-manage.py -r <container_name>. You should indeed destroy
>> containers first.
>>
>> 2. In order to remove cider-volume definitions from API, you would need
>> to run cinder-manage service remove <service> <host>
>>
>>
>> https://docs.openstack.org/cinder/rocky/man/cinder-manage.html#cinder-service
>>
>> cinder-manage binary can be found on cinder-api containers inside
>> virtualenv, ie /openstack/venvs/cinder-<venv_tag>/bin/cinder-manage
>>
>>
>> вс, 7 авг. 2022 г., 16:22 Dave Hall <kdhall at binghamton.edu>:
>>
>>> Hello,
>>>
>>> Please pardon the repost - I noticed this morning that I didn't finish
>>> the subject line.
>>>
>>> Problem summary:  I have a bunch of lingering non-functional cinder
>>> definitions and I'm looking for guidance on how to clean them up.
>>>
>>> Thanks.
>>>
>>> -Dave
>>>
>>> --
>>> Dave Hall
>>> Binghamton University
>>> kdhall at binghamton.edu
>>>
>>>
>>>
>>> On Sat, Aug 6, 2022 at 2:52 PM Dave Hall <kdhall at binghamton.edu> wrote:
>>>
>>>> Hello,
>>>>
>>>> I seem to have gotten myself in a bit of a mess trying to set up Cinder
>>>> with an NFS back-end.  After working with Glance and NFS, I started on
>>>> Cinder.  I noticed immediately that there weren't any NFS mounts in the
>>>> Cinder-API containers like there were in the Glance-API containers.  Also
>>>> that there were no NFS packages in the Cinder-API containers.
>>>>
>>>> In reading some Cinder documentation, I also got the impression that
>>>> each Cinder host/container needs to have its own NFS store.
>>>>
>>>> Pawing through the playbooks and documentation I saw that unlike
>>>> Glance, Cinder is split into two pieces - Cinder-API and Cinder-Volume.  I
>>>> found cinder-volume.yml.example in env.d, activated it, and created
>>>> Cinder-Volume containers on my 3 infra hosts.  I also created 3 separate
>>>> NFS shares and changed the storage-hosts section of my
>>>> openstack_user_config.yml accordingly.
>>>>
>>>> After this I found that while I was able to create volumes, the
>>>> prep_volume part of launching an instance was failing.
>>>>
>>>> Digging in, I found:
>>>>
>>>> # openstack volume service list
>>>>
>>>> +------------------+-------------------------------------------------------+------+---------+-------+----------------------------+
>>>> | Binary           | Host
>>>>    | Zone | Status  | State | Updated At                 |
>>>>
>>>> +------------------+-------------------------------------------------------+------+---------+-------+----------------------------+
>>>> | cinder-volume    | C6220-9 at nfs_volume
>>>>      | nova | enabled | down  | 2022-07-23T02:46:13.000000 |
>>>> | cinder-volume    | C6220-10 at nfs_volume
>>>>     | nova | enabled | down  | 2022-07-23T02:46:14.000000 |
>>>> | cinder-volume    | C6220-11 at nfs_volume
>>>>     | nova | enabled | down  | 2022-07-23T02:46:14.000000 |
>>>> | cinder-scheduler | infra36-cinder-api-container-da8e100f
>>>>     | nova | enabled | up    | 2022-08-06T13:29:10.000000 |
>>>> | cinder-scheduler | infra38-cinder-api-container-27219f93
>>>>     | nova | enabled | up    | 2022-08-06T13:29:10.000000 |
>>>> | cinder-scheduler | infra37-cinder-api-container-ea7f847b
>>>>     | nova | enabled | up    | 2022-08-06T13:29:10.000000 |
>>>> | cinder-volume    | C6220-9 at nfs_volume1
>>>>     | nova | enabled | up    | 2022-08-06T13:29:10.000000 |
>>>> | cinder-volume    |
>>>> infra37-cinder-volumes-container-5b9635ad at nfs_volume  | nova | enabled
>>>> | down  | 2022-08-04T18:32:53.000000 |
>>>> | cinder-volume    |
>>>> infra36-cinder-volumes-container-77190057 at nfs_volume1 | nova | enabled
>>>> | down  | 2022-08-06T13:03:03.000000 |
>>>> | cinder-volume    |
>>>> infra38-cinder-volumes-container-a7bcfc9b at nfs_volume  | nova | enabled
>>>> | down  | 2022-08-04T18:32:53.000000 |
>>>> | cinder-volume    |
>>>> infra37-cinder-volumes-container-5b9635ad at nfs_volume2 | nova | enabled
>>>> | down  | 2022-08-06T13:03:05.000000 |
>>>> | cinder-volume    | C6220-10 at nfs_volume2
>>>>      | nova | enabled | up    | 2022-08-06T13:29:10.000000 |
>>>> | cinder-volume    | C6220-11 at nfs_volume3
>>>>      | nova | enabled | up    | 2022-08-06T13:29:10.000000 |
>>>> | cinder-volume    |
>>>> infra38-cinder-volumes-container-a7bcfc9b at nfs_volume3 | nova | enabled
>>>> | down  | 2022-08-06T13:03:03.000000 |
>>>>
>>>> +------------------+-------------------------------------------------------+------+---------+-------+----------------------------+
>>>>
>>>> Thinking I could save this, I used containers-lxc-destroy.yml to
>>>> destroy my cinder-volumes containers and deactivated
>>>> cinder-volume.yml.example.  Then I ran setup-hosts.yml, which has restored
>>>> the cinder-volumes containers even though is_metal: false has been removed.
>>>>
>>>> Clearly a stronger intervention will be required.  I would like to
>>>> fully get rid of the cinder-volumes containers and go back to an is_metal:
>>>> true scenario.  I also need to get rid of  the unnumbered nfs_volume
>>>> referenes, which I assume are in some cinder config file somewhere.
>>>>
>>>> Below is a clip from my openstack_user_config.yml:
>>>>
>>>> storage_hosts:
>>>>   infra36:
>>>>     ip: 172.29.236.36
>>>>     container_vars:
>>>>       cinder_backends:
>>>>         nfs_volume1:
>>>>           volume_backend_name: NFS_VOLUME1
>>>>           volume_driver: cinder.volume.drivers.nfs.NfsDriver
>>>>           nfs_mount_options:
>>>> "rsize=65535,wsize=65535,timeo=1200,actimeo=120"
>>>>           nfs_shares_config: /etc/cinder/nfs_shares_volume1
>>>>           shares:
>>>>           - { ip: "172.29.244.27", share: "/NFS_VOLUME1" }
>>>>   infra37:
>>>>     ip: 172.29.236.37
>>>>     container_vars:
>>>>       cinder_backends:
>>>>         nfs_volume2:
>>>>           volume_backend_name: NFS_VOLUME2
>>>>           volume_driver: cinder.volume.drivers.nfs.NfsDriver
>>>>           nfs_mount_options:
>>>> "rsize=65535,wsize=65535,timeo=1200,actimeo=120"
>>>>           nfs_shares_config: /etc/cinder/nfs_shares_volume2
>>>>           shares:
>>>>           - { ip: "172.29.244.27", share: "/NFS_VOLUME2" }
>>>>   infra38:
>>>>     ip: 172.29.236.38
>>>>     container_vars:
>>>>       cinder_backends:
>>>>         nfs_volume3:
>>>>           volume_backend_name: NFS_VOLUME3
>>>>           volume_driver: cinder.volume.drivers.nfs.NfsDriver
>>>>           nfs_mount_options:
>>>> "rsize=65535,wsize=65535,timeo=1200,actimeo=120"
>>>>           nfs_shares_config: /etc/cinder/nfs_shares_volume3
>>>>           shares:
>>>>           - { ip: "172.29.244.27", share: "/NFS_VOLUME3" }
>>>>
>>>> Any advice would be greatly appreciated.
>>>>
>>>> Thanks.
>>>>
>>>> -Dave
>>>>
>>>> --
>>>> Dave Hall
>>>> Binghamton University
>>>> kdhall at binghamton.edu
>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20220807/4d98a917/attachment-0001.htm>