<div dir="ltr"><div>Hello,</div><div><br></div><div>I seem to have gotten myself in a bit of a mess trying to set up Cinder with an NFS back-end.  After working with Glance and NFS, I started on Cinder.  I noticed immediately that there weren't any NFS mounts in the Cinder-API containers like there were in the Glance-API containers.  Also that there were no NFS packages in the Cinder-API containers.  <br></div><div><br></div><div>In reading some Cinder documentation, I also got the impression that each Cinder host/container needs to have its own NFS store.  <br></div><div><br></div><div>Pawing through the playbooks and documentation I saw that unlike Glance, Cinder is split into two pieces - Cinder-API and Cinder-Volume.  I found cinder-volume.yml.example in env.d, activated it, and created Cinder-Volume containers on my 3 infra hosts.  I also created 3 separate NFS shares and changed the storage-hosts section of my openstack_user_config.yml accordingly.</div><div><br></div><div>After this I found that while I was able to create volumes, the prep_volume part of launching an instance was failing.  <br></div><div><br></div><div>Digging in, I found:</div><div style="margin-left:40px"><br></div><div style="margin-left:40px"><span style="font-family:monospace"># openstack volume service list<br>+------------------+-------------------------------------------------------+------+---------+-------+----------------------------+<br>| Binary           | Host                                                  | Zone | Status  | State | Updated At                 |<br>+------------------+-------------------------------------------------------+------+---------+-------+----------------------------+<br>| cinder-volume    | C6220-9@nfs_volume                                    | nova | enabled | down  | 2022-07-23T02:46:13.000000 |<br>| cinder-volume    | C6220-10@nfs_volume                                   | nova | enabled | down  | 2022-07-23T02:46:14.000000 |<br>| cinder-volume    | C6220-11@nfs_volume                                   | nova | enabled | down  | 2022-07-23T02:46:14.000000 |<br>| cinder-scheduler | infra36-cinder-api-container-da8e100f                 | nova | enabled | up    | 2022-08-06T13:29:10.000000 |<br>| cinder-scheduler | infra38-cinder-api-container-27219f93                 | nova | enabled | up    | 2022-08-06T13:29:10.000000 |<br>| cinder-scheduler | infra37-cinder-api-container-ea7f847b                 | nova | enabled | up    | 2022-08-06T13:29:10.000000 |<br>| cinder-volume    | C6220-9@nfs_volume1                                   | nova | enabled | up    | 2022-08-06T13:29:10.000000 |<br>| cinder-volume    | infra37-cinder-volumes-container-5b9635ad@nfs_volume  | nova | enabled | down  | 2022-08-04T18:32:53.000000 |<br>| cinder-volume    | infra36-cinder-volumes-container-77190057@nfs_volume1 | nova | enabled | down  | 2022-08-06T13:03:03.000000 |<br>| cinder-volume    | infra38-cinder-volumes-container-a7bcfc9b@nfs_volume  | nova | enabled | down  | 2022-08-04T18:32:53.000000 |<br>| cinder-volume    | infra37-cinder-volumes-container-5b9635ad@nfs_volume2 | nova | enabled | down  | 2022-08-06T13:03:05.000000 |<br>| cinder-volume    | C6220-10@nfs_volume2                                  | nova | enabled | up    | 2022-08-06T13:29:10.000000 |<br>| cinder-volume    | C6220-11@nfs_volume3                                  | nova | enabled | up    | 2022-08-06T13:29:10.000000 |<br>| cinder-volume    | infra38-cinder-volumes-container-a7bcfc9b@nfs_volume3 | nova | enabled | down  | 2022-08-06T13:03:03.000000 |<br>+------------------+-------------------------------------------------------+------+---------+-------+----------------------------+</span><br></div><div><br></div><div>Thinking I could save this, I used containers-lxc-destroy.yml to destroy my cinder-volumes containers and deactivated cinder-volume.yml.example.  Then I ran setup-hosts.yml, which has restored the cinder-volumes containers even though is_metal: false has been removed.</div><div><br></div><div>Clearly a stronger intervention will be required.  I would like to fully get rid of the cinder-volumes containers and go back to an is_metal: true scenario.  I also need to get rid of  the unnumbered nfs_volume referenes, which I assume are in some cinder config file somewhere.  <br></div><div><br></div><div>Below is a clip from my openstack_user_config.yml:</div><div><br></div><div style="margin-left:40px"><span style="font-family:monospace">storage_hosts:<br>  infra36:<br>    ip: 172.29.236.36<br>    container_vars:<br>      cinder_backends:<br>        nfs_volume1:<br>          volume_backend_name: NFS_VOLUME1<br>          volume_driver: cinder.volume.drivers.nfs.NfsDriver<br>          nfs_mount_options: "rsize=65535,wsize=65535,timeo=1200,actimeo=120"<br>          nfs_shares_config: /etc/cinder/nfs_shares_volume1<br>          shares:<br>          - { ip: "172.29.244.27", share: "/NFS_VOLUME1" }<br>  infra37:<br>    ip: 172.29.236.37<br>    container_vars:<br>      cinder_backends:<br>        nfs_volume2:<br>          volume_backend_name: NFS_VOLUME2<br>          volume_driver: cinder.volume.drivers.nfs.NfsDriver<br>          nfs_mount_options: "rsize=65535,wsize=65535,timeo=1200,actimeo=120"<br>          nfs_shares_config: /etc/cinder/nfs_shares_volume2<br>          shares:<br>          - { ip: "172.29.244.27", share: "/NFS_VOLUME2" }<br>  infra38:<br>    ip: 172.29.236.38<br>    container_vars:<br>      cinder_backends:<br>        nfs_volume3:<br>          volume_backend_name: NFS_VOLUME3<br>          volume_driver: cinder.volume.drivers.nfs.NfsDriver<br>          nfs_mount_options: "rsize=65535,wsize=65535,timeo=1200,actimeo=120"<br>          nfs_shares_config: /etc/cinder/nfs_shares_volume3<br>          shares:<br>          - { ip: "172.29.244.27", share: "/NFS_VOLUME3" }<br></span></div><div><br></div><div>Any advice would be greatly appreciated.  <br></div><div><br></div><div>Thanks.</div><div><br></div><div>-Dave<br></div><div><br></div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div>--</div><div>Dave Hall<br>Binghamton University<br><a href="mailto:kdhall@binghamton.edu" target="_blank">kdhall@binghamton.edu</a><br><br></div></div></div></div></div>