Also another thing, cat ./ceph-ansible/group_vars/osds.yml
looks that have not been modified over last re-deployments. delete'ing it again and removing config-download and everything from swift...

I do not like it do not override everything... especially when launching deployment, when there is no stack (I mean in undercloud host, as overcloud nodes should be cleaned up by undercloud).

Thank you, will keep updated.

On Mon, 21 Sep 2020 at 19:33, Ruslanas Gžibovskis <ruslanas@lpic.lt> wrote:
I have one thought.

stack@undercloudv3 v3]$ cat /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml  
resource_registry:
 OS::TripleO::Services::CephMgr: ../../deployment/ceph-ansible/ceph-mgr.yaml
 OS::TripleO::Services::CephMon: ../../deployment/ceph-ansible/ceph-mon.yaml
 OS::TripleO::Services::CephOSD: ../../deployment/ceph-ansible/ceph-osd.yaml
 OS::TripleO::Services::CephClient: ../../deployment/ceph-ansible/ceph-client.yaml

parameter_defaults:
 # Ensure that if user overrides CephAnsiblePlaybook via some env
 # file, we go back to default when they stop passing their env file.
 CephAnsiblePlaybook: ['default']

 CinderEnableIscsiBackend: false
 CinderEnableRbdBackend: true
 CinderBackupBackend: ceph
 NovaEnableRbdBackend: true
 GlanceBackend: rbd
 ## Uncomment below if enabling legacy telemetry
 # GnocchiBackend: rbd
[stack@undercloudv3 v3]$


And my deploy has:
   -e ${_THT}/environments/ceph-ansible/ceph-ansible.yaml \
   -e ${_THT}/environments/ceph-ansible/ceph-rgw.yaml \
   -e ${_THT}/environments/ceph-ansible/ceph-mds.yaml \
   -e ${_THT}/environments/ceph-ansible/ceph-dashboard.yaml \

generally the same files, BUT, they are specified by user, and it "might feel like" the user overwrote default settings?

Also I am thinking on the things you helped me tho find, John. And I recalled, what I have found strange. NFS part.
That it was trying to configure CephNfs... Or it should even I do not have it specified? From the output [1] here is the small part of it:
        "statically imported: /usr/share/ceph-ansible/roles/ceph-nfs/tasks/create_rgw_nfs_user.yml",
        "statically imported: /usr/share/ceph-ansible/roles/ceph-nfs/tasks/ganesha_selinux_fix.yml",
        "statically imported: /usr/share/ceph-ansible/roles/ceph-nfs/tasks/start_nfs.yml",





--
Ruslanas Gžibovskis
+370 6030 7030