Dear Franck,
For me, I needed a custom /etc/kolla/config/cinder.conf for ceph and cinder to work together.
Since I am using kayobe, and two named backends, this may not be your requirement.

But here're my cinder.conf

My /etc/kolla/config/cinder.conf is,


cat /etc/kolla/config/cinder.conf
# Ansible managed

#######################
# Extra configuration
#######################

[DEFAULT]
enabled_backends = ceph-ssd, ceph-hdd
glance_api_version = 2

[ceph-ssd]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
volume_backend_name = ceph-ssd
rbd_pool = volumes-ssd
rbd_ceph_conf = /etc/ceph/ceph.conf
rados_connect_timeout = 5
rbd_user = cinder
rbd_secret_uuid = <use cinder_rbd_secret_uuid value from /etc/kolla/passwords.yml or /etc/kayobe/kolla/passwords.yml>
report_discard_supported = True

[ceph-hdd]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
volume_backend_name = ceph-hdd
rbd_pool = volumes-hdd
rbd_ceph_conf = /etc/ceph/ceph.conf
rados_connect_timeout = 5
rbd_user = cinder
rbd_secret_uuid =  <use cinder_rbd_secret_uuid value from /etc/kolla/passwords.yml or /etc/kayobe/kolla/passwords.yml>
report_discard_supported = True

and my /etc/kayobe/kolla/config/cinder.conf is


cat /etc/kayobe/kolla/config/cinder.conf
[DEFAULT]
enabled_backends = ceph-ssd, ceph-hdd
glance_api_version = 2

[ceph-ssd]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
volume_backend_name = ceph-ssd
rbd_pool = volumes-ssd
rbd_ceph_conf = /etc/ceph/ceph.conf
rados_connect_timeout = 5
rbd_user = cinder
rbd_secret_uuid =  <use cinder_rbd_secret_uuid value from /etc/kolla/passwords.yml or /etc/kayobe/kolla/passwords.yml>
report_discard_supported = True

[ceph-hdd]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
volume_backend_name = ceph-hdd
rbd_pool = volumes-hdd
rbd_ceph_conf = /etc/ceph/ceph.conf
rados_connect_timeout = 5
rbd_user = cinder
rbd_secret_uuid =  <use cinder_rbd_secret_uuid value from /etc/kolla/passwords.yml or /etc/kayobe/kolla/passwords.yml>
report_discard_supported = True


Hope this helps

On Wed, 15 Nov 2023 at 18:31, Franck VEDEL <franck.vedel@univ-grenoble-alpes.fr> wrote:
Hi Pierre,
thanks for your help.
some informations about my lab:

If I try this, from open stack :
rbd -m ceph1 -n cinder.glance  -k /home/user1/ceph.client.cinder.keyring create images/test --size 1G

and after
rbd -m ceph1 -n client.cinder  -k /home/user1/ceph.client.cinder.keyring ls images
I can see:
test

I think it’s ok.

Logs from cinder-schedule:
2023-11-15 08:36:30.965 7 ERROR cinder.scheduler.flows.create_volume [req-8d476977-656b-48da-b03e-87ad193fe73d req-e0109262-976c-4afc-b0d3-b7d272f1a15b 9078f3f29a36400c8b217585b71b4e07 6bf25257fa744028b808f2aa5d261e7d - - - -] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for resource 6b7c2adb-9c05-4697-8768-5871b22e636b: cinder.exception.NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for resource 6b7c2adb-9c05-4697-8768-5871b22e636b

Logs from cinder_volume
2023-11-15 08:11:14.354 1018 INFO cinder.volume.manager [None req-c295a329-03af-492a-b5b7-9ac2fa913667 - - - - - -] Initializing RPC dependent components of volume driver RBDDriver (1.3.0)
2023-11-15 08:11:14.411 1018 INFO cinder.volume.manager [None req-c295a329-03af-492a-b5b7-9ac2fa913667 - - - - - -] Driver post RPC initialization completed successfully.

tree /etc/kolla/config

/etc/kolla/config
├── cinder
│   ├── ceph.client.cinder.keyring
│   ├── ceph.conf
│   ├── cinder-backup
│   │   └── ceph.client.cinder.keyring
│   └── cinder-volume
│       ├── ceph.client.cinder.keyring
│       └── ceph.conf
├── glance
│   ├── ceph.client.glance.keyring
│   └── ceph.conf
└── nova
    ├── ceph.client.cinder.keyring
    ├── ceph.client.nova.keyring
    └── ceph.conf

ceph/conf:
[global]
fsid = 7bcf95a2-7eef-11ee-a183-080027950478
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx

I'm probably forgetting something but I don't see what


Franck


Le 15 nov. 2023 à 09:54, Pierre Riteau <pierre@stackhpc.com> a écrit :

Hi Franck,

It would help if you could share more details about the error (check both cinder-scheduler and cinder-volume logs).

Do you have capabilities to access the images pool (presumably what is used by Glance) on your client.cinder user?

Best wishes,
Pierre Riteau (priteau)

On Wed, 15 Nov 2023 at 09:50, Franck VEDEL <franck.vedel@univ-grenoble-alpes.fr> wrote:
Good morning,
I'll come back for help, thanks in advance.
I am testing an Openstack 2023.1/Ceph Pacific environment.
The cluster works, and from Horizon for example, if I create a simple volume, it works normally, it is placed in the "volumes" pool in the Ceph cluster, exactly as expected.

if I create a volume from an image, ERROR cinder-schedule.
In the cinder_volume and cinder_scheduler containers, same settings in cinder.conf.

Before continuing to find out what the error could be, could there be a problem between Antelope (2023.1) and Ceph Pacific? (I'm on Ubuntu 22.04).

Franck






--

බුද්ධික සංජීව ගොඩාකුරු 
Buddhika Sanjeewa Godakuru 

Systems Analyst/Programmer 
Deputy Webmaster / University of Kelaniya

Information and Communication Technology Centre (ICTC) 
University of Kelaniya, Sri Lanka, 
Kelaniya, 
Sri Lanka. 

Mobile : (+94) 071 5696981 
Office : (+94) 011 2903420 / 2903424 

****
University of Kelaniya, Sri Lanka, accepts no liability for the content of this email or for the consequences of any actions taken on the basis of the information provided unless that information is subsequently confirmed in writing. 
If you are not the intended recipient, this email and/or any information it contains should not be copied, disclosed, retained or used by you or any other party, and the email and all its contents should be promptly deleted fully, and the sender informed.
E-mail transmissions cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete.
****