Le 15 nov. 2023 à 17:35, Buddhika S. Godakuru - University of Kelaniya <bsanjeewa@kln.ac.lk> a écrit :
Dear Franck,
For me, I needed a custom /etc/kolla/config/cinder.conf for ceph and cinder to work together.
Since I am using kayobe, and two named backends, this may not be your requirement.
But here're my cinder.conf
My /etc/kolla/config/cinder.conf is,
cat /etc/kolla/config/cinder.conf
# Ansible managed
#######################
# Extra configuration
#######################
[DEFAULT]
enabled_backends = ceph-ssd, ceph-hdd
glance_api_version = 2
[ceph-ssd]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
volume_backend_name = ceph-ssd
rbd_pool = volumes-ssd
rbd_ceph_conf = /etc/ceph/ceph.conf
rados_connect_timeout = 5
rbd_user = cinder
rbd_secret_uuid = <use cinder_rbd_secret_uuid value from /etc/kolla/passwords.yml or /etc/kayobe/kolla/passwords.yml>
report_discard_supported = True
[ceph-hdd]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
volume_backend_name = ceph-hdd
rbd_pool = volumes-hdd
rbd_ceph_conf = /etc/ceph/ceph.conf
rados_connect_timeout = 5
rbd_user = cinder
rbd_secret_uuid =
<use cinder_rbd_secret_uuid value from /etc/kolla/passwords.yml or /etc/kayobe/kolla/passwords.yml>
report_discard_supported = True
and my /etc/kayobe/kolla/config/cinder.conf is
cat /etc/kayobe/kolla/config/cinder.conf
[DEFAULT]
enabled_backends = ceph-ssd, ceph-hdd
glance_api_version = 2
[ceph-ssd]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
volume_backend_name = ceph-ssd
rbd_pool = volumes-ssd
rbd_ceph_conf = /etc/ceph/ceph.conf
rados_connect_timeout = 5
rbd_user = cinder
rbd_secret_uuid =
<use cinder_rbd_secret_uuid value from /etc/kolla/passwords.yml or /etc/kayobe/kolla/passwords.yml>
report_discard_supported = True
[ceph-hdd]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
volume_backend_name = ceph-hdd
rbd_pool = volumes-hdd
rbd_ceph_conf = /etc/ceph/ceph.conf
rados_connect_timeout = 5
rbd_user = cinder
rbd_secret_uuid =
<use cinder_rbd_secret_uuid value from /etc/kolla/passwords.yml or /etc/kayobe/kolla/passwords.yml>
report_discard_supported = True
Hope this helps
Hi Pierre,
thanks for your help.
some informations about my lab:
If I try this, from open stack :
rbd -m ceph1 -n cinder.glance -k /home/user1/ceph.client.cinder.keyring create images/test --size 1G
and after
rbd -m ceph1 -n client.cinder -k /home/user1/ceph.client.cinder.keyring ls images
I can see:
test
I think it’s ok.
Logs from cinder-schedule:
2023-11-15 08:36:30.965 7 ERROR cinder.scheduler.flows.create_volume [req-8d476977-656b-48da-b03e-87ad193fe73d req-e0109262-976c-4afc-b0d3-b7d272f1a15b 9078f3f29a36400c8b217585b71b4e07 6bf25257fa744028b808f2aa5d261e7d - - - -] Failed to run task cinder.scheduler.flows.create_volume.ScheduleCreateVolumeTask;volume:create: No valid backend was found. Exceeded max scheduling attempts 3 for resource 6b7c2adb-9c05-4697-8768-5871b22e636b: cinder.exception.NoValidBackend: No valid backend was found. Exceeded max scheduling attempts 3 for resource 6b7c2adb-9c05-4697-8768-5871b22e636b
Logs from cinder_volume
2023-11-15 08:11:14.354 1018 INFO cinder.volume.manager [None req-c295a329-03af-492a-b5b7-9ac2fa913667 - - - - - -] Initializing RPC dependent components of volume driver RBDDriver (1.3.0)
2023-11-15 08:11:14.411 1018 INFO cinder.volume.manager [None req-c295a329-03af-492a-b5b7-9ac2fa913667 - - - - - -] Driver post RPC initialization completed successfully.
tree /etc/kolla/config
/etc/kolla/config
├── cinder
│ ├── ceph.client.cinder.keyring
│ ├── ceph.conf
│ ├── cinder-backup
│ │ └── ceph.client.cinder.keyring
│ └── cinder-volume
│ ├── ceph.client.cinder.keyring
│ └── ceph.conf
├── glance
│ ├── ceph.client.glance.keyring
│ └── ceph.conf
└── nova
├── ceph.client.cinder.keyring
├── ceph.client.nova.keyring
└── ceph.conf
ceph/conf:
[global]
fsid = 7bcf95a2-7eef-11ee-a183-080027950478
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
I'm probably forgetting something but I don't see what
Hi Franck,
It would help if you could share more details about the error (check both cinder-scheduler and cinder-volume logs).
Do you have capabilities to access the images pool (presumably what is used by Glance) on your client.cinder user?
Best wishes,
Pierre Riteau (priteau)
Good morning,
I'll come back for help, thanks in advance.
I am testing an Openstack 2023.1/Ceph Pacific environment.
The cluster works, and from Horizon for example, if I create a simple volume, it works normally, it is placed in the "volumes" pool in the Ceph cluster, exactly as expected.
if I create a volume from an image, ERROR cinder-schedule.
In the cinder_volume and cinder_scheduler containers, same settings in cinder.conf.
Before continuing to find out what the error could be, could there be a problem between Antelope (2023.1) and Ceph Pacific? (I'm on Ubuntu 22.04).
--
බුද්ධික සංජීව ගොඩාකුරු
Buddhika Sanjeewa Godakuru
Systems Analyst/Programmer
Deputy Webmaster / University of Kelaniya
Information and Communication Technology Centre (ICTC)
University of Kelaniya, Sri Lanka,
Kelaniya,
Sri Lanka.
Mobile : (+94) 071 5696981
Office : (+94) 011 2903420 / 2903424
****
University of Kelaniya, Sri Lanka, accepts no liability for the
content of this email or for the consequences of any actions taken on
the basis of the information provided unless that information is
subsequently confirmed in writing.
If you are not the intended
recipient, this email and/or any information it contains should not be
copied, disclosed, retained or used by you or any other party, and the
email and all its contents should be promptly deleted fully, and the sender informed.
E-mail transmissions cannot be guaranteed to be secure or error-free
as information could be intercepted, corrupted, lost, destroyed, arrive
late or incomplete.
****