Hello Everyone,

I have a really strange problem with  iSCSI, Cinder, Kolla setup.

1. What I want to do:
I did setup with kolla 1 Controller, 4 Compute nodes and 2 storage nodes (iSCSI, Cinder, LVM).
Kolla Deploying works fine, all Machines can find all others.

2. Problem: happens when I try to create an instance

• Im creating an new instance what fails because the cinder volume couldn't be "mounted" on the compute node
• What I checked then:
• storage2 (the 2. cinder iscsi storage node) is available (by ping ) from the compute node
  
• Port 3260 on storage2 is not reachable from compute node
  
• so checking storage2
• the volume was created on LVM (checked with lvs)
• Port 3260 is not available local too
  
• it looks like that cinder creates an iSCSI target and deletes it after some seconds ... no reason shown (I activated debug mode in Kolla in the hope to see more but no ...)
  

Did anyone have some similar behavior or can give me a hint where else to look ?

Thanks a lot for your help !

best

Alex
 

Logfile of the storage2 node /var/log/kolla/cinder/cinder-volume.log:

the following lines came 5-6 times and then nothing more.

2025-03-03 08:17:02.625 42 DEBUG cinder.volume.drivers.lvm [None req-c3be57e8-eeac-4b14-99c7-f3f10474a3cf - - - - - -] Updating volume stats _update_volume_stats /var/lib/kolla/venv/lib/python3.12/site-packages/cinder/volume/drivers/lvm.py:227
2025-03-03 08:17:02.626 42 DEBUG oslo_concurrency.processutils [None req-c3be57e8-eeac-4b14-99c7-f3f10474a3cf - - - - - -] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C vgs --noheadings --unit=g -o name,size,free,lv_count,uuid --separator :
--nosuffix cinder-volumes execute /var/lib/kolla/venv/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349
2025-03-03 08:17:03.057 42 DEBUG oslo_concurrency.processutils [None req-c3be57e8-eeac-4b14-99c7-f3f10474a3cf - - - - - -] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C vgs --noheadings --unit=g -o name,size,free,lv_count,uuid --separator : --nosuffix cinder-vo
lumes" returned: 0 in 0.432s execute /var/lib/kolla/venv/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372
2025-03-03 08:17:03.059 42 DEBUG oslo_concurrency.processutils [None req-c3be57e8-eeac-4b14-99c7-f3f10474a3cf - - - - - -] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvs --noheadings --unit=g -o vg_name,name,size --nosuffix --readonly ci
nder-volumes execute /var/lib/kolla/venv/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349
2025-03-03 08:17:03.483 42 DEBUG oslo_concurrency.processutils [None req-c3be57e8-eeac-4b14-99c7-f3f10474a3cf - - - - - -] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvs --noheadings --unit=g -o vg_name,name,size --nosuffix --readonly cinder-volumes" returne
d: 0 in 0.425s execute /var/lib/kolla/venv/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372
2025-03-03 08:17:03.485 42 DEBUG oslo_concurrency.processutils [None req-c3be57e8-eeac-4b14-99c7-f3f10474a3cf - - - - - -] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvs --noheadings --unit=g -o size,data_percent --separator : --nosuffix
/dev/cinder-volumes/cinder-volumes-pool execute /var/lib/kolla/venv/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349
2025-03-03 08:17:03.899 42 DEBUG oslo_concurrency.processutils [None req-c3be57e8-eeac-4b14-99c7-f3f10474a3cf - - - - - -] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvs --noheadings --unit=g -o size,data_percent --separator : --nosuffix /dev/cinder-volumes/
cinder-volumes-pool" returned: 0 in 0.414s execute /var/lib/kolla/venv/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372
2025-03-03 08:17:03.900 42 DEBUG oslo_concurrency.processutils [None req-c3be57e8-eeac-4b14-99c7-f3f10474a3cf - - - - - -] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvs --noheadings --unit=g -o vg_name,name,size --nosuffix --readonly ci
nder-volumes execute /var/lib/kolla/venv/lib/python3.12/site-packages/oslo_concurrency/processutils.py:349
2025-03-03 08:17:04.319 42 DEBUG oslo_concurrency.processutils [None req-c3be57e8-eeac-4b14-99c7-f3f10474a3cf - - - - - -] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C lvs --noheadings --unit=g -o vg_name,name,size --nosuffix --readonly cinder-volumes" returne
d: 0 in 0.418s execute /var/lib/kolla/venv/lib/python3.12/site-packages/oslo_concurrency/processutils.py:372
2025-03-03 08:17:04.320 42 DEBUG cinder.manager [None req-c3be57e8-eeac-4b14-99c7-f3f10474a3cf - - - - - -] Notifying Schedulers of capabilities ... _publish_service_capabilities /var/lib/kolla/venv/lib/python3.12/site-packages/cinder/manager.py:202
2025-03-03 08:18:02.626 42 DEBUG oslo_service.periodic_task [None req-c3be57e8-eeac-4b14-99c7-f3f10474a3cf - - - - - -] Running periodic task VolumeManager.publish_service_capabilities run_periodic_tasks /var/lib/kolla/venv/lib/python3.12/site-packages/oslo_service/periodic_task.
py:210