Hi. I'm installing Cinder service in an storage server of my laboratory and I need you guys' help about Cinder. I'm installing Cinder by referring 'Cinder Installation Guide for Ubuntu' ( https://docs.openstack.org/cinder/queens/install/index-ubuntu.html). However, when I request creating an instance, it seems that Openstack cannot receive the volume which the instance needs from the storage server. And seeing status message of tgt service for iSCSI, it seems that the service needs certain kernel module. Status of tgt service is following:
● tgt.service - (i)SCSI target daemon Loaded: loaded (/lib/systemd/system/tgt.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2018-12-19 22:05:55 KST; 26min ago Docs: man:tgtd(8) Process: 2249 ExecStartPost=/usr/sbin/tgtadm --op update --mode sys --name State -v ready (code=exited, status=0/SUCCESS) Process: 2230 ExecStartPost=/usr/sbin/tgt-admin -e -c /etc/tgt/targets.conf (code=exited, status=0/SUCCESS) Process: 2206 ExecStartPost=/usr/sbin/tgtadm --op update --mode sys --name State -v offline (code=exited, status=0/SUCCESS) Main PID: 2170 (tgtd) Status: "Starting event loop..." Tasks: 1 Memory: 2.7M CPU: 87ms CGroup: /system.slice/tgt.service └─2170 /usr/sbin/tgtd -f Dec 19 22:05:55 node0 systemd[1]: Starting (i)SCSI target daemon... Dec 19 22:05:55 node0 tgtd[2170]: tgtd: iser_ib_init(3434) Failed to initialize RDMA; load kernel modules? Dec 19 22:05:55 node0 tgtd[2170]: tgtd: work_timer_start(150) use signal based scheduler Dec 19 22:05:55 node0 tgtd[2170]: tgtd: bs_init(387) use signalfd notification Dec 19 22:05:55 node0 systemd[1]: Started (i)SCSI target daemon.
And the status of Cinder-volume service as follows:
● cinder-volume.service - OpenStack Cinder Volume Loaded: loaded (/lib/systemd/system/cinder-volume.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2018-12-19 22:05:55 KST; 45min ago Main PID: 2110 (cinder-volume) Tasks: 2 Memory: 210.2M CPU: 1min 23.649s CGroup: /system.slice/cinder-volume.service ├─2110 /usr/bin/python2 /usr/bin/cinder-volume --config-file=/etc/cinder/cinder.conf --log-file=/var/log/cinder/cinder-volume.log └─2657 /usr/bin/python2 /usr/bin/cinder-volume --config-file=/etc/cinder/cinder.conf --log-file=/var/log/cinder/cinder-volume.log Dec 19 22:05:55 node0 systemd[1]: Started OpenStack Cinder Volume. Dec 19 22:06:00 node0 sudo[2658]: pam_unix(sudo:auth): auth could not identify password for [cinder]
Log message of Cinder-volume service continuously complains that the service cannot communicate with the internal driver (maybe, LVM?). In other words, when Cinder-volume is accessed (e.g. instance creation), an error message is printed and after that, the same message is repeated every second as follows: 2018-12-19 22:04:00.747 35996 INFO oslo_service.service
[req-bcc39e60-531e-4484-ab4b-bab085d112c4 - - - - -] Caught SIGTERM, stopping children 2018-12-19 22:04:00.784 35996 INFO oslo_service.service [req-bcc39e60-531e-4484-ab4b-bab085d112c4 - - - - -] Waiting on 1 children to exit 2018-12-19 22:04:00.809 35996 INFO oslo_service.service [req-bcc39e60-531e-4484-ab4b-bab085d112c4 - - - - -] Child 36039 killed by signal 15 2018-12-19 22:05:58.378 2110 INFO root [-] Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt 2018-12-19 22:05:58.403 2110 INFO root [-] Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt 2018-12-19 22:05:59.820 2110 INFO cinder.rpc [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Automatically selected cinder-scheduler objects version 1.35 as minimum service version. 2018-12-19 22:05:59.826 2110 INFO cinder.rpc [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Automatically selected cinder-scheduler RPC version 3.10 as minimum service version. 2018-12-19 22:05:59.875 2110 INFO cinder.volume.manager [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Determined volume DB was empty at startup. 2018-12-19 22:05:59.908 2110 WARNING oslo_config.cfg [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Option "iscsi_helper" from group "lvm" is deprecated. Use option "target_helper" from group "lvm". 2018-12-19 22:05:59.914 2110 WARNING oslo_config.cfg [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Option "iscsi_protocol" from group "lvm" is deprecated. Use option "target_protocol" from group "lvm". 2018-12-19 22:05:59.916 2110 INFO cinder.volume.manager [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Image-volume cache disabled for host node0@lvm. 2018-12-19 22:05:59.932 2110 INFO oslo_service.service [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Starting 1 workers 2018-12-19 22:05:59.940 2657 INFO cinder.service [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Starting cinder-volume node (version 13.0.0) 2018-12-19 22:05:59.942 2110 WARNING oslo_config.cfg [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Option "iscsi_helper" from group "DEFAULT" is deprecated. Use option "target_helper" from group "DEFAULT". 2018-12-19 22:05:59.952 2110 WARNING oslo_config.cfg [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Option "auth_uri" from group "keystone_authtoken" is deprecated for removal (The auth_uri option is deprecated in favor of www_authenticate_uri and will be removed in the S release.). Its value may be silently ignored in the future. 2018-12-19 22:05:59.953 2110 WARNING oslo_config.cfg [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Option "auth_uri" from group "keystone_authtoken" is deprecated. Use option "www_authenticate_uri" from group "keystone_authtoken". 2018-12-19 22:05:59.961 2657 INFO cinder.volume.manager [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Starting volume driver LVMVolumeDriver (3.0.0) 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Failed to initialize driver.: ProcessExecutionError: Unexpected error while running command. Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C vgs --version Exit code: 1 Stdout: u'' Stderr: u'sudo: no tty present and no askpass program specified\n' 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager Traceback (most recent call last): 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/manager.py", line 433, in init_host 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager self.driver.check_for_setup_error() 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/drivers/lvm.py", line 286, in check_for_setup_error 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager if volutils.supports_thin_provisioning(): 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/volume/utils.py", line 628, in supports_thin_provisioning 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager utils.get_root_helper()) 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/brick/local_dev/lvm.py", line 225, in supports_thin_provisioning 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager return LVM.get_lvm_version(root_helper) >= (2, 2, 95) 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager File "/opt/stack/cinder/cinder/brick/local_dev/lvm.py", line 202, in get_lvm_version 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager run_as_root=True) 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 424, in execute 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager cmd=sanitized_cmd) 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager ProcessExecutionError: Unexpected error while running command. 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C vgs --version 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager Exit code: 1 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager Stdout: u'' 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager Stderr: u'sudo: no tty present and no askpass program specified\n' 2018-12-19 22:06:00.007 2657 ERROR cinder.volume.manager 2018-12-19 22:06:00.075 2657 INFO cinder.volume.manager [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Initializing RPC dependent components of volume driver LVMVolumeDriver (3.0.0) 2018-12-19 22:06:00.076 2657 ERROR cinder.utils [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Volume driver LVMVolumeDriver not initialized 2018-12-19 22:06:00.076 2657 ERROR cinder.volume.manager [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: DriverNotInitialized: Volume driver not ready. 2018-12-19 22:06:10.077 2657 ERROR cinder.service [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Manager for service cinder-volume node0@lvm is reporting problems, not sending heartbeat. Service will appear "down". 2018-12-19 22:06:20.077 2657 ERROR cinder.service [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Manager for service cinder-volume node0@lvm is reporting problems, not sending heartbeat. Service will appear "down". 2018-12-19 22:06:30.078 2657 ERROR cinder.service [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Manager for service cinder-volume node0@lvm is reporting problems, not sending heartbeat. Service will appear "down". 2018-12-19 22:06:40.078 2657 ERROR cinder.service [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Manager for service cinder-volume node0@lvm is reporting problems, not sending heartbeat. Service will appear "down". 2018-12-19 22:06:50.079 2657 ERROR cinder.service [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Manager for service cinder-volume node0@lvm is reporting problems, not sending heartbeat. Service will appear "down". 2018-12-19 22:07:00.079 2657 ERROR cinder.service [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Manager for service cinder-volume node0@lvm is reporting problems, not sending heartbeat. Service will appear "down". 2018-12-19 22:07:10.078 2657 WARNING cinder.volume.manager [req-b3178448-4e17-489b-8267-2fd7992bc876 - - - - -] Update driver status failed: (config name lvm) is uninitialized.
In above log, 'Manage for service cinder -volume node0@lvm is ...' is the repeated part. If there is anyone who has experienced this kind of problem, please give me any advice. Thanks!