<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Apr 12, 2023 at 10:41 AM wodel youchi <<a href="mailto:wodel.youchi@gmail.com">wodel.youchi@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>I am trying to configure glance to use cinder as a backend.</div><div><br></div><div>This is my glance-api.conf <br></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">[cinder]
</span><br>cinder_store_auth_address = <a href="https://dashint.example.com:5000/v3" target="_blank">https://dashint.example.com:5000/v3</a>
<br>cinder_store_user_name = cinder
<br>cinder_store_password = cinderpass
<br>cinder_store_project_name = service
<br>cinder_volume_type = nfstype
<br>rootwrap_config = /etc/glance/rootwrap.conf<br></span></div><div><br></div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">==> /var/log/kolla/glance/glance-api.log <==
</span><br>2023-04-12 18:02:20.842 64 INFO oslo.privsep.daemon [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 b0f76b5c6dcb457fa716762bbf954837 - default default] Spawned new privsep daemon via rootwrap
<br>2023-04-12 18:02:20.733 360 INFO oslo.privsep.daemon [-] privsep daemon starting
<br>2023-04-12 18:02:20.735 360 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
<br><b>2023-04-12 18:02:20.737 360 ERROR oslo.privsep.daemon [-] [Errno 1] Operation not permitted
<br>Traceback (most recent call last):
</b><br> File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 584, in helper_main
<br> Daemon(channel, context).run()
<br> File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 394, in run
<br> self._drop_privs()
<br> File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 430, in _drop_privs
<br> capabilities.drop_all_caps_except(self.caps, self.caps, [])
<br> File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/capabilities.py", line 156, in drop_all_caps_except
<br> raise OSError(errno, os.strerror(errno))
<br>PermissionError: [Errno 1] Operation not permitted
<br>2023-04-12 18:02:20.844 64 WARNING oslo_privsep.comm [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 b0f76b5c6dcb457fa716762bbf954837 - default <br>default] Unexpected error: <class 'BrokenPipeError'>: BrokenPipeError: [Errno 32] Broken pipe
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 b0f76b5c6dcb457fa716762bbf954837 - default <br>default] Error while sending initial PING to privsep: [Errno 32] Broken pipe: BrokenPipeError: [Errno 32] Broken pipe
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon Traceback (most recent call last):
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 192, in exchange_ping
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon reply = self.send_recv((comm.Message.PING.value,))
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/comm.py", line 186, in send_recv
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon self.writer.send((myid, msg))
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/comm.py", line 60, in send
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon self.writesock.sendall(buf)
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon File "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/greenio/base.py", line 407, in sendall
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon tail = self.send(data, flags)
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon File "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/greenio/base.py", line 401, in send
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon return self._send_loop(self.fd.send, data, flags)
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon File "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/greenio/base.py", line 388, in _send_loop
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon return send_method(data, *args)
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon BrokenPipeError: [Errno 32] Broken pipe
<br>2023-04-12 18:02:20.844 64 ERROR oslo.privsep.daemon <br>2023-04-12 18:02:20.846 64 CRITICAL oslo.privsep.daemon [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 b0f76b5c6dcb457fa716762bbf954837 - defau<br>lt default] Privsep daemon failed to start
<br>2023-04-12 18:02:20.847 64 ERROR glance_store._drivers.cinder [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 b0f76b5c6dcb457fa716762bbf954837 -<br> default default] Failed to write to volume 46316c12-6c24-40af-afde-1c16edd616b6.: oslo_privsep.daemon.FailedToDropPrivileges: Privsep daemon failed to start
<br>2023-04-12 18:02:20.890 64 ERROR glance.api.v2.image_data [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 b0f76b5c6dcb457fa716762bbf954837 - def<br>ault default] <b>Failed to upload image data due to internal error: oslo_privsep.daemon.FailedToDropPrivileges: Privsep daemon failed to start
</b><br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 b0f76b5c6dcb457fa716762bbf954837 - default d<br>efault] Caught error: Privsep daemon failed to start: oslo_privsep.daemon.FailedToDropPrivileges: Privsep daemon failed to start
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi Traceback (most recent call last):
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/common/wsgi.py", line 1332, in __call__
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi request, **action_args)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/common/wsgi.py", line 1370, in dispatch
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi return method(*args, **kwargs)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/common/utils.py", line 414, in wrapped
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi return func(self, req, *args, **kwargs)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/api/v2/image_data.py", line 303, in upload
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi self._restore(image_repo, image)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi self.force_reraise()
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi raise self.value
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/api/v2/image_data.py", line 163, in upload
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi image.set_data(data, size, backend=backend)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/notifier.py", line 497, in set_data
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi _send_notification(notify_error, 'image.upload', msg)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi self.force_reraise()
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi raise self.value
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/notifier.py", line 444, in set_data
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi set_active=set_active)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/quota/__init__.py", line 323, in set_data
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi set_active=set_active)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/location.py", line 585, in set_data
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi self._upload_to_store(data, verifier, backend, size)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance/location.py", line 485, in _upload_to_store
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi verifier=verifier)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance_store/multi_backend.py", line 399, in add_with_multihash
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi image_id, data, size, hashing_algo, store, context, verifier)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance_store/multi_backend.py", line 481, in store_add_to_backe<br>nd_with_multihash
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi image_id, data, size, hashing_algo, context=context, verifier=verifier)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance_store/driver.py", line 279, in add_adapter
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi metadata_dict) = store_add_fun(*args, **kwargs)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance_store/capabilities.py", line 176, in op_checker
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi return store_op_fun(store, *args, **kwargs)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance_store/_drivers/cinder.py", line 985, in add
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi with self._open_cinder_volume(client, volume, 'wb') as f:
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/usr/lib64/python3.6/contextlib.py", line 81, in __enter__
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi return next(self.gen)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/glance_store/_drivers/cinder.py", line 739, in _open_cinder_vol<br>ume
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi root_helper, my_ip, use_multipath, enforce_multipath, host=host)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/os_brick/utils.py", line 169, in trace_logging_wrapper
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi return f(*args, **kwargs)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/os_brick/initiator/connector.py", line 240, in get_connector_pr<br>operties
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi execute=execute))
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 70, in get_connec<br>tor_properties
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi initiator = iscsi.get_initiator()
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/os_brick/initiator/connectors/iscsi.py", line 963, in get_initi<br>ator
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi root_helper=self._root_helper)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/os_brick/executor.py", line 53, in _execute
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi result = self.__execute(*args, **kwargs)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/os_brick/privileged/rootwrap.py", line 172, in execute
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi return execute_root(*cmd, **kwargs)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 269, in _wrap
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi self.start()
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 283, in start
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi channel = daemon.RootwrapClientChannel(context=self)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 374, in __init__
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi super(RootwrapClientChannel, self).__init__(sock, context)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 187, in __init__
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi self.exchange_ping()
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 201, in exchange_ping
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi raise FailedToDropPrivileges(msg)
<br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi <b>oslo_privsep.daemon.FailedToDropPrivileges: Privsep daemon failed to start
</b><br>2023-04-12 18:02:20.908 64 ERROR glance.common.wsgi <br>2023-04-12 18:02:20.927 64 INFO eventlet.wsgi.server [req-62883866-1b53-4032-bdc4-d9a294a837c4 0439953e7cfe4a13a1b4bb118b5dc3c4 b0f76b5c6dcb457fa716762bbf954837 - default <br>default] 20.3.0.34,127.0.0.1 - - [12/Apr/2023 18:02:20] "PUT /v2/images/52ed7ed7-330e-4249-abb9-5ec99712846f/file HTTP/1.1" 500 430 2.727683</span></div></blockquote><div><span style="font-family:monospace"><br></span></div><div><span style="font-family:monospace">It seems like a lack of privileges, any ideas?<br></span></div></div></blockquote><div><br></div><div>Yes, the glance-api container itself must run with "privileged: true" when glance is using cinder for a backend. For reference, you can see how TripleO handles this:</div><div><br></div><div><a href="https://github.com/openstack/tripleo-heat-templates/blob/2e6d826debd6099b3d85d0268430541b01560139/deployment/glance/glance-api-container-puppet.yaml#L790">https://github.com/openstack/tripleo-heat-templates/blob/2e6d826debd6099b3d85d0268430541b01560139/deployment/glance/glance-api-container-puppet.yaml#L790</a></div><div><br></div><div>Alan<br></div><div><br></div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><span style="font-family:monospace"></span></div><div><span style="font-family:monospace"><br></span></div><div><span style="font-family:monospace">Regards.</span></div><div><span style="font-family:monospace"><br></span></div></div>
</blockquote></div></div>