[Openstack] Integrating Ceph Jewel as Storage Backend for OpenStack Newton
Esteban Freire
caroubans at yahoo.es
Thu Mar 2 09:13:11 UTC 2017
Hello all,
I am testing OpenStack Newton on CentOS 7 and now I already have a OpenStack cloud infrastructure working, I would like to integrate Ceph with Cinder, Glance and Nova services.
I have found some information about how to performance this on previous releases for Openstack and Ceph:
http://docs.ceph.com/docs/jewel/rbd/rbd-openstack/
And I tried to update the variables according to https://docs.openstack.org/newton/config-reference/block-storage/drivers/ceph-rbd-volume-driver.html
This is my current glance-api.conf and which is working:
[glance_store]
stores = file,http
default_store = file
filesystem_store_datadir = /var/lib/glance/images/
At this is what I tried (at the moment, I have only tried it with the glance service but I did not get success):
* Install python-rbd and python-rados from centos-ceph-jewel repo on the controller node.
* Create a ceph user and add it to sudoers.
* On ceph admin node:
sudo ceph osd pool create images 150
sudo ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=images'
sudo ceph auth get-or-create client.glance | ssh ceph at controller-node1 sudo tee /etc/ceph/ceph.client.glance.keyring
ssh ceph at controller-node1 sudo chown glance:glance /etc/ceph/ceph.client.glance.keyring
* On the controller node, I edited the glance-api.conf file with the following variables:
[glance_store]
stores = file,http,rbd
default_store = rbd
rbd_pool = images
rbd_user = glance
rbd_ceph_conf = /etc/ceph/ceph.conf
rbd_store_chunk_size = 8
(!) I also have tried with stores = rbd but without success.
* And restart the service, systemctl restart openstack-glance-api
But when I try to create a new image, I get the following issue:
[openstackadmin at controller-node1 ~]$ openstack image create "cirros ceph" --file /home/openstackadmin/cirros-0.3.4-x86_64-disk.raw --disk-format raw --container-format bare --public
500 Internal Server Error
The server has either erred or is incapable of performing the requested operation.
(HTTP 500)
Is there any documentation about how to integrate Ceph jewel with OpenStack newton (Cinder, Glance and Nova services)? If it is so, could you please provide me the link?
On the other hand, Is there any way to chose the store when creating an image? I mean, to choose for example if save the image on /var/lib/glance/images/ or on Ceph.
I would appreciate if you could help me to set up this integration.
This is the most relevant info I can see on the logs and as far as I see from it, I have a permissions error but I am not sure what I need to modify. It is my first installation with OpenStack by the way and I am trying at home to see how it works :)
{{{
/var/log/glance/api.log
2017-03-01 23:43:41.419 4197 INFO eventlet.wsgi.server [req-eea55909-c963-4158-b30c-3f2779fd78c6 c41043a1ddc14ffba1b45c0a3287e0bf b2d1547f0e734f87a84feea75ccd6453 - default default] 192.168.56.2 - - [01/Mar/2017 23:43:41] "GET /v2/schemas/image HTTP/1.1" 200 4352 0.421281
2017-03-01 23:43:41.570 4197 INFO eventlet.wsgi.server [req-f5ef0b43-0ddf-4698-8459-2f75fb1822a3 c41043a1ddc14ffba1b45c0a3287e0bf b2d1547f0e734f87a84feea75ccd6453 - default default] 192.168.56.2 - - [01/Mar/2017 23:43:41] "POST /v2/images HTTP/1.1" 201 859 0.110681
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data [req-98c406fb-7bc0-47de-bace-2b0a2097b699 c41043a1ddc14ffba1b45c0a3287e0bf b2d1547f0e734f87a84feea75ccd6453 - default default] Failed to upload image data due to internal error
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data Traceback (most recent call last):
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 115, in upload
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data image.set_data(data, size)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 195, in set_data
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data self.base.set_data(data, size)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 479, in set_data
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data _send_notification(notify_error, 'image.upload', msg)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data self.force_reraise()
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data six.reraise(self.type_, self.value, self.tb)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 426, in set_data
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data self.repo.set_data(data, size)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 185, in set_data
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data return self.image.set_data(*args, **kwargs)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line 304, in set_data
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data self.image.set_data(data, size=size)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance/location.py", line 440, in set_data
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data verifier=verifier)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 447, in add_to_backend
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data verifier)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 420, in store_add_to_backend
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data verifier=verifier)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance_store/capabilities.py", line 225, in op_checker
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data return store_op_fun(store, *args, **kwargs)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 447, in add
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data rados_id=self.user) as conn:
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data return self.gen.next()
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 270, in get_connection
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data client = rados.Rados(conffile=conffile, rados_id=rados_id)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "rados.pyx", line 525, in rados.Rados.__init__ (rados.c:5719)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "rados.pyx", line 425, in rados.requires.wrapper.validate_func (rados.c:4106)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "rados.pyx", line 568, in rados.Rados.__setup (rados.c:6448)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "rados.pyx", line 425, in rados.requires.wrapper.validate_func (rados.c:4106)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "rados.pyx", line 631, in rados.Rados.conf_read_file (rados.c:7384)
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data PermissionDeniedError: error calling conf_read_file
2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi [req-98c406fb-7bc0-47de-bace-2b0a2097b699 c41043a1ddc14ffba1b45c0a3287e0bf b2d1547f0e734f87a84feea75ccd6453 - default default] Caught error: error calling conf_read_file
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi Traceback (most recent call last):
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1113, in __call__
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi request, **action_args)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1152, in dispatch
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi return method(*args, **kwargs)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 363, in wrapped
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi return func(self, req, *args, **kwargs)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 250, in upload
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi self._restore(image_repo, image)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi self.force_reraise()
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 115, in upload
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi image.set_data(data, size)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 195, in set_data
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi self.base.set_data(data, size)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 479, in set_data
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi _send_notification(notify_error, 'image.upload', msg)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi self.force_reraise()
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 426, in set_data
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi self.repo.set_data(data, size)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 185, in set_data
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi return self.image.set_data(*args, **kwargs)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line 304, in set_data
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi self.image.set_data(data, size=size)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/location.py", line 440, in set_data
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi verifier=verifier)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 447, in add_to_backend
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi verifier)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 420, in store_add_to_backend
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi verifier=verifier)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/capabilities.py", line 225, in op_checker
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi return store_op_fun(store, *args, **kwargs)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 447, in add
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi rados_id=self.user) as conn:
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi return self.gen.next()
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 270, in get_connection
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi client = rados.Rados(conffile=conffile, rados_id=rados_id)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "rados.pyx", line 525, in rados.Rados.__init__ (rados.c:5719)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "rados.pyx", line 425, in rados.requires.wrapper.validate_func (rados.c:4106)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "rados.pyx", line 568, in rados.Rados.__setup (rados.c:6448)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "rados.pyx", line 425, in rados.requires.wrapper.validate_func (rados.c:4106)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "rados.pyx", line 631, in rados.Rados.conf_read_file (rados.c:7384)
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi PermissionDeniedError: error calling conf_read_file
2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi
}}}
Thanks in advance,
Esteban
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20170302/4ec1b057/attachment.html>
More information about the Openstack
mailing list