<div dir="ltr"><div><div></div>After loading RBD driver. Maybe my setup is incorrect.<br></div><div>Thanks<br></div><div><br></div>setup and error<br><div><br>rados pools<br><br><div>data<br>metadata<br>rbd<br>images<br>volumes<br>backups<br>vms<br><br><br>cinder-volume log<br><br>2016-03-08 07:39:38.713 7856 INFO cinder.service [-] Starting cinder-volume node (version 7.0.1)<br>2016-03-08 07:39:38.715 7856 INFO cinder.volume.manager [req-6e635315-b503-4e04-874e-53be817244ee - - - - -] Starting volume driver RBDDriver (1.2.0)<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup [req-6bc8b049-aafe-4302-8fe1-457dce30ed0f - - - - -] 'max_avail'<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup Traceback (most recent call last):<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup File "/usr/lib/python2.7/dist-packages/oslo_service/threadgroup.py", line 154, in wait<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup x.wait()<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup File "/usr/lib/python2.7/dist-packages/oslo_service/threadgroup.py", line 51, in wait<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup return self.thread.wait()<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 175, in wait<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup return self._exit_event.wait()<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 121, in wait<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup return hubs.get_hub().switch()<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 294, in switch<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup return self.greenlet.switch()<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup result = function(*args, **kwargs)<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup File "/usr/lib/python2.7/dist-packages/oslo_service/service.py", line 645, in run_service<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup service.start()<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup File "/usr/lib/python2.7/dist-packages/cinder/service.py", line 146, in start<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup self.manager.init_host()<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup return f(*args, **kwargs)<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 378, in init_host<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup self.driver.init_capabilities()<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup return f(*args, **kwargs)<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup File "/usr/lib/python2.7/dist-packages/cinder/volume/driver.py", line 662, in init_capabilities<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup stats = self.get_volume_stats(True)<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup return f(*args, **kwargs)<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/rbd.py", line 420, in get_volume_stats<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup self._update_volume_stats()<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/rbd.py", line 405, in _update_volume_stats<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup pool_stats['max_avail'] // units.Gi)<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup KeyError: 'max_avail'<br>2016-03-08 07:39:38.776 7856 ERROR oslo_service.threadgroup <br>2016-03-08 07:39:38.781 5012 INFO oslo_service.service [req-6bc8b049-aafe-4302-8fe1-457dce30ed0f - - - - -] Child 7856 exited with status 0<br></div><div><br></div><div>I turned it of it was looping forking to fast...<br><br></div><div>cinder.conf<br></div><div class="gmail_extra"><br>[DEFAULT]<br>verbose=True<br>rootwrap_config = /etc/cinder/rootwrap.conf<br>api_paste_confg = /etc/cinder/api-paste.ini<br>iscsi_helper = tgtadm<br>volume_name_template = volume-%s<br>volume_group = cinder-volumes<br>verbose = True<br>auth_strategy = keystone<br>state_path = /var/lib/cinder<br>lock_path = /var/lock/cinder<br>volumes_dir = /var/lib/cinder/volumes<br>rpc_backend = rabbit<br>auth_strategy = keystone<br>my_ip = 10.0.5.11<br>glance_host = controller<br>volume_driver = cinder.volume.drivers.rbd.RBDDriver<br>rbd_pool = volumes<br>rbd_ceph_conf = /etc/ceph/ceph.conf<br>rbd_user = cinder<br>rbd_secret_uuid = XXXXXXXXXXXXXXXXXXXXXXXXX<br>rbd_flatten_volume_from_snapshot = false<br>rbd_max_clone_depth = 5<br>rbd_store_chunk_size = 4<br>rados_connect_timeout = -1<br>glance_api_version = 2<br><br>[database]<br>connection = mysql+pymysql://cinder:XXXXXXXXXXXXXXXXXX@controller/cinder<br><br>[oslo_messaging_rabbit]<br>rabbit_host = controller<br>rabbit_userid = openstack<br>rabbit_password = XXXXXXXXXXXXXXXXXX<br><br>[oslo_concurrency]<br>lock_path = /var/lib/cinder/tmp<br><br>[keystone_authtoken]<br>auth_uri = <a href="http://controller:5000">http://controller:5000</a><br>auth_url = <a href="http://controller:35357">http://controller:35357</a><br>auth_plugin = password<br>project_domain_id = default<br>user_domain_id = default<br>project_name = service<br>username = cinder<br>password = 5f4e5210eeed8c900601<br><br><br><div class="gmail_quote">On 8 March 2016 at 09:10, Geo Varghese <span dir="ltr"><<a href="mailto:gvarghese@aqorn.com" target="_blank">gvarghese@aqorn.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">which error it is showing?<br></div><div class="gmail_extra"><div><div class="h5"><br><div class="gmail_quote">On Tue, Mar 8, 2016 at 11:37 AM, Martin Wilderoth <span dir="ltr"><<a href="mailto:martin.wilderoth@linserv.se" target="_blank">martin.wilderoth@linserv.se</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Thanks Both,<div><br></div><div>I will run it on controller node.</div><div><br></div><div>My cinder-volume crashed.</div><div>Any dependensis or is my ceph cluster to old </div><div>(Im running dumpling) I will investigate</div><div><br></div><div>Thanks</div><div><br></div><div><br></div><div class="gmail_extra"><div class="gmail_quote"><div><div>On 8 March 2016 at 06:32, Mike Smith <span dir="ltr"><<a href="mailto:mismith@overstock.com" target="_blank">mismith@overstock.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word"><div><div>
If you are using Ceph as a Cinder backend, you would likely want to run cinder-volume on your controller node(s). You could run it anywhere I suppose, including on the Ceph nodes themselves, but I’d recommend having it on the controllers. Wherever you run
it, you’d need a properly configured ceph.conf, and if you are using cephx authentication, you’d need the keyring files. Your compute nodes would need that conf and keys also.
<div><br>
</div>
<div>You can also run Ceph for nova ephemeral disks without Cinder at all. You’d do that in nova.conf.</div>
<div><br>
</div>
<div>We use both at Overstock. Ceph for nova ephemeral for general use, and also Ceph as one option in a multi-backend Cinder configuration. We also use it for a Glance store, which is a fantastic option because it makes disk provisioning for Nova
instant, since you’re essentially snapshotting and image RBD into an RBD for Nova/Cinder.</div>
<div>
<div>
<div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<br>
Mike Smith<br>
Lead Cloud Systems Architect</div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<a href="http://Overstock.com" target="_blank">Overstock.com</a><br>
<br>
<br>
</div>
</div>
</div>
</div>
<br>
<div>
<blockquote type="cite"><div><div>
<div>On Mar 7, 2016, at 10:04 PM, Martin Wilderoth <<a href="mailto:martin.wilderoth@linserv.se" target="_blank">martin.wilderoth@linserv.se</a>> wrote:</div>
<br>
</div></div><div><div><div>
<div dir="ltr"><br clear="all">
<div>
<div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div>Hello</div>
<div><br>
</div>
<div>Where should I run cinder-volume when i user ceph</div>
<div>on the controller ?<br>
</div>
<div>on the ceph mon or mds ?</div>
<div>or ?<br>
</div>
<div><br>
</div>
<div>Maybe it dosn't matter ?</div>
<div><br>
</div>
<div>Thanks in advance</div>
<div><br>
</div>
<div>Regards Martin</div>
<span></span></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div></div><span>
_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
</span></div>
</blockquote>
</div>
<br>
</div>
</div>
<br>
</div></div><hr>
<font face="Arial" color="Gray" size="1"><br>
CONFIDENTIALITY NOTICE: This message is intended only for the use and review of the individual or entity to which it is addressed and may contain information that is privileged and confidential. If the reader of this message is not the intended recipient, or
the employee or agent responsible for delivering the message solely to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in
error, please notify sender immediately by telephone or return email. Thank you.<br>
</font>
</div>
</blockquote></div><br></div></div>
<br>_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
<br></blockquote></div><br><br clear="all"><br></div></div><span class="HOEnZb"><font color="#888888">-- <br><div><div dir="ltr">--<div>Regards,</div><div>Geo Varghese</div></div></div>
</font></span></div>
</blockquote></div><br></div></div></div>