<div dir="ltr">Hello Mike,<br><br>Yeah I can run the command withou "--user cinder" and works fine. Also I do "virsh secret-list" on the compute host and works fine.<div><br></div><div>I am working with a HA installation, HAproxy, Galera and RabbitMQ in cluster mode, can be related to that?</div><div><br></div><div>Glance is storing local in the controller.</div><div><br></div><div>Thanks for your help!</div><div><br></div><div><br></div></div><br><div class="gmail_quote">El mar., 12 de may. de 2015 a la(s) 8:52 a. m., Mike Smith <<a href="mailto:mismith@overstock.com">mismith@overstock.com</a>> escribió:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
OK, so it looks like your controller and compute nodes have the proper keyrings in place and are allowed to view the RBDs.
<div><br>
</div>
<div>(On the storage node you probably can run without the “—user cinder” flag and it will work as long you have the correct /etc/ceph/ceph.client.admin.keyring in place)</div>
<div><br>
</div>
<div>What are you using for your glance backend? From the error I can’t tell if it’s failing to create the (ceph-based) volume image or failing to read the glance image. </div>
<div><br>
</div>
<div>Also, do you get a valid response back from “virsh secret-list” on the compute host, showing you the secret token you defined for use with cephx authentication? </div></div><div style="word-wrap:break-word"><div><br>
<div><br>
<br>
Mike </div></div></div><div style="word-wrap:break-word"><div>
<div><br>
</div>
<br>
<div>
<blockquote type="cite">
<div>On May 11, 2015, at 5:19 PM, Guillermo Alvarado <<a href="mailto:guillermoalvarado89@gmail.com" target="_blank">guillermoalvarado89@gmail.com</a>> wrote:</div>
<br>
<div>
<div dir="ltr">Hello, thanks Mike. Here the output:<br>
<br>
Controller:
<div><br>
</div>
<div>
<div>root@ctrl-net01:/var/lib# rbd --user cinder --pool volumes ls</div>
<div>testa1</div>
<div>volume-2b20fca2-752b-4c93-9dd1-600c96b2a6c4</div>
<div>volume-526df27f-f8a5-42c0-96f1-1976ff1cc1a9</div>
<div>volume-a15e66ee-5d9a-4032-a888-74a173584fd0</div>
<div>volume-b49d05bc-07e5-4104-9adf-4d7e2daf7839<br>
<br>
<br>
<br>
Compute:</div>
</div>
<div>
<div>root@comp01:/etc/ceph# rbd --user cinder --pool volumes ls</div>
<div>testa1</div>
<div>volume-2b20fca2-752b-4c93-9dd1-600c96b2a6c4</div>
<div>volume-526df27f-f8a5-42c0-96f1-1976ff1cc1a9</div>
<div>volume-a15e66ee-5d9a-4032-a888-74a173584fd0</div>
<div>volume-b49d05bc-07e5-4104-9adf-4d7e2daf7839<br>
<br>
<br>
Storage node(Ceph):</div>
<div><br>
</div>
<div>
<div>root@stor01:/etc/ceph# rbd --user cinder --pool volumes ls</div>
<div>2015-05-11 18:14:51.469680 7fe859185840 -1 monclient(hunting): ERROR: missing keyring, cannot use cephx for authentication</div>
<div>2015-05-11 18:14:51.469686 7fe859185840 0 librados: client.cinder initialization error (2) No such file or directory</div>
<div>rbd: couldn't connect to the cluster!</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Also I recreated all the steps above but without glance configurations because I do not want use RBd or ceph to my glance implementation.</div>
<div><br>
</div>
<div>Thanks in advance.</div>
<div><br>
</div>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">El lun., 11 de may. de 2015 a la(s) 6:06 p. m., Mike Smith <<a href="mailto:mismith@overstock.com" target="_blank">mismith@overstock.com</a>> escribió:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">I’ve been playing with Ceph quite a bit the last couple weeks and can probably help you out. The problem on looks to be:
</div>
<div style="word-wrap:break-word">
<div><br>
</div>
<div> Image 94dc5715-7ce6-410b-90a9-d9a2f9b2e1e5 could not be found. </div>
<div><br>
</div>
</div>
<div style="word-wrap:break-word">
<div>Are you able to create any empty volumes successfully? If so, and assuming you followed the steps from <a href="http://docs.ceph.com/docs/master/rbd/rbd-openstack/" target="_blank">http://docs.ceph.com/docs/master/rbd/rbd-openstack/</a>,
try this from your compute host:
<div><br>
</div>
<div>rbd --user cinder --pool volumes ls</div>
<div><br>
</div>
<div>Also try that from the node where the cinder volume service is running. If you get an error trying that from either host, retrace the steps on the document above. If it works fine, then the issue is likely related to the "libvirtd secrets”
section and making sure nova and libvirt are set up.</div>
<div><br>
</div>
<div>Let us know what you see and I’ll see if I can help.</div>
<div><br>
</div>
<div>Mike</div>
<div><br>
</div>
<div><br>
</div>
<div>Mike Smith<br>
Principal Engineer, Website Systems<br>
<a href="http://overstock.com/" target="_blank">Overstock.com</a></div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div><br>
</div>
<div>
<div>
<div><br>
</div>
<br>
<div>
<blockquote type="cite"></blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
<div style="word-wrap:break-word">
<div>
<div>
<div>
<div>
<div>
<blockquote type="cite">
<div>On May 11, 2015, at 4:49 PM, Guillermo Alvarado <<a href="mailto:guillermoalvarado89@gmail.com" target="_blank">guillermoalvarado89@gmail.com</a>> wrote:</div>
<br>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
<div style="word-wrap:break-word">
<div>
<div>
<div>
<div>
<div>
<blockquote type="cite">
<div>
<div dir="ltr">Hello, I have access to a running Ceph Storage Cluster. I am able to create volumes with cinder, but not VMs (boot from image, creates new volume) Here cinder-volume.log Anybody can help me? Thanks in advance. 2015-05-11 17:39:27.879
58175 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher incoming.message)) 2015-05-11 17:39:27.879
58175 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt,
args) 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint,
method)(ctxt, **new_args) 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher return f(*args,
**kwargs) 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 381, in create_volume 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher _run_flow() 2015-05-11
17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 374, in _run_flow 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher flow_engine.run() 2015-05-11 17:39:27.879 58175
TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 89, in run 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher for _state in self.run_iter(): 2015-05-11 17:39:27.879 58175
TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/engine.py", line 137, in run_iter 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher misc.Failure.reraise_if_any(failures.values()) 2015-05-11
17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/taskflow/utils/misc.py", line 797, in reraise_if_any 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher failures[0].reraise() 2015-05-11 17:39:27.879
58175 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/taskflow/utils/misc.py", line 804, in reraise 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher six.reraise(*self._exc_info) 2015-05-11 17:39:27.879 58175 TRACE
oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 34, in _execute_task 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher result = task.execute(**arguments) 2015-05-11 17:39:27.879
58175 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/manager/create_volume.py", line 638, in execute 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher **volume_spec) 2015-05-11 17:39:27.879 58175
TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/manager/create_volume.py", line 590, in _create_from_image 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher image_id, image_location, image_service)
2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/cinder/volume/flows/manager/create_volume.py", line 504, in _copy_image_to_volume 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher raise
exception.ImageCopyFailure(reason=ex) 2015-05-11 17:39:27.879 58175 TRACE oslo.messaging.rpc.dispatcher ImageCopyFailure: Failed to copy image to volume: Image 94dc5715-7ce6-410b-90a9-d9a2f9b2e1e5 could not be found.
<br>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
<div style="word-wrap:break-word">
<div>
<div>
<div>
<div>
<div>
<blockquote type="cite">
<div>_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">
openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br>
</div></div><div style="word-wrap:break-word">
<br>
<hr>
<font face="Arial" color="Gray" size="1"><br>
CONFIDENTIALITY NOTICE: This message is intended only for the use and review of the individual or entity to which it is addressed and may contain information that is privileged and confidential. If the reader of this message is not the intended recipient, or
the employee or agent responsible for delivering the message solely to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in
error, please notify sender immediately by telephone or return email. Thank you.<br>
</font>
</div></blockquote></div>