<div dir="ltr"><div class="gmail_default" style="font-family:courier new,monospace"><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Dec 4, 2013 at 2:26 PM, Byron Briggs <span dir="ltr"><<a href="mailto:byron.briggs@ait.com" target="_blank">byron.briggs@ait.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">root@Compute01-CodeNode:/etc/cinder# vgs<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">File descriptor 3 (/usr/share/bash-completion/completions) leaked on vgs invocation. Parent PID 10778: -bash<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> VG #PV #LV #SN Attr VSize VFree <u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> cinder-volumes 1 0 0 wz--n- 100.00g 100.00g<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">My control nodes don’t but that shouldn’t matter from my understanding.<u></u><u></u></span></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">root@control01:/etc/cinder# vgs<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> VG #PV #LV #SN Attr VSize VFree <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> control01-vg 1 2 0 wz--n- 931.27g 44.00m<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> John Griffith [mailto:<a href="mailto:john.griffith@solidfire.com" target="_blank">john.griffith@solidfire.com</a>] <br>
<b>Sent:</b> Wednesday, December 04, 2013 3:20 PM<br><b>To:</b> Byron Briggs<br><b>Cc:</b> <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a>; SYSADMIN</span></p><div class="im">
<br><b>Subject:</b> Re: [Openstack] [CINDER]Cinder Volume Creation<u></u><u></u></div><p></p><p class="MsoNormal"><u></u> <u></u></p><div><div><p class="MsoNormal"><span style="font-family:"Courier New""><u></u> <u></u></span></p>
</div><div><p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p><div><p class="MsoNormal">On Wed, Dec 4, 2013 at 12:43 PM, Byron Briggs <<a href="mailto:byron.briggs@ait.com" target="_blank">byron.briggs@ait.com</a>> wrote:<u></u><u></u></p>
<div><div class="h5"><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in"><div><div><p class="MsoNormal">When running<u></u><u></u></p><p class="MsoNormal">
<u></u><u></u></p><p class="MsoNormal"><b>cinder create --display_name test 10</b><u></u><u></u></p><p class="MsoNormal"><b> </b><u></u><u></u></p><p class="MsoNormal">From the compute01NovaCompute.dmz-pod2 Node listed below. It is also running xenapi for a xenserver.<u></u><u></u></p>
<p class="MsoNormal">( Control01,2,3 are an HA cluster sitting behind haproxy/keepalivd running all the communication and schedulers -all that is working well.)<u></u><u></u></p><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal">
<u></u><u></u></p><p class="MsoNormal">I get this error in my cinder-volume.log on the control nodes(since they run schedulers) There is nothing else to go off of other then “ERROR” on the volume status.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal">2013-12-04 12:55:32 WARNING [cinder.scheduler.host_manager] service is down or disabled.<u></u><u></u></p><p class="MsoNormal">2013-12-04 12:55:32 ERROR [cinder.scheduler.manager] Failed to schedule_create_volume: No valid host was found. <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal">In case you can’t see excel below.<u></u><u></u></p><p class="MsoNormal">192.168.220.40 Is the proxy being distributed into the three control nodes.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal">control01.dmz-pod2(192.168.220.41) ->cinder-api,cinder-scheduler<u></u><u></u></p><p class="MsoNormal">control02.dmz-pod2(192.168.220.42) ->cinder-api,cinder-scheduler<u></u><u></u></p>
<p class="MsoNormal">control03.dmz-pod2(192.168.220.43) ->cinder-api,cinder-scheduler<u></u><u></u></p><p class="MsoNormal">compute01NovaCompute.dmz-pod2(192.168.220.101) ->cinder-volume<u></u><u></u></p><p class="MsoNormal">
<u></u><u></u></p><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal"> <u></u><u></u></p><table border="0" cellspacing="0" cellpadding="0" width="824" style="width:618.0pt;margin-left:.1pt;border-collapse:collapse">
<tbody><tr style="height:19.25pt"><td width="239" nowrap valign="bottom" style="width:179.5pt;border:solid windowtext 1.0pt;border-right:none;background:#ffc000;padding:0in 5.4pt 0in 5.4pt;height:19.25pt"><p class="MsoNormal">
Server Name<u></u><u></u></p></td><td width="131" nowrap valign="bottom" style="width:98.5pt;border:solid windowtext 1.0pt;border-bottom:none;background:#ffc000;padding:0in 5.4pt 0in 5.4pt;height:19.25pt"><p class="MsoNormal">
cinder-api<u></u><u></u></p></td><td width="205" nowrap valign="bottom" style="width:154.0pt;border:none;border-top:solid windowtext 1.0pt;background:#ffc000;padding:0in 5.4pt 0in 5.4pt;height:19.25pt"><p class="MsoNormal">
cinder-scheduler<u></u><u></u></p></td><td width="248" nowrap valign="bottom" style="width:186.0pt;border:solid windowtext 1.0pt;background:#ffc000;padding:0in 5.4pt 0in 5.4pt;height:19.25pt"><p class="MsoNormal">cinder-volume<u></u><u></u></p>
</td></tr><tr style="height:11.6pt"><td width="239" nowrap valign="bottom" style="width:179.5pt;border-top:none;border-left:solid windowtext 1.0pt;border-bottom:solid windowtext 1.0pt;border-right:none;background:#ffc000;padding:0in 5.4pt 0in 5.4pt;height:11.6pt">
<p class="MsoNormal">control01.dmz-pod2<u></u><u></u></p></td><td width="131" nowrap valign="bottom" style="width:98.5pt;border-top:solid windowtext 1.0pt;border-left:solid windowtext 1.0pt;border-bottom:none;border-right:none;background:#92d050;padding:0in 5.4pt 0in 5.4pt;height:11.6pt">
<p class="MsoNormal">YES<u></u><u></u></p></td><td width="205" nowrap valign="bottom" style="width:154.0pt;border:none;border-top:solid windowtext 1.0pt;background:#92d050;padding:0in 5.4pt 0in 5.4pt;height:11.6pt"><p class="MsoNormal">
YES<u></u><u></u></p></td><td width="248" nowrap valign="bottom" style="width:186.0pt;border:none;border-right:solid windowtext 1.0pt;background:red;padding:0in 5.4pt 0in 5.4pt;height:11.6pt"><p class="MsoNormal">NO<u></u><u></u></p>
</td></tr><tr style="height:15.0pt"><td width="239" nowrap valign="bottom" style="width:179.5pt;border-top:none;border-left:solid windowtext 1.0pt;border-bottom:solid windowtext 1.0pt;border-right:none;background:#ffc000;padding:0in 5.4pt 0in 5.4pt;height:15.0pt">
<p class="MsoNormal">control02.dmz-pod2<u></u><u></u></p></td><td width="131" nowrap valign="bottom" style="width:98.5pt;border:none;border-left:solid windowtext 1.0pt;background:#92d050;padding:0in 5.4pt 0in 5.4pt;height:15.0pt">
<p class="MsoNormal">YES<u></u><u></u></p></td><td width="205" nowrap valign="bottom" style="width:154.0pt;background:#92d050;padding:0in 5.4pt 0in 5.4pt;height:15.0pt"><p class="MsoNormal">YES<u></u><u></u></p></td><td width="248" nowrap valign="bottom" style="width:186.0pt;border:none;border-right:solid windowtext 1.0pt;background:red;padding:0in 5.4pt 0in 5.4pt;height:15.0pt">
<p class="MsoNormal">NO<u></u><u></u></p></td></tr><tr style="height:15.0pt"><td width="239" nowrap valign="bottom" style="width:179.5pt;border-top:none;border-left:solid windowtext 1.0pt;border-bottom:solid windowtext 1.0pt;border-right:none;background:#ffc000;padding:0in 5.4pt 0in 5.4pt;height:15.0pt">
<p class="MsoNormal">control03.dmz-pod2<u></u><u></u></p></td><td width="131" nowrap valign="bottom" style="width:98.5pt;border:none;border-left:solid windowtext 1.0pt;background:#92d050;padding:0in 5.4pt 0in 5.4pt;height:15.0pt">
<p class="MsoNormal">YES<u></u><u></u></p></td><td width="205" nowrap valign="bottom" style="width:154.0pt;background:#92d050;padding:0in 5.4pt 0in 5.4pt;height:15.0pt"><p class="MsoNormal">YES<u></u><u></u></p></td><td width="248" nowrap valign="bottom" style="width:186.0pt;border:none;border-right:solid windowtext 1.0pt;background:red;padding:0in 5.4pt 0in 5.4pt;height:15.0pt">
<p class="MsoNormal">NO<u></u><u></u></p></td></tr><tr style="height:15.0pt"><td width="239" nowrap valign="bottom" style="width:179.5pt;border-top:none;border-left:solid windowtext 1.0pt;border-bottom:solid windowtext 1.0pt;border-right:none;background:#ffc000;padding:0in 5.4pt 0in 5.4pt;height:15.0pt">
<p class="MsoNormal">compute01NovaCompute.dmz-pod2<u></u><u></u></p></td><td width="131" nowrap valign="bottom" style="width:98.5pt;border:none;border-left:solid windowtext 1.0pt;background:red;padding:0in 5.4pt 0in 5.4pt;height:15.0pt">
<p class="MsoNormal">NO<u></u><u></u></p></td><td width="205" nowrap valign="bottom" style="width:154.0pt;background:red;padding:0in 5.4pt 0in 5.4pt;height:15.0pt"><p class="MsoNormal">NO<u></u><u></u></p></td><td width="248" nowrap valign="bottom" style="width:186.0pt;border:none;border-right:solid windowtext 1.0pt;background:#92d050;padding:0in 5.4pt 0in 5.4pt;height:15.0pt">
<p class="MsoNormal">NO<u></u><u></u></p></td></tr></tbody></table><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal">All services are running with no log errors.<u></u><u></u></p><p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Here is my configs<u></u><u></u></p><p class="MsoNormal"><b>compute01NovaCompute.dmz-pod2</b><u></u><u></u></p><p class="MsoNormal"><u>/etc/cinder/api-paste.ini</u><u></u><u></u></p><p class="MsoNormal">
[filter:authtoken]<u></u><u></u></p><p class="MsoNormal">paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory<u></u><u></u></p><p class="MsoNormal">service_protocol = http<u></u><u></u></p><p class="MsoNormal">
service_host = %SERVICE_TENANT_NAME%<u></u><u></u></p><p class="MsoNormal">service_port = 5000<u></u><u></u></p><p class="MsoNormal">auth_host = 127.0.0.1<u></u><u></u></p><p class="MsoNormal">auth_port = 35357<u></u><u></u></p>
<p class="MsoNormal">auth_protocol = http<u></u><u></u></p><p class="MsoNormal">admin_tenant_name = services<u></u><u></u></p><p class="MsoNormal">admin_user = %SERVICE_USER%<u></u><u></u></p><p class="MsoNormal">admin_password = %SERVICE_PASSWORD%<u></u><u></u></p>
<p class="MsoNormal">signing_dir = /var/lib/cinder<u></u><u></u></p><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal"><u>/etc/cinder/cinder.conf</u><u></u><u></u></p><p class="MsoNormal">[DEFAULT]<u></u><u></u></p>
<p class="MsoNormal">iscsi_ip_address=192.168.220.101<u></u><u></u></p><p class="MsoNormal">rabbit_ha_queues=True<u></u><u></u></p><p class="MsoNormal">rabbit_hosts=control01:5672,control02:5672,control03:5672<u></u><u></u></p>
<p class="MsoNormal">rabbit_userid=openstack_rabbit_user<u></u><u></u></p><p class="MsoNormal">rabbit_password=openstack_rabbit_password<u></u><u></u></p><p class="MsoNormal">sql_connection = mysql://<a href="http://cinder:cinder_pass@192.168.220.40/cinder" target="_blank">cinder:cinder_pass@192.168.220.40/cinder</a><u></u><u></u></p>
<p class="MsoNormal">rootwrap_config = /etc/cinder/rootwrap.conf<u></u><u></u></p><p class="MsoNormal">api_paste_confg = /etc/cinder/api-paste.ini<u></u><u></u></p><p class="MsoNormal">iscsi_helper = tgtadm<u></u><u></u></p>
<p class="MsoNormal">volume_name_template = volume-%s<u></u><u></u></p><p class="MsoNormal">volume_group = cinder-volumes<u></u><u></u></p><p class="MsoNormal">verbose = True<u></u><u></u></p><p class="MsoNormal">auth_strategy = keystone<u></u><u></u></p>
<p class="MsoNormal">state_path = /var/lib/cinder<u></u><u></u></p><p class="MsoNormal">lock_path = /var/lock/cinder<u></u><u></u></p><p class="MsoNormal">volumes_dir = /var/lib/cinder/volumes<u></u><u></u></p><p class="MsoNormal">
<u></u><u></u></p><p class="MsoNormal"><u>pvscan</u><u></u><u></u></p><p class="MsoNormal">File descriptor 3 (/usr/share/bash-completion/completions) leaked on pvscan invocation. Parent PID 10778: -bash<u></u><u></u></p>
<p class="MsoNormal"> PV /dev/xvdb VG cinder-volumes lvm2 [100.00 GiB / 100.00 GiB free]<u></u><u></u></p><p class="MsoNormal"> Total: 1 [100.00 GiB] / in use: 1 [100.00 GiB] / in no VG: 0 [0 ]<u></u><u></u></p><p class="MsoNormal">
<u></u><u></u></p><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal"><b>control01,2,3</b><u></u><u></u></p><p class="MsoNormal"><u>/etc/cinder/api-paste.ini</u><u></u><u></u></p><p class="MsoNormal">[filter:authtoken]<u></u><u></u></p>
<p class="MsoNormal">paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory<u></u><u></u></p><p class="MsoNormal">service_protocol = http<u></u><u></u></p><p class="MsoNormal">service_host = 192.168.220.40<u></u><u></u></p>
<p class="MsoNormal">service_port = 5000<u></u><u></u></p><p class="MsoNormal">auth_host = 192.168.220.40<u></u><u></u></p><p class="MsoNormal">auth_port = 35357<u></u><u></u></p><p class="MsoNormal">auth_protocol = http<u></u><u></u></p>
<p class="MsoNormal">admin_tenant_name = services<u></u><u></u></p><p class="MsoNormal">admin_user = cinder<u></u><u></u></p><p class="MsoNormal">admin_password = keystone_admin<u></u><u></u></p><p class="MsoNormal">signing_dir = /var/lib/cinder<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal"><u>/etc/cinder/conder.conf</u><u></u><u></u></p><p class="MsoNormal">[DEFAULT]<u></u><u></u></p><p class="MsoNormal">sql_idle_timeout=30 <u></u><u></u></p><p class="MsoNormal">
rabbit_ha_queues=True<u></u><u></u></p><p class="MsoNormal">rabbit_hosts=control01:5672,control02:5672,control03:5672<u></u><u></u></p><p class="MsoNormal">rabbit_userid=openstack_rabbit_user<u></u><u></u></p><p class="MsoNormal">
rabbit_password=openstack_rabbit_password<u></u><u></u></p><p class="MsoNormal">sql_connection = mysql://<a href="http://cinder:cinder_pass@192.168.220.40/cinder" target="_blank">cinder:cinder_pass@192.168.220.40/cinder</a><u></u><u></u></p>
<p class="MsoNormal">osapi_volume_listen = 192.168.220.41<u></u><u></u></p><p class="MsoNormal">rootwrap_config = /etc/cinder/rootwrap.conf<u></u><u></u></p><p class="MsoNormal">api_paste_confg = /etc/cinder/api-paste.ini<u></u><u></u></p>
<p class="MsoNormal">iscsi_helper = tgtadm<u></u><u></u></p><p class="MsoNormal">volume_name_template = volume-%s<u></u><u></u></p><p class="MsoNormal">volume_group = nova-volumes<u></u><u></u></p><p class="MsoNormal">verbose = True<u></u><u></u></p>
<p class="MsoNormal">auth_strategy = keystone<u></u><u></u></p><p class="MsoNormal">state_path = /var/lib/cinder<u></u><u></u></p><p class="MsoNormal">lock_path = /var/lock/cinder<u></u><u></u></p><p class="MsoNormal">volumes_dir = /var/lib/cinder/volumes<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal">Grizzly Release<u></u><u></u></p><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal">
Any ideas on where to look more into the issue or something with my config?<u></u><u></u></p><p class="MsoNormal"> <u></u><u></u></p></div></div><p class="MsoNormal" style="margin-bottom:12.0pt"><br>_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><u></u><u></u></p></blockquote></div></div></div><p class="MsoNormal">
<u></u> <u></u></p></div><div><div class="h5"><div><div><p class="MsoNormal"><span style="font-family:"Courier New"">So this happens in a couple of situations, the most common is when there's not enough space/capacity being reported by the configured backend driver to allocate the amount of space you're requesting. Try a "sudo vgs" and verify that you have enough capacity on your backing store (nova-volumes) to deploy a 10G volume.<u></u><u></u></span></p>
</div><p class="MsoNormal"><u></u> <u></u></p></div></div></div></div></div></div></blockquote></div><br></div><div class="gmail_extra"><div class="gmail_default" style="font-family:'courier new',monospace">Ooops, sorry I didn't catch the separate controller node and only saw the volume_group setting there. Any chance you could link a paste-bin to the cinder-scheduler logs?</div>
<br></div></div>