<div dir="ltr">Hello John, my cinder list is empty, I've done cinder delete #storage-id but when I do lvdisplay they're still there. I've already tried dmsetup remove but no success too. And the storages are not in use with any VM.<div>
<br></div><div>Regards.</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">2013/11/13 John Griffith <span dir="ltr"><<a href="mailto:john.griffith@solidfire.com" target="_blank">john.griffith@solidfire.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Wed, Nov 13, 2013 at 5:50 AM, Guilherme Russi<br>
<div><div class="h5"><<a href="mailto:luisguilherme.cr@gmail.com">luisguilherme.cr@gmail.com</a>> wrote:<br>
> Hello Razique, I'm here opening this thread again, I've done some cinder<br>
> delete but when I try to create another storeges it returns there's no space<br>
> to create a new volume.<br>
><br>
> Here is part of my lvdisplay output:<br>
><br>
> Alloc PE / Size 52224 / 204,00 GiB<br>
> Free PE / Size 19350 / 75,59 GiB<br>
><br>
> And here is my lvdisplay:<br>
><br>
><br>
> --- Logical volume ---<br>
> LV Name<br>
> /dev/cinder-volumes/volume-06ccd141-91c4-45e4-b21f-595f4a36779b<br>
> VG Name cinder-volumes<br>
> LV UUID wdqxVd-GgUQ-21O4-OWlR-sRT3-HvUA-Q8j9kL<br>
> LV Write Access read/write<br>
> LV snapshot status source of<br>
><br>
> /dev/cinder-volumes/_snapshot-04e8414e-2c0e-4fc2-8bff-43dd80ecca09 [active]<br>
> LV Status available<br>
> # open 0<br>
> LV Size 10,00 GiB<br>
> Current LE 2560<br>
> Segments 1<br>
> Allocation inherit<br>
> Read ahead sectors auto<br>
> - currently set to 256<br>
> Block device 252:1<br>
><br>
> --- Logical volume ---<br>
> LV Name<br>
> /dev/cinder-volumes/_snapshot-04e8414e-2c0e-4fc2-8bff-43dd80ecca09<br>
> VG Name cinder-volumes<br>
> LV UUID EZz1lC-a8H2-1PlN-pJTN-XAIm-wW0q-qtUQOc<br>
> LV Write Access read/write<br>
> LV snapshot status active destination for<br>
> /dev/cinder-volumes/volume-06ccd141-91c4-45e4-b21f-595f4a36779b<br>
> LV Status available<br>
> # open 0<br>
> LV Size 10,00 GiB<br>
> Current LE 2560<br>
> COW-table size 10,00 GiB<br>
> COW-table LE 2560<br>
> Allocated to snapshot 0,00%<br>
> Snapshot chunk size 4,00 KiB<br>
> Segments 1<br>
> Allocation inherit<br>
> Read ahead sectors auto<br>
> - currently set to 256<br>
> Block device 252:3<br>
><br>
> --- Logical volume ---<br>
> LV Name<br>
> /dev/cinder-volumes/volume-ca36920e-938e-4ad1-b9c4-74c1e28abd31<br>
> VG Name cinder-volumes<br>
> LV UUID b40kQV-P8N4-R6jt-k97Z-I2a1-9TXm-5GXqfz<br>
> LV Write Access read/write<br>
> LV Status available<br>
> # open 1<br>
> LV Size 60,00 GiB<br>
> Current LE 15360<br>
> Segments 1<br>
> Allocation inherit<br>
> Read ahead sectors auto<br>
> - currently set to 256<br>
> Block device 252:4<br>
><br>
> --- Logical volume ---<br>
> LV Name<br>
> /dev/cinder-volumes/volume-70be4f36-10bd-4877-b841-80333ccfe985<br>
> VG Name cinder-volumes<br>
> LV UUID 2YDrMs-BrYo-aQcZ-8AlX-A4La-HET1-9UQ0gV<br>
> LV Write Access read/write<br>
> LV Status available<br>
> # open 1<br>
> LV Size 1,00 GiB<br>
> Current LE 256<br>
> Segments 1<br>
> Allocation inherit<br>
> Read ahead sectors auto<br>
> - currently set to 256<br>
> Block device 252:5<br>
><br>
> --- Logical volume ---<br>
> LV Name<br>
> /dev/cinder-volumes/volume-00c532bd-91fb-4a38-b340-4389fb7f0ed5<br>
> VG Name cinder-volumes<br>
> LV UUID MfVOuB-5x5A-jne3-H4Ul-4NP8-eI7b-UYSYE7<br>
> LV Write Access read/write<br>
> LV Status available<br>
> # open 0<br>
> LV Size 1,00 GiB<br>
> Current LE 256<br>
> Segments 1<br>
> Allocation inherit<br>
> Read ahead sectors auto<br>
> - currently set to 256<br>
> Block device 252:6<br>
><br>
> --- Logical volume ---<br>
> LV Name<br>
> /dev/cinder-volumes/volume-ae133dbc-6141-48cf-beeb-9d6576e57a45<br>
> VG Name cinder-volumes<br>
> LV UUID 53w8j3-WT4V-8m52-r6LK-ZYd3-mMHA-FtuyXV<br>
> LV Write Access read/write<br>
> LV Status available<br>
> # open 0<br>
> LV Size 1,00 GiB<br>
> Current LE 256<br>
> Segments 1<br>
> Allocation inherit<br>
> Read ahead sectors auto<br>
> - currently set to 256<br>
> Block device 252:7<br>
><br>
> --- Logical volume ---<br>
> LV Name<br>
> /dev/cinder-volumes/volume-954d2f1b-837b-4ba5-abfd-b3610597be5e<br>
> VG Name cinder-volumes<br>
> LV UUID belquE-WxQ2-gt6Y-WlPE-Hmq3-B9Am-zcYD3P<br>
> LV Write Access read/write<br>
> LV Status available<br>
> # open 0<br>
> LV Size 60,00 GiB<br>
> Current LE 15360<br>
> Segments 1<br>
> Allocation inherit<br>
> Read ahead sectors auto<br>
> - currently set to 256<br>
> Block device 252:8<br>
><br>
> --- Logical volume ---<br>
> LV Name<br>
> /dev/cinder-volumes/volume-05d037d1-4e61-4419-929a-fe340e00e1af<br>
> VG Name cinder-volumes<br>
> LV UUID Pt61e7-l3Nu-1IdX-T2sb-0GQD-PhS6-XtIIUj<br>
> LV Write Access read/write<br>
> LV Status available<br>
> # open 1<br>
> LV Size 1,00 GiB<br>
> Current LE 256<br>
> Segments 1<br>
> Allocation inherit<br>
> Read ahead sectors auto<br>
> - currently set to 256<br>
> Block device 252:9<br>
><br>
> --- Logical volume ---<br>
> LV Name<br>
> /dev/cinder-volumes/volume-316f77c6-bf13-4ea4-9b98-028198f3922f<br>
> VG Name cinder-volumes<br>
> LV UUID e46mBx-CRps-HYKk-aJsc-XFRd-B1Rv-UVk8gT<br>
> LV Write Access read/write<br>
> LV Status available<br>
> # open 1<br>
> LV Size 60,00 GiB<br>
> Current LE 15360<br>
> Segments 1<br>
> Allocation inherit<br>
> Read ahead sectors auto<br>
> - currently set to 256<br>
> Block device 252:10<br>
><br>
><br>
> Do you know how I can remove all of them? When I try lvremove<br>
> /dev/cinder-volumes/volume-316f77c6-bf13-4ea4-9b98-028198f3922f, for<br>
> example, I get:<br>
><br>
> Can't remove open logical volume<br>
> "volume-316f77c6-bf13-4ea4-9b98-028198f3922f"<br>
><br>
> Thank you again.<br>
><br>
><br>
><br>
> 2013/11/8 Guilherme Russi <<a href="mailto:luisguilherme.cr@gmail.com">luisguilherme.cr@gmail.com</a>><br>
>><br>
>> Very thanks again.<br>
>><br>
>> Best regards.<br>
>><br>
>><br>
>> 2013/11/8 Razique Mahroua <<a href="mailto:razique.mahroua@gmail.com">razique.mahroua@gmail.com</a>><br>
>>><br>
>>> Oh yah true!<br>
>>> not sure “conductors” exist yet for Cinder, meaning meanwhile, every node<br>
>>> needs a direct access to the database<br>
>>> glad to hear it’s working :)<br>
>>><br>
>>> On 08 Nov 2013, at 08:53, Guilherme Russi <<a href="mailto:luisguilherme.cr@gmail.com">luisguilherme.cr@gmail.com</a>><br>
>>> wrote:<br>
>>><br>
>>> Hello again Razique, I've found the problem, I need to add the grants on<br>
>>> the mysql to my another IP. Now it's working really good :D<br>
>>> I've found this link too if someone needs:<br>
>>> <a href="http://docs.openstack.org/admin-guide-cloud/content//managing-volumes.html" target="_blank">http://docs.openstack.org/admin-guide-cloud/content//managing-volumes.html</a><br>
>>><br>
>>> Thank you so much, and if you need me just let me know.<br>
>>><br>
>>> Best regards.<br>
>>><br>
>>> Guilherme.<br>
>>><br>
>>><br>
>>><br>
>>> 2013/11/8 Guilherme Russi <<a href="mailto:luisguilherme.cr@gmail.com">luisguilherme.cr@gmail.com</a>><br>
>>>><br>
>>>> Hello Razique, I got a couple of doubts, do you know if I need to do<br>
>>>> something else that's is not on the link you sent me? I'm asking because I<br>
>>>> followed the configuration but it's not working, here is what I get: I've<br>
>>>> installed the cinder-volume at the second computer that have the HD, and<br>
>>>> I've changed it's cinder.conf. I've changed too the master's cinder.conf<br>
>>>> like is following:<br>
>>>><br>
>>>><br>
>>>> [DEFAULT]<br>
>>>> rootwrap_config = /etc/cinder/rootwrap.conf<br>
>>>> sql_connection = mysql://cinder:password@localhost/cinder<br>
>>>> api_paste_confg = /etc/cinder/api-paste.ini<br>
>>>> #iscsi_helper=iscsiadm<br>
>>>> #iscsi_helper = ietadm<br>
>>>> iscsi_helper = tgtadm<br>
>>>> volume_name_template = volume-%s<br>
>>>> #volume_group = cinder-volumes<br>
>>>> verbose = True<br>
>>>> auth_strategy = keystone<br>
>>>> iscsi_ip_address = 192.168.3.1<br>
>>>> scheduler_driver=cinder.scheduler.filter_scheduler.FilterScheduler<br>
>>>><br>
>>>> # Rabbit authorization<br>
>>>> rabbit_host = localhost<br>
>>>> rabbit_port = 5672<br>
>>>> rabbit_hosts = $rabbit_host:$rabbit_port<br>
>>>> rabbit_use_ssl = false<br>
>>>> rabbit_userid = guest<br>
>>>> rabbit_password = password<br>
>>>> #rabbit_virtual_host = /nova<br>
>>>><br>
>>>> state_path = /var/lib/cinder<br>
>>>> lock_path = /var/lock/cinder<br>
>>>> volumes_dir = /var/lib/cinder/volumes<br>
>>>> #rpc_backend = cinder.rpc.impl_kombu<br>
>>>><br>
>>>> enabled_backends=orion-1,orion-4<br>
>>>> [orion-1]<br>
>>>> volume_group=cinder-volumes<br>
>>>> volume_driver=cinder.volume.drivers.lvm.LVMISCSIDriver<br>
>>>> volume_backend_name=LVM_iSCSI<br>
>>>> [orion-4]<br>
>>>> volume_group=cinder-volumes-2<br>
>>>> volume_driver=cinder.volume.drivers.lvm.LVMISCSIDriver<br>
>>>> volume_backend_name=LVM_iSCSI<br>
>>>><br>
>>>> The cinder.conf on the second computer is like this but the IPs are<br>
>>>> changed with the controller IP (It has the cinder-api), and when I run<br>
>>>> service cinder-volume restart at the second computer it's status is<br>
>>>> stop/waiting.<br>
>>>><br>
>>>> Any ideas?<br>
>>>><br>
>>>> Thanks :)<br>
>>>><br>
>>>><br>
>>>> 2013/11/8 Razique Mahroua <<a href="mailto:razique.mahroua@gmail.com">razique.mahroua@gmail.com</a>><br>
>>>>><br>
>>>>> sure :)<br>
>>>>><br>
>>>>> On 08 Nov 2013, at 05:39, Guilherme Russi <<a href="mailto:luisguilherme.cr@gmail.com">luisguilherme.cr@gmail.com</a>><br>
>>>>> wrote:<br>
>>>>><br>
>>>>> Oh great! I'll try here and send you the results.<br>
>>>>><br>
>>>>> Very thanks :)<br>
>>>>><br>
>>>>><br>
>>>>> 2013/11/8 Razique Mahroua <<a href="mailto:razique.mahroua@gmail.com">razique.mahroua@gmail.com</a>><br>
>>>>>><br>
>>>>>> If I’m not mistaken, you only need to install the “cinder-volume’<br>
>>>>>> service that will update its status to your main node<br>
>>>>>> :)<br>
>>>>>><br>
>>>>>> On 08 Nov 2013, at 05:34, Guilherme Russi <<a href="mailto:luisguilherme.cr@gmail.com">luisguilherme.cr@gmail.com</a>><br>
>>>>>> wrote:<br>
>>>>>><br>
>>>>>> Great! I was reading the link and I have one question, do I need to<br>
>>>>>> install cinder at the other computer too?<br>
>>>>>><br>
>>>>>> Thanks :)<br>
>>>>>><br>
>>>>>><br>
>>>>>> 2013/11/8 Razique Mahroua <<a href="mailto:razique.mahroua@gmail.com">razique.mahroua@gmail.com</a>><br>
>>>>>>><br>
>>>>>>> Ok in that case, with Grizzly you can use the “multi-backends”<br>
>>>>>>> feature:<br>
>>>>>>> <a href="https://wiki.openstack.org/wiki/Cinder-multi-backend" target="_blank">https://wiki.openstack.org/wiki/Cinder-multi-backend</a><br>
>>>>>>><br>
>>>>>>> and that should do it :)<br>
>>>>>>><br>
>>>>>>> On 08 Nov 2013, at 05:29, Guilherme Russi<br>
>>>>>>> <<a href="mailto:luisguilherme.cr@gmail.com">luisguilherme.cr@gmail.com</a>> wrote:<br>
>>>>>>><br>
>>>>>>> It is a hard disk, my scenario is one Controller (where I have my<br>
>>>>>>> storage cinder and my network quantum) and four compute nodes.<br>
>>>>>>><br>
>>>>>>><br>
>>>>>>> 2013/11/8 Razique Mahroua <<a href="mailto:razique.mahroua@gmail.com">razique.mahroua@gmail.com</a>><br>
>>>>>>>><br>
>>>>>>>> ok !<br>
>>>>>>>> what is your actual Cinder backend? Is it a hard disk, a SAN, a<br>
>>>>>>>> network volume, etc…<br>
>>>>>>>><br>
>>>>>>>> On 08 Nov 2013, at 05:20, Guilherme Russi<br>
>>>>>>>> <<a href="mailto:luisguilherme.cr@gmail.com">luisguilherme.cr@gmail.com</a>> wrote:<br>
>>>>>>>><br>
>>>>>>>> Hi Razique, thank you for answering, I want to expand my cinder<br>
>>>>>>>> storage, is it the block storage? I'll use the storage to allow VMs to have<br>
>>>>>>>> more hard disk space.<br>
>>>>>>>><br>
>>>>>>>> Regards.<br>
>>>>>>>><br>
>>>>>>>> Guilherme.<br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>>> 2013/11/8 Razique Mahroua <<a href="mailto:razique.mahroua@gmail.com">razique.mahroua@gmail.com</a>><br>
>>>>>>>>><br>
>>>>>>>>> Hi Guilherme !<br>
>>>>>>>>> Which storage do you precisely want to expand?<br>
>>>>>>>>><br>
>>>>>>>>> Regards,<br>
>>>>>>>>> Razique<br>
>>>>>>>>><br>
>>>>>>>>><br>
>>>>>>>>> On 08 Nov 2013, at 04:52, Guilherme Russi<br>
>>>>>>>>> <<a href="mailto:luisguilherme.cr@gmail.com">luisguilherme.cr@gmail.com</a>> wrote:<br>
>>>>>>>>><br>
>>>>>>>>> > Hello guys, I have a Grizzly deployment running fine with 5<br>
>>>>>>>>> > nodes, and I want to add more storage on it. My question is, can I install a<br>
>>>>>>>>> > new HD on another computer thats not the controller and link this HD with my<br>
>>>>>>>>> > cinder that it can be a storage too?<br>
>>>>>>>>> > The computer I will install my new HD is at the same network as<br>
>>>>>>>>> > my cloud is. I'm asking because I haven't seen a question like that here.<br>
>>>>>>>>> > Does anybody knows how to do that? Have a clue? Any help is welcome.<br>
>>>>>>>>> ><br>
>>>>>>>>> > Thank you all.<br>
>>>>>>>>> ><br>
>>>>>>>>> > Best regards.<br>
>>>>>>>>> > _______________________________________________<br>
>>>>>>>>> > Mailing list:<br>
>>>>>>>>> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
>>>>>>>>> > Post to : <a href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
>>>>>>>>> > Unsubscribe :<br>
>>>>>>>>> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
>>>>>>>>><br>
>>>>>>>><br>
>>>>>>>><br>
>>>>>>><br>
>>>>>>><br>
>>>>>><br>
>>>>>><br>
>>>>><br>
>>>>><br>
>>>><br>
>>><br>
>>><br>
>><br>
><br>
><br>
> _______________________________________________<br>
> Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
> Post to : <a href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
> Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
><br>
<br>
</div></div>It appears that you have in fact exceeded your VG space. As far as<br>
the errors on lvremove; Is your system in a state where you can use<br>
Cinder (cinder delete xxxx) to remove the volume?<br>
<br>
There are a number of ways to try and clean up the busy device on<br>
lvremove. You can try using the -f flag and retrying, sometimes udev<br>
settle is helpful (but I don't think this applies in your case) maybe<br>
"dmsetup remove" will work in your case. I'd be curious if your<br>
volumes are actually in use currently and the system is trying to<br>
"help" you here :)<br>
</blockquote></div><br></div>