<font size=2 face="sans-serif">Ritesh,</font>
<br>
<br><font size=2 face="sans-serif">I am not familiar with cgroups, so I
am not sure if that could have an affect. I would try re-enabling
it for good measure, though the initial information I get when googling
cgroups doesn't seem to be a smoking gun.</font>
<br>
<br><font size=2 face="sans-serif">I think that the feedback that John
Griffith provided mirrors my original response that the source of the problem
is likely that you already have other volumes attached and are looking
at the size information for those mount points rather than the new volumes
you are attaching given that the volumes appear to be the right size on
the v3700.</font>
<br>
<br><font size=2 face="sans-serif">Can you provide more details on how
you are determining the size of the mounted volume?<br>
</font>
<br><font size=3><b><i><br>
<br>
Jay S. Bryant</i></b><i><br>
Linux Developer - <br>
OpenStack Enterprise Edition</i><br>
</font><img src=cid:_2_0AC310340AC30BCC0069079386257BF7 alt="OpenStack Community Logo"><font size=3><br>
Department 7YLA, Building 015-2, Office E125, Rochester, MN<br>
Telephone: (507) 253-4270, FAX (507) 253-6410<br>
TIE Line: 553-4270<br>
E-Mail: jsbryant@us.ibm.com<br>
--------------------------------------------------------------------<br>
All the world's a stage and most of us are desperately unrehearsed.<br>
-- Sean
O'Casey<br>
--------------------------------------------------------------------</font>
<br>
<br>
<br>
<br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Ritesh Nanda <riteshnanda09@gmail.com></font>
<br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">Jay S Bryant/Rochester/IBM@IBMUS,
</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">09/30/2013 02:28 PM</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [Openstack]
cinder volume weird behavior</font>
<br>
<hr noshade>
<br>
<br>
<br><font size=3>Hello Jay , <br>
</font>
<br><font size=3>I use ubuntu 12.04 for controller and compute nodes ,
one change i did which i recall is stopping the cgroups so
that it gives performance to the libvirt.</font>
<br><font size=3>Can it be the issue , rest all packages are the latest
version, i checked all. <br>
</font>
<br><font size=3> is cgroups related something to iscsi + libvirt?</font>
<br><font size=3><br>
</font>
<br><font size=3>On Tue, Oct 1, 2013 at 12:12 AM, Jay S Bryant <</font><a href=mailto:jsbryant@us.ibm.com target=_blank><font size=3 color=blue><u>jsbryant@us.ibm.com</u></font></a><font size=3>>
wrote:</font>
<br><font size=3 face="sans-serif">Ritesh,</font><font size=3> <br>
</font><font size=3 face="sans-serif"><br>
What are you running for the host operating system on the Control Node
and Compute Nodes? </font><font size=3> <br>
</font><font size=3 face="sans-serif"><br>
As I Google around for the errors you are seeing it seems that there have
been some changes to libvirt and qemu that might help to avoid the problem
you are seeing.</font><font size=3> <br>
</font><font size=3 face="sans-serif"><br>
Have you tried updating libvirt and qemu?</font>
<br><font size=3><b><i><br>
<br>
<br>
Jay S. Bryant</i></b><i><br>
Linux Developer - <br>
OpenStack Enterprise Edition</i><br>
</font><img src=cid:_2_08C760DC08C75DB40069079386257BF7 width=155 height=155 alt="OpenStack Community Logo"><font size=3><br>
Department 7YLA, Building 015-2, Office E125, Rochester, MN<br>
Telephone: (507) 253-4270, FAX (507) 253-6410<br>
TIE Line: 553-4270<br>
E-Mail: </font><a href=mailto:jsbryant@us.ibm.com target=_blank><font size=3 color=blue><u>jsbryant@us.ibm.com</u></font></a><font size=3><br>
--------------------------------------------------------------------<br>
All the world's a stage and most of us are desperately unrehearsed.<br>
-- Sean
O'Casey<br>
-------------------------------------------------------------------- <br>
<br>
<br>
</font><font size=1 color=#5f5f5f face="sans-serif"><br>
From: </font><font size=1 face="sans-serif">Ritesh
Nanda <</font><a href=mailto:riteshnanda09@gmail.com target=_blank><font size=1 color=blue face="sans-serif"><u>riteshnanda09@gmail.com</u></font></a><font size=1 face="sans-serif">></font><font size=3>
</font>
<br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">Jay S Bryant/Rochester/IBM@IBMUS,
</font><font size=1 color=#5f5f5f face="sans-serif"><br>
Date: </font><font size=1 face="sans-serif">09/30/2013
01:04 PM</font><font size=3> </font><font size=1 color=#5f5f5f face="sans-serif"><br>
Subject: </font><font size=1 face="sans-serif">Re:
[Openstack] cinder volume weird behavior</font><font size=3> </font>
<br>
<hr noshade><font size=3><br>
<br>
<br>
Hello Jay,<br>
<br>
Bug there for hyperV does'nt look related to my issue , as i have been
specifying mount point like /dev/vdb , /dev/vdc, not using the auto functionality
, When i manually try to attach a created volume from the horizon
on the compute node on which the vm resides nova-compute log shows the
error.<b><br>
<br>
libvirtError: internal error unable to execute QEMU command 'device_add':
Duplicate ID 'virtio-disk2' for device</b><br>
<br>
Given something random mount point while attaching to a vm solves like
specifying /dev/vfg attaches the volume but the below error persists. <br>
<br>
Other problem which accompanies with this is volumes i create for e.g 4gb
gets correctly created on v3700 , but when i attach to a vm , in vm console
it shows always a different size. <br>
<br>
<br>
Giving a restart to open-iscsi on the cinder-volume node solves the issue
for few minutes , then it comes back.<br>
<br>
I have a multi-node grizzly with multi-host nova-network (vlan manager)
running. <br>
Cinder api and scheduler, volume running on the controller and cinder-volume
on all the compute nodes.<br>
<br>
<br>
<br>
<br>
On Mon, Sep 30, 2013 at 11:07 PM, Jay S Bryant <</font><a href=mailto:jsbryant@us.ibm.com target=_blank><font size=3 color=blue><u>jsbryant@us.ibm.com</u></font></a><font size=3>>
wrote: </font><font size=3 face="sans-serif"><br>
Ritesh,</font><font size=3> </font><font size=3 face="sans-serif"><br>
<br>
I have noticed that the 'auto' option for the mount point is not working
as expected right now. There is an issue opened that seems similar:
</font><a href="https://bugs.launchpad.net/nova/+bug/1153842" target=_blank><font size=3 color=blue face="sans-serif"><u>https://bugs.launchpad.net/nova/+bug/1153842</u></font></a><font size=3 face="sans-serif">
The bug spells out HyperV but I believe the same is happening for
KVM. As you have noted, it is pretty easy to work around. I
hope to look at the issue after I get some other bugs fixed.</font><font size=3>
</font><font size=3 face="sans-serif"><br>
<br>
With regards to the size problem, what size the volume appear to be on
the 3700? Does the size of the volume appear as expected there? Is
it associated with the expected host? Are you sure you are looking
at the right device from the compute node? </font><font size=3><b><i><br>
<br>
<br>
<br>
Jay S. Bryant</i></b><i><br>
Linux Developer - <br>
OpenStack Enterprise Edition</i><br>
</font><img src=cid:_2_08C77F0008C76E9C0069079386257BF7 alt="OpenStack Community Logo"><font size=3><br>
Department 7YLA, Building 015-2, Office E125, Rochester, MN<br>
Telephone: (507) 253-4270, FAX (507) 253-6410<br>
TIE Line: 553-4270<br>
E-Mail: </font><a href=mailto:jsbryant@us.ibm.com target=_blank><font size=3 color=blue><u>jsbryant@us.ibm.com</u></font></a><font size=3><br>
--------------------------------------------------------------------<br>
All the world's a stage and most of us are desperately unrehearsed.<br>
-- Sean
O'Casey<br>
-------------------------------------------------------------------- <br>
<br>
</font><font size=1 color=#5f5f5f face="sans-serif"><br>
<br>
From: </font><font size=1 face="sans-serif">Ritesh
Nanda <</font><a href=mailto:riteshnanda09@gmail.com target=_blank><font size=1 color=blue face="sans-serif"><u>riteshnanda09@gmail.com</u></font></a><font size=1 face="sans-serif">></font><font size=3>
</font><font size=1 color=#5f5f5f face="sans-serif"><br>
To: </font><font size=1 face="sans-serif">"</font><a href=mailto:openstack@lists.openstack.org target=_blank><font size=1 color=blue face="sans-serif"><u>openstack@lists.openstack.org</u></font></a><font size=1 face="sans-serif">"
<</font><a href=mailto:openstack@lists.openstack.org target=_blank><font size=1 color=blue face="sans-serif"><u>openstack@lists.openstack.org</u></font></a><font size=1 face="sans-serif">>,
</font><font size=1 color=#5f5f5f face="sans-serif"><br>
Date: </font><font size=1 face="sans-serif">09/30/2013
05:37 AM</font><font size=3> </font><font size=1 color=#5f5f5f face="sans-serif"><br>
Subject: </font><font size=1 face="sans-serif">[Openstack]
cinder volume weird behavior</font><font size=3> <br>
</font>
<hr noshade><font size=3><br>
<br>
<br>
<br>
Hello ,<br>
<br>
I have grizzly setup , in which i run cinder using IBM storvize 3700. <br>
Cinder shows a weird behavior every time i create a volume of some size
and attach it to a vm , it shows some different size .<br>
<br>
e.g i create a 4gb volume and attach it to a vm it shows of 15gb , this
is every-time different sometimes it shows a volume smaller of the size
it created.<br>
<br>
While attaching a volume to a vm sometimes i get error on compute-nodes
stating <br>
<br>
<br>
d9f36a440abdf2fdd] [instance: b9f128a9-d3e3-42a1-9511-74868b625b1b] <b>Failed
to attach volume 676ef5b1-129b-4d42-b38d-df2005a3d634 at /dev/vdc</b><br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] Traceback (most recent call last):<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 2878, in _attach_volume<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] mountpoint)<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py",
line 981, in attach_volume<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] disk_dev)<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] File "/usr/lib/python2.7/contextlib.py",
line 24, in __exit__<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] self.gen.next()<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py",
line 968, in attach_volume<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] virt_dom.attachDeviceFlags(conf.to_xml(),
flags)<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py",
line 187, in doit<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] result =
proxy_call(self._autowrap, f, *args, **kwargs)<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py",
line 147, in proxy_call<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] rv = execute(f,*args,**kwargs)<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py",
line 76, in tworker<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] rv = meth(*args,**kwargs)<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] File "/usr/lib/python2.7/dist-packages/libvirt.py",
line 422, in attachDeviceFlags<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] if ret ==
-1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)<br>
2013-09-30 15:17:</font><a href=tel:46.562%2031953 target=_blank><font size=3 color=blue><u>46.562
31953</u></font></a><font size=3> TRACE nova.compute.manager [instance:
b9f128a9-d3e3-42a1-9511-74868b625b1b] <b>libvirtError: internal error unable
to execute QEMU command 'device_add': Duplicate ID 'virtio-disk2' for device</b><br>
<br>
Then if i change the mount point from /dev/vdc to something random mount
point , it attaches the disk. But still showing different sizes problem
remains.<br>
<br>
Restarting open-iscsi services and reattaching the volume to the
vm solves the issue.<br>
<br>
Attaching my cinder.conf<br>
<br>
<br>
Has anyone encountered this problem , or any help would be really appreciated.<br>
<br>
<br>
<br>
<br>
<br>
-- </font><font size=3 color=#0000a1 face="Arial"><b><br>
With Regards <br>
Ritesh Nanda</b></font><font size=3> </font>
<p><a href=http://www.ericsson.com/ target=_blank></a><font size=1 face="Arial"><br>
<br>
[attachment "cinder.conf" deleted by Jay S Bryant/Rochester/IBM]
</font><tt><font size=3>_______________________________________________<br>
Mailing list: </font></tt><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target=_blank><tt><font size=3 color=blue><u>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</u></font></tt></a><tt><font size=3><br>
Post to : </font></tt><a href=mailto:openstack@lists.openstack.org target=_blank><tt><font size=3 color=blue><u>openstack@lists.openstack.org</u></font></tt></a><tt><font size=3><br>
Unsubscribe : </font></tt><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target=_blank><tt><font size=3 color=blue><u>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</u></font></tt></a>
<p><font size=3><br>
<br>
<br>
-- </font><font size=3 color=#0000a1 face="Arial"><b><br>
With Regards <br>
Ritesh Nanda</b></font><font size=3> </font>
<p><a href=http://www.ericsson.com/ target=_blank></a><font size=1 face="Arial"><br>
</font><font size=3><br>
</font>
<p><font size=3><br>
<br>
<br>
-- </font>
<br><font size=2 color=#0000a1 face="Arial"><b> With Regards
</b></font>
<br><font size=2 color=#0000a1 face="Arial"><b> Ritesh Nanda</b></font>
<br>
<p><a href=http://www.ericsson.com/ target=_blank></a>
<br><font size=1 face="Arial"><br>
</font>
<br>