<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; "><div>Hi,</div><div><br></div><div>About the volume services support in Xenserver, the iSCSI driver should work simply by starting the volume service with default flags. </div><div><br></div><div>You could take a look at the volumes.sh exercises script (<a href="https://github.com/openstack-dev/devstack/blob/master/exercises/volumes.sh">https://github.com/openstack-dev/devstack/blob/master/exercises/volumes.sh</a>) to check if you missed something.</div><div><br></div><div>If you would like to use the storage manager volume driver for Xen, (<a href="http://docs.openstack.org/trunk/openstack-compute/admin/content/xensm.html">http://docs.openstack.org/trunk/openstack-compute/admin/content/xensm.html</a>), here are the configuration details: <a href="http://docs.openstack.org/trunk/openstack-compute/admin/content/xensmconfig.html">http://docs.openstack.org/trunk/openstack-compute/admin/content/xensmconfig.html</a></div><div><br></div><div>Thanks,</div><div>Renuka.</div><div><br></div><span id="OLK_SRC_BODY_SECTION"><div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt"><span style="font-weight:bold">From: </span> "Lv, Hui" <<a href="mailto:hui.lv@intel.com">hui.lv@intel.com</a>><br><span style="font-weight:bold">Date: </span> Mon, 12 Mar 2012 08:04:55 -0700<br><span style="font-weight:bold">To: </span> "<a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a>" <<a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a>><br><span style="font-weight:bold">Subject: </span> [Openstack] two problems about the volume service<br></div><div><br></div><div xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><meta name="Generator" content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
font-size:10.5pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
/* Page Definitions */
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--><div lang="ZH-CN" link="blue" vlink="purple" style="text-justify-trim:punctuation"><div class="WordSection1"><p class="MsoNormal"><span lang="EN-US">Hi openstack list,<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p><p class="MsoNormal"><span lang="EN-US">I have two problems related to openstack volume service. I really appreciate your kind reply.<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p><p class="MsoNormal"><span lang="EN-US">1)Recently, we have measured the volume service performance by selecting the “tgt” as the target management software.(We use Diablo Openstack and KVM as the underlying hypervisor)<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US">We created 8 instances, each of which was assigned one iscsi disk from openstack volume service (RAID0 storage as the LVM volumes with up to 800 MB/s local sequential READ I/O badnwidth).
<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US">These 8 instances were distributed on 8 physical machines, each of which was assigned 1Gb network bandwidth.<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US">We measured the sequential READ (aio-stress) simultaneously from these 8 instances and identified the total READ I/O bandwidth can only reach up to ~100 MB/s.<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US">We observed the iscsi target was not created in “O_DIRECT” mode, (hard coded in */nova/volume/driver.py).<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US">After changing to use “O_DIRECT” mode, by adding “--bsoflags=direct”, the sequential READ I/O bandwidth can reach to ~250 MB/s, greatly improved.<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p><p class="MsoNormal"><span lang="EN-US">My question is that, why not use “O_DIRECT” mode for “tgt” iscsi target creation? Is there any special consideration? As I know, another target management software “iet” created the iscsi target as “O_DIRECT” mode by
default.<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US">Also, we observed that “iet” has better I/O performance than “tgt”. We measured this in 8 native system instead of instances, since our openstack (diablo) does not support iet.<br><br><o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US">We added the “O_DIRECT” mode in the following way:<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US">self._execute('tgtadm', '--op', 'new',<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US"> '--lld=iscsi', '--mode=logicalunit',<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US"> "--tid=%s" % iscsi_target,<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US"> '--lun=1',<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US"> "--backing-store=%s,Type=fileio" % volume_path,<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US"> '--bsoflags=direct',<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US"> run_as_root=True)<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p><p class="MsoNormal"><span lang="EN-US">2)Does Xenserver support volume services? Is there any document to introduce how to enable volume for Xenserver?<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US">Because, when I attached a volume target to an instance created on Xenserver by using euca tools, no such iscsi target appeared on the instance. Also there were no error messages from log files.<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p><p class="MsoNormal"><span lang="EN-US">Best regards,<o:p></o:p></span></p><p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p><p class="MsoNormal"><span lang="EN-US">Hui<o:p></o:p></span></p></div></div></div></span></body></html>