<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
font-size:10.5pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
/* Page Definitions */
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="ZH-CN" link="blue" vlink="purple" style="text-justify-trim:punctuation">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">Hi openstack list,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I have two problems related to openstack volume service. I really appreciate your kind reply.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">1)Recently, we have measured the volume service performance by selecting the “tgt” as the target management software.(We use Diablo Openstack and KVM as the underlying hypervisor)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">We created 8 instances, each of which was assigned one iscsi disk from openstack volume service (RAID0 storage as the LVM volumes with up to 800 MB/s local sequential READ I/O badnwidth).
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">These 8 instances were distributed on 8 physical machines, each of which was assigned 1Gb network bandwidth.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">We measured the sequential READ (aio-stress) simultaneously from these 8 instances and identified the total READ I/O bandwidth can only reach up to ~100 MB/s.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">We observed the iscsi target was not created in “O_DIRECT” mode, (hard coded in */nova/volume/driver.py).<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">After changing to use “O_DIRECT” mode, by adding “--bsoflags=direct”, the sequential READ I/O bandwidth can reach to ~250 MB/s, greatly improved.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">My question is that, why not use “O_DIRECT” mode for “tgt” iscsi target creation? Is there any special consideration? As I know, another target management software “iet” created the iscsi target as “O_DIRECT” mode by
default.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Also, we observed that “iet” has better I/O performance than “tgt”. We measured this in 8 native system instead of instances, since our openstack (diablo) does not support iet.<br>
<br>
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">We added the “O_DIRECT” mode in the following way:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">self._execute('tgtadm', '--op', 'new',<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> '--lld=iscsi', '--mode=logicalunit',<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> "--tid=%s" % iscsi_target,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> '--lun=1',<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> "--backing-store=%s,Type=fileio" % volume_path,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> '--bsoflags=direct',<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"> run_as_root=True)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">2)Does Xenserver support volume services? Is there any document to introduce how to enable volume for Xenserver?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Because, when I attached a volume target to an instance created on Xenserver by using euca tools, no such iscsi target appeared on the instance. Also there were no error messages from log files.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Best regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Hui<o:p></o:p></span></p>
</div>
</body>
</html>