<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<STYLE type=text/css> <!--@import url(scrollbar.css); --></STYLE>
<META content="text/html; charset=utf-8" http-equiv=Content-Type>
<STYLE> body{FONT-SIZE:12pt; FONT-FAMILY:宋体,serif;} </STYLE>
<META name=GENERATOR content="MSHTML 8.00.7600.16385"><BASE
target=_blank></HEAD>
<BODY
style="LINE-HEIGHT: 1.3; BORDER-RIGHT-WIDTH: 0px; MARGIN: 12px; BORDER-TOP-WIDTH: 0px; BORDER-BOTTOM-WIDTH: 0px; BORDER-LEFT-WIDTH: 0px"
marginheight="0" marginwidth="0">
<DIV><FONT color=#000000 size=3 face=宋体>Hi yunhong,</FONT></DIV>
<DIV>I agree with you of the taking I/O bandwidth as a resource, but it may
be not so easy to implement.</DIV>
<DIV>Your another thinking about the launch time may be not so terrible, only
the first boot it will be affected.</DIV>
<DIV> </DIV>
<DIV align=left><FONT color=#c0c0c0 size=2 face=Verdana>2014-02-17</FONT></DIV>
<DIV align=left><FONT size=2 face=Verdana>
<HR style="WIDTH: 122px; HEIGHT: 2px" id=SignNameHR align=left SIZE=2>
</FONT></DIV>
<DIV align=left><FONT color=#c0c0c0 size=2 face=Verdana><SPAN
id=_FlashSignName>Wangpan</SPAN></FONT></DIV>
<DIV><FONT size=2 face=Verdana>
<HR>
</FONT></DIV>
<DIV><FONT size=2 face=Verdana><STRONG>发件人：</STRONG>yunhong jiang
<yunhong.jiang@linux.intel.com></FONT></DIV>
<DIV><FONT size=2
face=Verdana><STRONG>发送时间：</STRONG>2014-02-15 08:21</FONT></DIV>
<DIV><FONT size=2 face=Verdana><STRONG>主题：</STRONG>Re: [openstack-dev] [nova]
Should we limit the disk IO bandwidth in copy_image while creating new
instance?</FONT></DIV>
<DIV><FONT size=2 face=Verdana><STRONG>收件人：</STRONG>"OpenStack Development
Mailing List (not for usage
questions)"<openstack-dev@lists.openstack.org></FONT></DIV>
<DIV><FONT size=2 face=Verdana><STRONG>抄送：</STRONG></FONT></DIV>
<DIV><FONT size=2 face=Verdana></FONT> </DIV>
<DIV><FONT size=2 face=Verdana>
<DIV>On Fri, 2014-02-14 at 10:22 +0100, Sylvain Bauza wrote: </DIV>
<DIV>> Instead of limitating the consumed bandwidth by proposiong a </DIV>
<DIV>> configuration flag (yet another one, and which default value to be </DIV>
<DIV>> set ?), I would propose to only decrease the niceness of the process </DIV>
<DIV>> itself, so that other processes would get first the I/O access. </DIV>
<DIV>> That's not perfect I assume, but that's a quick workaround limitating </DIV>
<DIV>> the frustration. </DIV>
<DIV>> </DIV>
<DIV>> </DIV>
<DIV>> -Sylvain </DIV>
<DIV>> </DIV>
<DIV>Decrease goodness is good for a short term, Some small concerns are, </DIV>
<DIV>will that cause long launch time if the host is I/O intensive? And if </DIV>
<DIV>launch time is billed also, then not fair for the new instance also. </DIV>
<DIV> </DIV>
<DIV>I think the ideal world is I/O QoS like through cgroup, take I/O </DIV>
<DIV>bandwidth as a resource, and take the copy_image as an consumption of </DIV>
<DIV>the I/O bandwidth resource. </DIV>
<DIV> </DIV>
<DIV>Thanks </DIV>
<DIV>--jyh </DIV>
<DIV>> </DIV>
<DIV>> 2014-02-14 4:52 GMT+01:00 Wangpan <hzwangpan@corp.netease.com>: </DIV>
<DIV>> Currently nova doesn't limit the disk IO bandwidth in </DIV>
<DIV>> copy_image() method while creating a new instance, so the </DIV>
<DIV>> other instances on this host may be affected by this high disk </DIV>
<DIV>> IO consuming operation, and some time-sensitive business(e.g </DIV>
<DIV>> RDS instance with heartbeat) may be switched between master </DIV>
<DIV>> and slave. </DIV>
<DIV>> </DIV>
<DIV>> So can we use the `rsync --bwlimit=${bandwidth} src dst` </DIV>
<DIV>> command instead of `cp src dst` while copy_image in </DIV>
<DIV>> create_image() of libvirt driver, the remote image copy </DIV>
<DIV>> operation also can be limited by `rsync --bwlimit= </DIV>
<DIV>> ${bandwidth}` or `scp -l=${bandwidth}`, this parameter </DIV>
<DIV>> ${bandwidth} can be a new configuration in nova.conf which </DIV>
<DIV>> allow cloud admin to config it, it's default value is 0 which </DIV>
<DIV>> means no limitation, then the instances on this host will be </DIV>
<DIV>> not affected while a new instance with not cached image is </DIV>
<DIV>> creating. </DIV>
<DIV>> </DIV>
<DIV>> the example codes: </DIV>
<DIV>> nova/virt/libvit/utils.py: </DIV>
<DIV>> diff --git a/nova/virt/libvirt/utils.py </DIV>
<DIV>> b/nova/virt/libvirt/utils.py </DIV>
<DIV>> index e926d3d..5d7c935 100644 </DIV>
<DIV>> --- a/nova/virt/libvirt/utils.py </DIV>
<DIV>> +++ b/nova/virt/libvirt/utils.py </DIV>
<DIV>> @@ -473,7 +473,10 @@ def copy_image(src, dest, host=None): </DIV>
<DIV>> # sparse files. I.E. holes will not be written to </DIV>
<DIV>> DEST, </DIV>
<DIV>> # rather recreated efficiently. In addition, since </DIV>
<DIV>> # coreutils 8.11, holes can be read efficiently too. </DIV>
<DIV>> - execute('cp', src, dest) </DIV>
<DIV>> + if CONF.mbps_in_copy_image > 0: </DIV>
<DIV>> + execute('rsync', '--bwlimit=%s' % </DIV>
<DIV>> CONF.mbps_in_copy_image * 1024, src, dest) </DIV>
<DIV>> + else: </DIV>
<DIV>> + execute('cp', src, dest) </DIV>
<DIV>> else: </DIV>
<DIV>> dest = "%s:%s" % (host, dest) </DIV>
<DIV>> # Try rsync first as that can compress and create </DIV>
<DIV>> sparse dest files. </DIV>
<DIV>> @@ -484,11 +487,22 @@ def copy_image(src, dest, host=None): </DIV>
<DIV>> # Do a relatively light weight test first, so </DIV>
<DIV>> that we </DIV>
<DIV>> # can fall back to scp, without having run out of </DIV>
<DIV>> space </DIV>
<DIV>> # on the destination for example. </DIV>
<DIV>> - execute('rsync', '--sparse', '--compress', </DIV>
<DIV>> '--dry-run', src, dest) </DIV>
<DIV>> + if CONF.mbps_in_copy_image > 0: </DIV>
<DIV>> + execute('rsync', '--sparse', '--compress', </DIV>
<DIV>> '--dry-run', </DIV>
<DIV>> + '--bwlimit=%s' % </DIV>
<DIV>> CONF.mbps_in_copy_image * 1024, src, dest) </DIV>
<DIV>> + else: </DIV>
<DIV>> + execute('rsync', '--sparse', '--compress', </DIV>
<DIV>> '--dry-run', src, dest) </DIV>
<DIV>> except processutils.ProcessExecutionError: </DIV>
<DIV>> - execute('scp', src, dest) </DIV>
<DIV>> + if CONF.mbps_in_copy_image > 0: </DIV>
<DIV>> + execute('scp', '-l', '%s' % </DIV>
<DIV>> CONF.mbps_in_copy_image * 1024 * 8, src, dest) </DIV>
<DIV>> + else: </DIV>
<DIV>> + execute('scp', src, dest) </DIV>
<DIV>> else: </DIV>
<DIV>> - execute('rsync', '--sparse', '--compress', src, </DIV>
<DIV>> dest) </DIV>
<DIV>> + if CONF.mbps_in_copy_image > 0: </DIV>
<DIV>> + execute('rsync', '--sparse', '--compress', </DIV>
<DIV>> + '--bwlimit=%s' % </DIV>
<DIV>> CONF.mbps_in_copy_image * 1024, src, dest) </DIV>
<DIV>> + else: </DIV>
<DIV>> + execute('rsync', '--sparse', '--compress', </DIV>
<DIV>> src, dest) </DIV>
<DIV>> </DIV>
<DIV>> </DIV>
<DIV>> 2014-02-14 </DIV>
<DIV>> </DIV>
<DIV>> ______________________________________________________________ </DIV>
<DIV>> Wangpan </DIV>
<DIV>> </DIV>
<DIV>> _______________________________________________ </DIV>
<DIV>> OpenStack-dev mailing list </DIV>
<DIV>> OpenStack-dev@lists.openstack.org </DIV>
<DIV>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev </DIV>
<DIV>> </DIV>
<DIV>> </DIV>
<DIV>> </DIV>
<DIV>> _______________________________________________ </DIV>
<DIV>> OpenStack-dev mailing list </DIV>
<DIV>> OpenStack-dev@lists.openstack.org </DIV>
<DIV>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev </DIV>
<DIV> </DIV>
<DIV> </DIV>
<DIV> </DIV>
<DIV> </DIV>
<DIV>_______________________________________________ </DIV>
<DIV>OpenStack-dev mailing list </DIV>
<DIV>OpenStack-dev@lists.openstack.org </DIV>
<DIV>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev </DIV></FONT></DIV></BODY></HTML>