<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=gb2312">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; ">
<div>Waiting on review.openstack.org to come back live to look at the demo code and provide more accurate feedback…</div>
<div><br>
</div>
<div>Interesting and good to hear the code moved easily. The possibility to have a functional a common image transfer service wasn't questioned (IMHO). What I was stating was that we'll need strong data to point to how the common code doesn't degrade download
performance for various driver/deployments. I do think having a common set of configuration (and driver calls?) for the download options makes a lot of sense (like glance has done for image_service.download). I'm just a little more cautious when it comes
to true common download code at this point.</div>
<div><br>
</div>
<div>Christopher</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Sheng Bo Hou <<a href="mailto:sbhou@cn.ibm.com">sbhou@cn.ibm.com</a>><br>
<span style="font-weight:bold">Reply-To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Date: </span>Sunday, April 27, 2014 9:33 AM<br>
<span style="font-weight:bold">To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [openstack-dev] [Nova][blueprint] Accelerate the booting process of a number of vms via VMThunder<br>
</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div>
<div><font size="2" face="sans-serif">I have done a little test for the image download and upload. I created an API for the image access, containing copyFrom and sendTo. I moved the image download and upload code from XenApi into the implementation for Http
with some modifications, and the code worked for libvirt as well. </font><br>
<font size="2" face="sans-serif">copyFrom means to download the image and return the image data, and different hypervisors can choose to save it in a file or import it to the datastore; sendTo is used to upload the image and the image data is passed in as a
parameter.</font> <br>
<br>
<font size="2" face="sans-serif">I also did an investigation about how each hypervisor is doing the image upload and download.</font><br>
<br>
<font size="2" face="sans-serif">For the download:</font> <br>
<font size="2" face="sans-serif">libvirt, hyper-v and baremetal use the code image_service.download to download the image and save it into a file.</font><br>
<font size="2" face="sans-serif">vmwareapi uses the code image_service.download to download the image and import it into the datastore.</font><br>
<font size="2" face="sans-serif">XenAPi uses image_service.download to download the image for VHD image.</font><br>
<br>
<font size="2" face="sans-serif">For the upload:</font> <br>
<font size="2" face="sans-serif">They use image_service.upload to upload the image.</font><br>
<br>
<font size="2" face="sans-serif">I think we can conclude that it is possible to have a common image transfer library with different implementations for different protocols.</font><br>
<font size="2" face="sans-serif">This is a small demo code for the library: </font>
<a href="https://review.openstack.org/#/c/90601/"><font size="2" color="blue" face="sans-serif">https://review.openstack.org/#/c/90601/</font></a><font size="2" face="sans-serif">(Jay, is it close to the library as you mentioned?). I just replaced the upload
and download part with the http implementation for the imageapi and it worked fine.</font><br>
<br>
<font size="2" face="sans-serif">Best wishes,<br>
Vincent Hou (侯胜博)<br>
<br>
Staff Software Engineer, Open Standards and Open Source Team, Emerging Technology Institute, IBM China Software Development Lab<br>
<br>
Tel: 86-10-82450778 Fax: 86-10-82453660<br>
Notes ID: Sheng Bo Hou/China/IBM@IBMCN E-mail: <a href="mailto:sbhou@cn.ibm.com">
sbhou@cn.ibm.com</a> <br>
Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C.100193<br>
地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层邮编：100193</font> <br>
<br>
<br>
<table width="100%">
<tbody>
<tr valign="top">
<td width="40%"><font size="1" face="sans-serif"><b>Solly Ross <<a href="mailto:sross@redhat.com">sross@redhat.com</a>></b></font>
<p><font size="1" face="sans-serif">2014/04/25 01:46</font>
<table border="">
<tbody>
<tr valign="top">
<td bgcolor="white">
<div align="center"><font size="1" face="sans-serif">Please respond to<br>
"OpenStack Development Mailing List \(not for usage questions\)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>></font></div>
</td>
</tr>
</tbody>
</table>
<br>
</p>
</td>
<td width="59%">
<table width="100%">
<tbody>
<tr valign="top">
<td>
<div align="right"><font size="1" face="sans-serif">To</font></div>
</td>
<td><font size="1" face="sans-serif">"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>>,
</font></td>
</tr>
<tr valign="top">
<td>
<div align="right"><font size="1" face="sans-serif">cc</font></div>
</td>
<td></td>
</tr>
<tr valign="top">
<td>
<div align="right"><font size="1" face="sans-serif">Subject</font></div>
</td>
<td><font size="1" face="sans-serif">Re: [openstack-dev] [Nova][blueprint] Accelerate the booting process of a number of vms via VMThunder</font></td>
</tr>
</tbody>
</table>
<br>
<table>
<tbody>
<tr valign="top">
<td></td>
<td></td>
</tr>
</tbody>
</table>
<br>
</td>
</tr>
</tbody>
</table>
<br>
<br>
<br>
<tt><font size="2">Something to be aware of when planing an image transfer library is that individual drivers<br>
might have optimized support for image transfer in certain cases (especially when dealing<br>
with transferring between different formats, like raw to qcow2, etc). This builds on what<br>
Christopher was saying -- there's actually a reason why we have code for each driver. While<br>
having a common image copying library would be nice, I think a better way to do it would be to<br>
have some sort of library composed of building blocks, such that each driver could make use of<br>
common functionality while still tailoring the operation to the quirks of the particular drivers.<br>
<br>
Best Regards,<br>
Solly Ross<br>
<br>
----- Original Message -----<br>
From: "Christopher Lefelhocz" <<a href="mailto:christopher.lefelhoc@RACKSPACE.COM">christopher.lefelhoc@RACKSPACE.COM</a>><br>
To: "OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
Sent: Thursday, April 24, 2014 11:17:41 AM<br>
Subject: Re: [openstack-dev] [Nova][blueprint] Accelerate the booting process of a number of vms via VMThunder<br>
<br>
Apologies for coming to this discussion late...<br>
<br>
On 4/22/14 6:21 PM, "Jay Pipes" <<a href="mailto:jaypipes@gmail.com">jaypipes@gmail.com</a>> wrote:<br>
<br>
><br>
>Right, a good solution would allow for some flexibility via multiple<br>
>transfer drivers.<br>
<br>
+1. In particular I don't think this discussion should degenerate into<br>
zero-copy vs. pre caching. I see both as possible solutions depending on<br>
deployer/environment needs.<br>
<br>
><br>
>> Jay Pipes has suggested we figure out a blueprint for a separate<br>
>> library dedicated to the data(byte) transfer, which may be put in oslo<br>
>> and used by any projects in need (Hoping Jay can come in:-)). Huiba,<br>
>> Zhiyan, everyone else, do you think we come up with a blueprint about<br>
>> the data transfer in oslo can work?<br>
><br>
>Yes, so I believe the most appropriate solution is to create a library<br>
>-- in oslo or a standalone library like taskflow -- that would offer a<br>
>simple byte streaming library that could be used by nova.image to expose<br>
>a neat and clean task-based API.<br>
><br>
>Right now, there is a bunch of random image transfer code spread<br>
>throughout nova.image and in each of the virt drivers there seems to be<br>
>different re-implementations of similar functionality. I propose we<br>
>clean all that up and have nova.image expose an API so that a virt<br>
>driver could do something like this:<br>
><br>
>from nova.image import api as image_api<br>
><br>
>...<br>
><br>
>task = image_api.copy(from_path_or_uri, to_path_or_uri)<br>
># do some other work<br>
>copy_task_result = task.wait()<br>
><br>
>Within nova.image.api.copy(), we would use the aforementioned transfer<br>
>library to move the image bits from the source to the destination using<br>
>the most appropriate method.<br>
<br>
If I understand correctly, we'll create some common library around this.<br>
It would be good to understand the details a bit better. I've thought a<br>
bit about this issue. The one area that I get stuck at is providing a<br>
common set of downloads which work across drivers effectively. Part of<br>
the reason there are a bunch or random image transfers is historical, but<br>
also because performance was already a problem. Examples include:<br>
transferring to compute first then copying to dom0 causing performance<br>
issues, needs in some drivers to download image completely to validate<br>
prior to putting in place, etc.<br>
<br>
It may be easy to say we'll push most of this to the dom0, but I know for<br>
Xen our python stack is somewhat limited so that may be an issue.<br>
<br>
By the way we've been working on proposing a simpler image pre caching<br>
system/strategy. It focuses specifically on the image caching portion of<br>
this discussion. For those interested, see the nova-spec<br>
</font></tt><a href="https://review.openstack.org/#/c/85792"><tt><font size="2">https://review.openstack.org/#/c/85792</font></tt></a><tt><font size="2">. We'd like to leverage whatever<br>
optimized image download strategy is available.<br>
<br>
Christopher <br>
<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
</font></tt><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"><tt><font size="2">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</font></tt></a><tt><font size="2"><br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
</font></tt><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"><tt><font size="2">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</font></tt></a><tt><font size="2"><br>
<br>
</font></tt><br>
</div>
</div>
</blockquote>
</span>
</body>
</html>