Open Stack

Right, our discussion ends up with two directions to access the image: 
full-copying(upload and download the whole image) and 
zero-copying(remotely access the image and copy on demand).
For image transfer via "full-copying", nova.image module works, because we 
only care about how the data(bytes/image) is going to send. It does not 
matter what the hypervisor is and what the storage it is. I think we can 
take the library approach with task = image_api.copy(from_path_or_uri, 
to_path_or_uri) as Jay proposed.
For image transfer via "zero-copying" as Zhi Yan mentioned, depends on the 
hypervisor and the backend storage. nova.virt is able to access hypervisor 
and storage  context, but not nova.image. The image does not have to be in 
the same location, where to launch the VM. We care about how the 
data(image) is accessed(how the VM is launched.)
Each of them needs one module to put, and it is hard to merge them.

Best wishes,
Vincent Hou (侯胜博)

Staff Software Engineer, Open Standards and Open Source Team, Emerging 
Technology Institute, IBM China Software Development Lab

Tel: 86-10-82450778 Fax: 86-10-82453660
Notes ID: Sheng Bo Hou/China/IBM at IBMCN    E-mail: sbhou at cn.ibm.com 
Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang 
West Road, Haidian District, Beijing, P.R.C.100193
地址:北京市海淀区东北旺西路8号中关村软件园28号楼环宇大厦3层 邮编：100193



Zhi Yan Liu <lzy.dev at gmail.com> 
2014/04/24 14:30
Please respond to
"OpenStack Development Mailing List \(not for usage questions\)" 
<openstack-dev at lists.openstack.org>


To
"OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev at lists.openstack.org>, 
cc

Subject
Re: [openstack-dev] [Nova][blueprint] Accelerate the booting process of a 
number of vms via VMThunder






First, after the discussing between I and Vincent, I'm sure what we
talked for "zero-copying" in above mail threads is different with
"full-copying"/transferring. The transferring/full-copying means image
bits duplication, it focuses on using some method to accelerate image
bits replication and transferring between Glance/backend-storage and
Nova compute host, like P2P, FTP, HTTP, etc., but zero-copying means
there is NO any bits duplication and transferring happened between
those two sides during the image preparing for VM provisioning, so the
latter one focuses on a) how to attaching remote image volume/disk
within Glance managed backend-storage from Nova compute host directly,
and b) making the VM's root disk from remote template image  based on
such hypervisor and particular storage technology, btw c) preventing
image bits uploading for VM snapshot/capture case. They are totally
different to me. (refer: review comments in
https://review.openstack.org/#/c/84671/ )

Second, on the implementation level, I consider to put all image
handling related code into nova.image namespace sounds neat but I
think it can not work (the "leak" as last mail side here). IMO, the
transferring/full-copying logic is more applicable for nova.image
namespace, such transferring approach can be implemented based on
existing download module plugins structure, e.g. P2P, FTP, but for the
zero-copying, regarding to my above points of view, I consider to
implement it in nova.virt + nova.virt.<hypervisor> is make more sense,
since it's more related with particular hypervisor and/or storage
technology. (refer: inline comments in
https://review.openstack.org/#/c/86583/6/specs/juno/image-multiple-location.rst

)

zhiyan

On Wed, Apr 23, 2014 at 11:02 PM, Jay Pipes <jaypipes at gmail.com> wrote:
> On Wed, 2014-04-23 at 13:56 +0800, lihuiba wrote:
>> >For live migration, we use shared storage so I don't think it's quite
>> >the same as getting/putting image bits from/to arbitrary locations.
>> With a good zero-copy transfer lib, live migration support can be
>> extended to non-shared storage, or cross-datacenter. It's a kind of
>> value.
>
> Hmm, I totally see the value of doing this. Not sure that there could be
> the same kinds of "liveness" guarantees with non-shared-storage, but I
> am certainly happy to see a proof of concept in this area! :)
>
>> >task = image_api.copy(from_path_or_uri, to_path_or_uri)
>> ># do some other work
>> >copy_task_result = task.wait()
>> +1  looks cool!
>> how about zero-copying?
>
> It would be an implementation detail within nova.image.api.copy()
> function (and the aforementioned "image bits mover library") :)
>
> The key here is to leak as little implementation detail out of the
> nova.image.api module
>
> Best,
> -jay
>
>> At 2014-04-23 07:21:27,"Jay Pipes" <jaypipes at gmail.com> wrote:
>> >Hi Vincent, Zhi, Huiba, sorry for delayed response. See comments 
inline.
>> >
>> >On Tue, 2014-04-22 at 10:59 +0800, Sheng Bo Hou wrote:
>> >> I actually support the idea Huiba has proposed, and I am thinking of
>> >> how to optimize the large data transfer(for example, 100G in a short
>> >> time) as well.
>> >> I registered two blueprints in nova-specs, one is for an image 
upload
>> >> plug-in to upload the image to
>> >> glance(https://review.openstack.org/#/c/84671/), the other is a data
>> >> transfer plug-in(https://review.openstack.org/#/c/87207/) for data
>> >> migration among nova nodes. I would like to see other transfer
>> >> protocols, like FTP, bitTorrent, p2p, etc, implemented for data
>> >> transfer in OpenStack besides HTTP.
>> >>
>> >> Data transfer may have many use cases. I summarize them into two
>> >> catalogs. Please feel free to comment on it.
>> >> 1. The machines are located in one network, e.g. one domain, one
>> >> cluster, etc. The characteristic is the machines can access each 
other
>> >> directly via the IP addresses(VPN is beyond consideration). In this
>> >> case, data can be transferred via iSCSI, NFS, and definitive 
zero-copy
>> >> as Zhiyan mentioned.
>> >> 2. The machines are located in different networks, e.g. two data
>> >> centers, two firewalls, etc. The characteristic is the machines can
>> >> not access each other directly via the IP addresses(VPN is beyond
>> >> consideration). The machines are isolated, so they can not be
>> >> connected with iSCSI, NFS, etc. In this case, data have to go via 
the
>> >> protocols, like HTTP, FTP, p2p, etc. I am not sure whether zero-copy
>> >> can work for this case. Zhiyan, please help me with this doubt.
>> >>
>> >> I guess for data transfer, including image downloading, image
>> >> uploading, live migration, etc, OpenStack needs to taken into 
account
>> >> the above two catalogs for data transfer.
>> >
>> >For live migration, we use shared storage so I don't think it's quite
>> >the same as getting/putting image bits from/to arbitrary locations.
>> >
>> >>  It is hard to say that one protocol is better than another, and one
>> >> approach prevails another(BitTorrent is very cool, but if there is
>> >> only one source and only one target, it would not be that faster 
than
>> >> a direct FTP). The key is the use
>> >> case(FYI:
http://amigotechnotes.wordpress.com/2013/12/23/file-transmission-with-different-sharing-solution-on-nas/
).
>> >
>> >Right, a good solution would allow for some flexibility via multiple
>> >transfer drivers.
>> >
>> >> Jay Pipes has suggested we figure out a blueprint for a separate
>> >> library dedicated to the data(byte) transfer, which may be put in 
oslo
>> >> and used by any projects in need (Hoping Jay can come in:-)). Huiba,
>> >> Zhiyan, everyone else, do you think we come up with a blueprint 
about
>> >> the data transfer in oslo can work?
>> >
>> >Yes, so I believe the most appropriate solution is to create a library
>> >-- in oslo or a standalone library like taskflow -- that would offer a
>> >simple byte streaming library that could be used by nova.image to 
expose
>> >a neat and clean task-based API.
>> >
>> >Right now, there is a bunch of random image transfer code spread
>> >throughout nova.image and in each of the virt drivers there seems to 
be
>> >different re-implementations of similar functionality. I propose we
>> >clean all that up and have nova.image expose an API so that a virt
>> >driver could do something like this:
>> >
>> >from nova.image import api as image_api
>> >
>> >...
>> >
>> >task = image_api.copy(from_path_or_uri, to_path_or_uri)
>> ># do some other work
>> >copy_task_result = task.wait()
>> >
>> >Within nova.image.api.copy(), we would use the aforementioned transfer
>> >library to move the image bits from the source to the destination 
using
>> >the most appropriate method.
>> >
>> >Best,
>> >-jay
>> >
>> >
>> >_______________________________________________
>> >OpenStack-dev mailing list
>> >OpenStack-dev at lists.openstack.org
>> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140424/1f58a280/attachment.html>