Open Stack

First, after the discussing between I and Vincent, I'm sure what we
talked for "zero-copying" in above mail threads is different with
"full-copying"/transferring. The transferring/full-copying means image
bits duplication, it focuses on using some method to accelerate image
bits replication and transferring between Glance/backend-storage and
Nova compute host, like P2P, FTP, HTTP, etc., but zero-copying means
there is NO any bits duplication and transferring happened between
those two sides during the image preparing for VM provisioning, so the
latter one focuses on a) how to attaching remote image volume/disk
within Glance managed backend-storage from Nova compute host directly,
and b) making the VM's root disk from remote template image  based on
such hypervisor and particular storage technology, btw c) preventing
image bits uploading for VM snapshot/capture case. They are totally
different to me. (refer: review comments in
https://review.openstack.org/#/c/84671/ )

Second, on the implementation level, I consider to put all image
handling related code into nova.image namespace sounds neat but I
think it can not work (the "leak" as last mail side here). IMO, the
transferring/full-copying logic is more applicable for nova.image
namespace, such transferring approach can be implemented based on
existing download module plugins structure, e.g. P2P, FTP, but for the
zero-copying, regarding to my above points of view, I consider to
implement it in nova.virt + nova.virt.<hypervisor> is make more sense,
since it's more related with particular hypervisor and/or storage
technology. (refer: inline comments in
https://review.openstack.org/#/c/86583/6/specs/juno/image-multiple-location.rst
)

zhiyan

On Wed, Apr 23, 2014 at 11:02 PM, Jay Pipes <jaypipes at gmail.com> wrote:
> On Wed, 2014-04-23 at 13:56 +0800, lihuiba wrote:
>> >For live migration, we use shared storage so I don't think it's quite
>> >the same as getting/putting image bits from/to arbitrary locations.
>> With a good zero-copy transfer lib, live migration support can be
>> extended to non-shared storage, or cross-datacenter. It's a kind of
>> value.
>
> Hmm, I totally see the value of doing this. Not sure that there could be
> the same kinds of "liveness" guarantees with non-shared-storage, but I
> am certainly happy to see a proof of concept in this area! :)
>
>> >task = image_api.copy(from_path_or_uri, to_path_or_uri)
>> ># do some other work
>> >copy_task_result = task.wait()
>> +1  looks cool!
>> how about zero-copying?
>
> It would be an implementation detail within nova.image.api.copy()
> function (and the aforementioned "image bits mover library") :)
>
> The key here is to leak as little implementation detail out of the
> nova.image.api module
>
> Best,
> -jay
>
>> At 2014-04-23 07:21:27,"Jay Pipes" <jaypipes at gmail.com> wrote:
>> >Hi Vincent, Zhi, Huiba, sorry for delayed response. See comments inline.
>> >
>> >On Tue, 2014-04-22 at 10:59 +0800, Sheng Bo Hou wrote:
>> >> I actually support the idea Huiba has proposed, and I am thinking of
>> >> how to optimize the large data transfer(for example, 100G in a short
>> >> time) as well.
>> >> I registered two blueprints in nova-specs, one is for an image upload
>> >> plug-in to upload the image to
>> >> glance(https://review.openstack.org/#/c/84671/), the other is a data
>> >> transfer plug-in(https://review.openstack.org/#/c/87207/) for data
>> >> migration among nova nodes. I would like to see other transfer
>> >> protocols, like FTP, bitTorrent, p2p, etc, implemented for data
>> >> transfer in OpenStack besides HTTP.
>> >>
>> >> Data transfer may have many use cases. I summarize them into two
>> >> catalogs. Please feel free to comment on it.
>> >> 1. The machines are located in one network, e.g. one domain, one
>> >> cluster, etc. The characteristic is the machines can access each other
>> >> directly via the IP addresses(VPN is beyond consideration). In this
>> >> case, data can be transferred via iSCSI, NFS, and definitive zero-copy
>> >> as Zhiyan mentioned.
>> >> 2. The machines are located in different networks, e.g. two data
>> >> centers, two firewalls, etc. The characteristic is the machines can
>> >> not access each other directly via the IP addresses(VPN is beyond
>> >> consideration). The machines are isolated, so they can not be
>> >> connected with iSCSI, NFS, etc. In this case, data have to go via the
>> >> protocols, like HTTP, FTP, p2p, etc. I am not sure whether zero-copy
>> >> can work for this case. Zhiyan, please help me with this doubt.
>> >>
>> >> I guess for data transfer, including image downloading, image
>> >> uploading, live migration, etc, OpenStack needs to taken into account
>> >> the above two catalogs for data transfer.
>> >
>> >For live migration, we use shared storage so I don't think it's quite
>> >the same as getting/putting image bits from/to arbitrary locations.
>> >
>> >>  It is hard to say that one protocol is better than another, and one
>> >> approach prevails another(BitTorrent is very cool, but if there is
>> >> only one source and only one target, it would not be that faster than
>> >> a direct FTP). The key is the use
>> >> case(FYI:http://amigotechnotes.wordpress.com/2013/12/23/file-transmission-with-different-sharing-solution-on-nas/).
>> >
>> >Right, a good solution would allow for some flexibility via multiple
>> >transfer drivers.
>> >
>> >> Jay Pipes has suggested we figure out a blueprint for a separate
>> >> library dedicated to the data(byte) transfer, which may be put in oslo
>> >> and used by any projects in need (Hoping Jay can come in:-)). Huiba,
>> >> Zhiyan, everyone else, do you think we come up with a blueprint about
>> >> the data transfer in oslo can work?
>> >
>> >Yes, so I believe the most appropriate solution is to create a library
>> >-- in oslo or a standalone library like taskflow -- that would offer a
>> >simple byte streaming library that could be used by nova.image to expose
>> >a neat and clean task-based API.
>> >
>> >Right now, there is a bunch of random image transfer code spread
>> >throughout nova.image and in each of the virt drivers there seems to be
>> >different re-implementations of similar functionality. I propose we
>> >clean all that up and have nova.image expose an API so that a virt
>> >driver could do something like this:
>> >
>> >from nova.image import api as image_api
>> >
>> >...
>> >
>> >task = image_api.copy(from_path_or_uri, to_path_or_uri)
>> ># do some other work
>> >copy_task_result = task.wait()
>> >
>> >Within nova.image.api.copy(), we would use the aforementioned transfer
>> >library to move the image bits from the source to the destination using
>> >the most appropriate method.
>> >
>> >Best,
>> >-jay
>> >
>> >
>> >_______________________________________________
>> >OpenStack-dev mailing list
>> >OpenStack-dev at lists.openstack.org
>> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev