Open Stack

On 09/17/2013 07:33 PM, Zhi Yan Liu wrote:
> On Wed, Sep 18, 2013 at 6:16 AM, Mike Perez <thingee at gmail.com> wrote:
>> Folks,
>>
>> Currently in Havana development, RBD as ephemeral storage has serious
>> stability
>> and performance issues that makes the Ceph cluster a bottleneck for using an
>> image as a source.
>>
>> Nova has to currently communicate with the external service Glance, which
>> has
>> to talk to the separate Ceph storage backend to fetch path information. The
>> entire image is then downloaded to local disk, and then imported from local
>> disk to RBD. This leaves a stability concern, especially with large images
>> for
>> the instance to be successfully created, due to unnecessary data pulling and
>> pushing for solutions like RBD.
>>
>> Due to the fact we have to do a import from local disk to RBD, this can make
>> performance even slower than a normal backend filesystem since the import is
>> single threaded.
>>
>> This can be eliminated by instead having Nova's RBD image backend utility
>> communicate directly with the Ceph backend to do a copy-on-write of the
>> image.
>> Not only does this greatly improve stability, but performance is drastically
>> improved by not having to do a full copy of the image. A lot of the code to
>> make this happen came from the RBD Cinder driver which has been stable and
>> merged for quite a while.
>>
>> Bug: https://code.launchpad.net/bugs/1226351
>> Patch: https://review.openstack.org/#/c/46879/1
>>
>> Thanks,
>> Mike Perez
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> Hi Mike Perez, folks,
>
> I absolutely agree use zero-copy approach to prepare template image is
> a good idea, such as CoW. But after check your patch I have some
> concerns on the currently implementation.
>
> Actually I had prepared some dedicated BPs [1][2] and a patch [3] to
> cover such requirements and problems around zero-copy (aka your
> 'direct_fetch') image preparing, it been implemented as a framework
> and allow other people realize such plug-ins for a particular image
> storage backend/location. So I'd very like to invite you (and Josh
> Durgin) to take a look on them, I believe (and welcome) your stuff
> within #46879 around RBD image handling can be implemented as a
> RBDImangeHandler plug-ins under my framework.

I like the plugin framework you've created there, but it was not
merged for Havana, which is why I did not use it in this patch. I
didn't want to include more new code than necessary. I simply used the
same approach that cinder uses to minimize the risk of regression.

> I consider above implementation is better, since framework code within
> #33409 can handle most common logic such as plug-ins loading, image
> handler selecting base on image location, image multiple location
> supporting and etc.. And each particular image handler can just
> implement such special methods easily and don't need to rebuild the
> existed (and tested) part.

I'm fine with that approach, and the direct fetch method in my patch
can be easily refactored into an RBDImageHandler like you suggest with
a little bit more information passed to the image handlers. I'm happy
to help with this for Icehouse.

Thanks!
Josh

> Of cause, with the production of new handlers we probably need add
> more interfaces and pass more context data to the structure /
> ImageHandler base class as needed,  we can talk this in irc.
>
> [1] https://blueprints.launchpad.net/nova/+spec/image-multiple-location
> [2] https://blueprints.launchpad.net/nova/+spec/effective-template-base-image-preparing
> [3] https://review.openstack.org/#/c/33409/
>
> thanks,
> zhiyan
>