Open Stack

On Wed, Sep 18, 2013 at 6:16 AM, Mike Perez <thingee at gmail.com> wrote:
> Folks,
>
> Currently in Havana development, RBD as ephemeral storage has serious
> stability
> and performance issues that makes the Ceph cluster a bottleneck for using an
> image as a source.
>
> Nova has to currently communicate with the external service Glance, which
> has
> to talk to the separate Ceph storage backend to fetch path information. The
> entire image is then downloaded to local disk, and then imported from local
> disk to RBD. This leaves a stability concern, especially with large images
> for
> the instance to be successfully created, due to unnecessary data pulling and
> pushing for solutions like RBD.
>
> Due to the fact we have to do a import from local disk to RBD, this can make
> performance even slower than a normal backend filesystem since the import is
> single threaded.
>
> This can be eliminated by instead having Nova's RBD image backend utility
> communicate directly with the Ceph backend to do a copy-on-write of the
> image.
> Not only does this greatly improve stability, but performance is drastically
> improved by not having to do a full copy of the image. A lot of the code to
> make this happen came from the RBD Cinder driver which has been stable and
> merged for quite a while.
>
> Bug: https://code.launchpad.net/bugs/1226351
> Patch: https://review.openstack.org/#/c/46879/1
>
> Thanks,
> Mike Perez
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


Hi Mike Perez, folks,

I absolutely agree use zero-copy approach to prepare template image is
a good idea, such as CoW. But after check your patch I have some
concerns on the currently implementation.

Actually I had prepared some dedicated BPs [1][2] and a patch [3] to
cover such requirements and problems around zero-copy (aka your
'direct_fetch') image preparing, it been implemented as a framework
and allow other people realize such plug-ins for a particular image
storage backend/location. So I'd very like to invite you (and Josh
Durgin) to take a look on them, I believe (and welcome) your stuff
within #46879 around RBD image handling can be implemented as a
RBDImangeHandler plug-ins under my framework.

I consider above implementation is better, since framework code within
#33409 can handle most common logic such as plug-ins loading, image
handler selecting base on image location, image multiple location
supporting and etc.. And each particular image handler can just
implement such special methods easily and don't need to rebuild the
existed (and tested) part.

Of cause, with the production of new handlers we probably need add
more interfaces and pass more context data to the structure /
ImageHandler base class as needed,  we can talk this in irc.

[1] https://blueprints.launchpad.net/nova/+spec/image-multiple-location
[2] https://blueprints.launchpad.net/nova/+spec/effective-template-base-image-preparing
[3] https://review.openstack.org/#/c/33409/

thanks,
zhiyan