Open Stack

On Tue, May 17, 2011 at 11:59 AM, Glen Campbell
<glen.campbell at rackspace.com> wrote:
> If we are going to deploy Glance to support a global deployment of Nova, would it make sense to have replicas in different regions for better performance?
> Or, to put it another way, is there a recommended way to keep multiple Glance installations in sync?

Hi Glen!

I think a better idea than having multiple copies of an image in
different regions is to do two things:

a) Use a proxy caching server like Varnish or Squid to cache pieces or
all of an image in various zones
b) Use a highly-available storage system like Swift behind the global
Glance server

For a) we need to complete the HTTP 1.1 Cache headers blueprint
(https://blueprints.launchpad.net/glance/+spec/caching) and for b) you
would simply use the Swift backend, configured appropriately for a
large Swift cluster.

> Users doing snapshots/backups, etc., would presumably get better performance if Glance was local, but how would we keep the base/shared images in sync?

This is actually something that Rick H and Chris McG are working on.
The basic strategy that they came up with was to add a parent ID
attribute to the image and for any snapshot image, simply refer to the
base image as the snapshot image's parent. The glance client would
check for a parent_id that wasn't null and continue streaming the
image while it found a parent URI/ID.

For example, let's say you have a "golden image" with the URI:
http://glance.example.com/12345. A user creates an instance with this
image and some time later, decides to do a snapshot or backup of their
running instance. The snapshotting code in the virtualization layer
produces what is essentially a differential snapshot, containing only
the differing bits of the existing image with the base golden image.
This snapshot (typically much smaller than the original image) could
be stored in the local (zone-local) Glance server with a call to POST
/images. When pushing this snapshot image to the local Glance server,
we would set the parent ID to http://glance.example.com/12345.

Let's say at some later time, the user wanted to restore from this
backup. The virtualization layer that implemented the restore call
would need to stream the backup image from the local Glance server. In
doing so, it would use the glance client class' get_image() method.
When calling this method, the glance client would first return the
snapshot image piece. Noticing the image had a parent ID, it would
continue to stream the golden image from the global image Glance
server in-line, essentially enabling us to store only the small diff
of the snapshot locally while streaming the bulk of the image master
from the global Glance server.

I'll let Rick elaborate on the above and correct any mistakes I made
in my description. :)

-jay