Open Stack

On 01/13/2015 04:55 PM, Boden Russell wrote:
> Looking for some feedback from the glance dev team on a potential BP…
>
> The use case I’m trying to solve is —>
>
> "As an admin, I want my glance image bits replicated to multiple store
> locations (of the same store type) during a glance create operation."
>
> For example, I have 3 HTTP based backend locations I want to store my
> glance image bits on. When I create / upload a new glance image, I want
> those bits put onto all 3 HTTP based locations and have the image's
> 'locations' metadata properly reflect all stored locations.
>
> There are obviously multiple approaches to getting this done.
>
> [1] Allow per glance store drivers the ability to manage config and
> "connectivity" to multiple backends. For example in the glance-api.conf:
>
> [DEFAULT]
> store_backends = http1,http2,http3
> ...
> [http1]
> # http 1 backend props
> ...
> [http2]
> # http 2 backend props
> ...
> [http2]
> # http 2 backend props
> ...
>
> And then in the HTTP store driver use a configuration approach like
> cinder multi-backend does (e.g.:
> https://github.com/openstack/cinder/blob/2f09c3031ef2d2db598ec4c56f6127e33d29b2cc/cinder/volume/configuration.py#L52).
> Here, the store driver handles all the logic w/r/t pushing the image
> bits to all backends, etc..

The problem with this solution is that the HTTP Glance storage backend 
is readonly. You cannot upload an image to Glance using the http backend.

> [2] A separate (3rd party) "process" which handles the image replication
> and location metadata updates... For example listens for the glance
> notification on create and then takes the steps necessary to replicate
> the bits elsewhere and update the image metadata (locations).

This is the solution that I would recommend. Frankly, this kind of 
replication should be an async out-of-band process similar to 
bittorrent. Just have bittorrent or rsync or whatever replicate the 
image bits to a set of target locations and then call the 
glanceclient.v2.client.images.add_location() method:

https://github.com/openstack/python-glanceclient/blob/master/glanceclient/v2/images.py#L211

to add the URI of the replicated image bits.

> [3] etc...
>
>
> In a prototype I implemented #1 which can be done with no impact outside
> of the store driver code itself.

I'm not entirely sure how you did that considering the http storage 
backend is readonly. Are you saying you implemented the add() method for 
the glance_store._drivers.http.Store class?

Best,
-jay

 > I prefer #1 over #2 given approach #2
> may need pull the image bits back down from the initial location in
> order to push for replication; additional processing.
>
> Is the dev team adverse to option #1 for the store driver's who wish to
> implement it and / or what are the other (preferred) options here?
>
>
> Thank you,
> - boden
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>