Open Stack

On 13/01/15 21:24 -0500, Jay Pipes wrote:
>On 01/13/2015 04:55 PM, Boden Russell wrote:
>>Looking for some feedback from the glance dev team on a potential BP…
>>
>>The use case I’m trying to solve is —>
>>
>>"As an admin, I want my glance image bits replicated to multiple store
>>locations (of the same store type) during a glance create operation."
>>
>>For example, I have 3 HTTP based backend locations I want to store my
>>glance image bits on. When I create / upload a new glance image, I want
>>those bits put onto all 3 HTTP based locations and have the image's
>>'locations' metadata properly reflect all stored locations.
>>
>>There are obviously multiple approaches to getting this done.
>>
>>[1] Allow per glance store drivers the ability to manage config and
>>"connectivity" to multiple backends. For example in the glance-api.conf:
>>
>>[DEFAULT]
>>store_backends = http1,http2,http3
>>...
>>[http1]
>># http 1 backend props
>>...
>>[http2]
>># http 2 backend props
>>...
>>[http2]
>># http 2 backend props
>>...
>>
>>And then in the HTTP store driver use a configuration approach like
>>cinder multi-backend does (e.g.:
>>https://github.com/openstack/cinder/blob/2f09c3031ef2d2db598ec4c56f6127e33d29b2cc/cinder/volume/configuration.py#L52).
>>Here, the store driver handles all the logic w/r/t pushing the image
>>bits to all backends, etc..
>
>The problem with this solution is that the HTTP Glance storage backend 
>is readonly. You cannot upload an image to Glance using the http 
>backend.
>
>>[2] A separate (3rd party) "process" which handles the image replication
>>and location metadata updates... For example listens for the glance
>>notification on create and then takes the steps necessary to replicate
>>the bits elsewhere and update the image metadata (locations).
>
>This is the solution that I would recommend. Frankly, this kind of 
>replication should be an async out-of-band process similar to 
>bittorrent. Just have bittorrent or rsync or whatever replicate the 
>image bits to a set of target locations and then call the 
>glanceclient.v2.client.images.add_location() method:
>
>https://github.com/openstack/python-glanceclient/blob/master/glanceclient/v2/images.py#L211
>
>to add the URI of the replicated image bits.

It recently landed in Glance an async workers engine (?) that allows
for this kind of things to exists. For instance, it'll be used for
image introspection to extract information from images after they have
been *imported* into glance.

The right hooks that trigger this async workers maybe need to be
defined better but the infrastructure is there. Once that's more
solid, you'll be able to write your own plugin that will do that job
on every glance image import.

That said, I'd suggest you to follow Jay's advice for now.

Cheers,
Flavio

>
>>[3] etc...
>>
>>
>>In a prototype I implemented #1 which can be done with no impact outside
>>of the store driver code itself.
>
>I'm not entirely sure how you did that considering the http storage 
>backend is readonly. Are you saying you implemented the add() method 
>for the glance_store._drivers.http.Store class?
>
>Best,
>-jay
>
>> I prefer #1 over #2 given approach #2
>>may need pull the image bits back down from the initial location in
>>order to push for replication; additional processing.
>>
>>Is the dev team adverse to option #1 for the store driver's who wish to
>>implement it and / or what are the other (preferred) options here?
>>
>>
>>Thank you,
>>- boden
>>
>>
>>__________________________________________________________________________
>>OpenStack Development Mailing List (not for usage questions)
>>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 
@flaper87
Flavio Percoco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150114/a10d9056/attachment.pgp>