Open Stack

On 1/14/15 1:38 AM, Flavio Percoco wrote:
> On 13/01/15 21:24 -0500, Jay Pipes wrote:
>> On 01/13/2015 04:55 PM, Boden Russell wrote:
>>> Looking for some feedback from the glance dev team on a potential BP…
>>
>> This is the solution that I would recommend. Frankly, this kind of
>> replication should be an async out-of-band process similar to
>> bittorrent. Just have bittorrent or rsync or whatever replicate the
>> image bits to a set of target locations and then call the
>> glanceclient.v2.client.images.add_location() method:
>>
>> https://github.com/openstack/python-glanceclient/blob/master/glanceclient/v2/images.py#L211
>>
>>
>> to add the URI of the replicated image bits.
> 
> It recently landed in Glance an async workers engine (?) that allows
> for this kind of things to exists. For instance, it'll be used for
> image introspection to extract information from images after they have
> been *imported* into glance.
> 
> The right hooks that trigger this async workers maybe need to be
> defined better but the infrastructure is there. Once that's more
> solid, you'll be able to write your own plugin that will do that job
> on every glance image import.
> 

While I understand the motivation for suggesting the "out of band"
approach (async workers or separate process), my major concern here is
the additional processing required. In my particular scenario this would
require the out of band process to pull the image bits back down from
the remote location and then push them back up to the replication
locations. If the image size is decent, this could be a fairly expensive
operation. Moreover an out of band process (IMO) would make for a less
than optimal user experience as users would have to query the image
locations metadata to understand if the image has replicated yet.
Perhaps async workers improves this user experience a bit (can query
worker status), but it still seems cleaner (IMO) to have the replication
happen in-line with the image create flow.


>>> In a prototype I implemented #1 which can be done with no impact outside
>>> of the store driver code itself.
>>
>> I'm not entirely sure how you did that considering the http storage
>> backend is readonly. Are you saying you implemented the add() method
>> for the glance_store._drivers.http.Store class?

I was trying to generalize my use case to other glance store drivers,
but my generalization using the http store driver was obviously a poor
choice... My interest and PoC is based on the VMware datastore driver.


Let me ask more directly -- if we wanted to enhance the VMware datastore
driver to support replication (as I described in approach #1 of my
initial email) is this something the community would consider (assume
changes are contained to the VMware datastore driver), or would such an
enhancement be an uphill battle to get reviewed / merged?