[openstack-dev] [nova-compute][nova][libvirt] Extending Nova-Compute for image prefetching

John Garbutt john at johngarbutt.com
Thu Oct 15 09:20:14 UTC 2015


On 10 October 2015 at 09:35, Alberto Geniola <albertogeniola at gmail.com>
wrote:

> Hi Michael,
>
> and thank you for your answer.
>
> Indeed, what I want to do is to add methods to the ImageCache.py module
> (listing, adding, deleting). So far, this module only takes care of image
> deletion: this represents the "cache" of images. Now, I want to populate
> the cache with some images on the hypervisor (as you mention) without
> having any instance running it, yet. The method I want like to add should
> call the appropriate method from the hypervisor driver (let's say libvirt)
> to trigger the image download/creation without starting it (I guess
> something like calling _create_image() should do the trick).
>
> Your question is actually good: how will an user be  able to trigger this
> image caching mechanism?
> My idea is to extend the HTTP Nova Compute API. I would lilke to add a
> resource, let's say "CachedImages", to the API tree. In this way,
> interacting via CRUD operation we should be able to CALL/CAST rpc api and
> interact with the imagecache module.
>
> Is it clearer now? Do you see any problem in this approach?
>

I think prefetching is a problem that needs to be solved.

It would be great if you can submit a nova-spec on this, so we can discuss
it there:
https://github.com/openstack/nova-specs#readme

>From previous discussions, I like the idea of an API to tell a specific
compute node to add an image into its cache. Probably need to specify an
expiry time, for when it gets deleted if never used, etc.

In terms of a CRUD API, I am less sure about the cost/benefit. Thats mostly
because an RPC call in the API always goes badly (with timeouts, etc), and
adding DB tables feels like overkill, and creates a state sync problem. But
maybe I am not seeing the full picture there.

This seems like a good thing to discuss in the spec review, where there
will be more context around the use cases, etc.

Thanks,
johnthetubaguy




>
> On Thu, Oct 8, 2015 at 11:45 PM, Michael Still <mikal at stillhq.com> wrote:
>
>> I think I'd rephrase your definition of pre-fetched to be honest --
>> something more like "images on this hypervisor node without a currently
>> running instance". So, your operations would become:
>>
>>  - trigger an image prefetching
>>  - list unused base images (and perhaps when they were last used)
>>  - delete an unused image
>>
>> All of that would need to tie into the image cache management code so
>> that its not stomping on your images. In fact, you're probably best of
>> adding all of this as tweaks to the image cache manager anyways.
>>
>> One question though -- who is calling these APIs? Are you adding a
>> central service to orchestrate these calls?
>>
>> Michael
>>
>>
>>
>> On Thu, Oct 8, 2015 at 10:50 PM, Alberto Geniola <
>> albertogeniola at gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I'm considering to extend the Nova-Compute API in order to provide
>>> image-prefetching capabilities to OS.
>>>
>>> In order to allow image prefetching, I ended up with the need to add
>>> three different APIs on the nova-compute nodes:
>>>
>>>   1. Trigger an image prefetching
>>>
>>>   2. List prefetched images
>>>
>>>   3. Delete a prefetched image
>>>
>>>
>>>
>>> About the point 1 I saw I can re-use the libvirt driver function
>>> _create_image() to trigger the image prefetching. However, this approach
>>> will not store any information about the stored image locally. This leads
>>> to an issue: how do I retrieve a list of already fetched images? A quick
>>> and simple possibility would be having a local file, storing information
>>> about the fetched images. Would it be acceptable? Is there any best
>>> practice in OS community?
>>>
>>>
>>>
>>> Any ideas?
>>>
>>>
>>> Ty,
>>>
>>> Al.
>>>
>>> --
>>> Dott. Alberto Geniola
>>>
>>>   albertogeniola at gmail.com
>>>   +39-346-6271105
>>>   https://www.linkedin.com/in/albertogeniola
>>>
>>> Web: http://www.hw4u.it
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Rackspace Australia
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Dott. Alberto Geniola
>
>   albertogeniola at gmail.com
>   +39-346-6271105
>   https://www.linkedin.com/in/albertogeniola
>
> Web: http://www.hw4u.it
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151015/4b4d076d/attachment.html>


More information about the OpenStack-dev mailing list