Open Stack

On 06/07/2015 04:22 PM, Robert Collins wrote:
> On 6 June 2015 at 13:08, Ian Cordasco <ian.cordasco at rackspace.com> wrote:

>> So the problem is with how we use ResponseSerializer and the ChunkedFile
>> (https://git.openstack.org/cgit/openstack/glance/tree/glance/api/v2/image_d
>> ata.py#n222). I think the problem we'll have is that webob provides
>> nothing on a Response
>> (https://webob.readthedocs.org/en/latest/modules/webob.html#response) to
>> hook into so we can close the ChunkedFile.
>>
>> I wonder if we used the body_file attribute if webob would close the file
>> when the response is closed (because I'm assuming that nova/glanceclient
>> are closing the response with which it's downloading the data).
>
> But the maximum leak time is a single GC run, which we don't expect to
> be long, unless the server is super quiet (and if it is, the temporary
> leak is less like to be an issue, no?).
>
> I wonder if there's actually something else going on here. E.g. a
> broken LB in front of glance-api which is preventing the HTTP
> connection termination from being detected, and the thread is staying
> open-and-stalled. That would explain the symptoms just as well - and
> is deployer specific so could also explain the (perceived) trickiness
> in reproduction.

Hi, original reporter here.

There's no LB involved.  The issue was noticed in a test lab that is tight on 
disk space.  When an instance failed to boot the person using the lab tried to 
delete some images to free up space, at which point it was noticed that space 
didn't actually free up.  (For at least half an hour, exact time unknown.)

I'm more of a nova guy, so could you elaborate a bit on the GC?  Is something 
going to delete the ChunkedFile object after a certain amount of inactivity? 
What triggers the GC to run?

Thanks,
Chris