[openstack-dev] [oslo][glance] what to do about tons of http connection pool is full warnings in g-api log?
Ben Nemec
openstack at nemebean.com
Mon Jul 14 22:18:32 UTC 2014
On 07/14/2014 04:21 PM, Matt Riedemann wrote:
>
>
> On 7/14/2014 4:09 PM, Matt Riedemann wrote:
>> I opened bug 1341777 [1] against glance but it looks like it's due to
>> the default log level for requests.packages.urllib3.connectionpool in
>> oslo's log module.
>>
>> The problem is this warning shows up nearly 420K times in 7 days in
>> Tempest runs:
>>
>> WARNING urllib3.connectionpool [-] HttpConnectionPool is full,
>> discarding connection: 127.0.0.1
>>
>> So either glance is doing something wrong, or that's logging too high of
>> a level (I think it should be debug in this case). I'm not really sure
>> how to scope this down though, or figure out what is so damn chatty in
>> glance-api that is causing this. It doesn't seem to be causing test
>> failures, but the rate at which this is logged in glance-api is surprising.
>>
>> [1] https://bugs.launchpad.net/glance/+bug/1341777
>>
>
> I found this older thread [1] which led to this in oslo [2] but I'm not
> really sure how to use it to make the connectionpool logging quieter in
> glance, any guidance there? It looks like in Joe's change to nova for
> oslo.messaging he just changed the value directly in the log module in
> nova, something I thought was forbidden.
>
> [1]
> http://lists.openstack.org/pipermail/openstack-dev/2014-March/030763.html
> [2] https://review.openstack.org/#/c/94001/
>
There was a change recently in incubator to address something related,
but since it's setting to WARN I don't think it would get rid of this
message:
https://github.com/openstack/oslo-incubator/commit/3310d8d2d3643da2fc249fdcad8f5000866c4389
It looks like Joe's change was a cherry-pick of the incubator change to
add oslo.messaging, so discouraged but not forbidden (and apparently
during feature freeze, which is understandable).
-Ben
More information about the OpenStack-dev
mailing list