I agree with this being class D (security hardening opportunity). Since
the report is already public, we can just go ahead and switch the
security advisory task to won't fix, then change it back later if there
are objections. I've done so now.
** Changed in: ossa
Status: Incomplete => Won't Fix
--
You received this bug notification because you are a member of OpenStack
Security, which is subscribed to OpenStack.
https://bugs.launchpad.net/bugs/1572719
Title:
Client may hold socket open after ChunkWriteTimeout
Status in OpenStack Security Advisory:
Won't Fix
Status in OpenStack Object Storage (swift):
Confirmed
Bug description:
You can reproduce this by issuing a GET request for a few hundred MB
file and never consuming the response, but keep the client socket
open. Swift will log a 499 but the socket does not always close.
ChunkWriteTimeout is meant to protect the proxy from a slow reading client:
https://github.com/openstack/swift/blob/master/swift/proxy/controllers/base.py#L889-L905
Sometimes when this exception is thrown there is still data in the process socket buffer, so when eventlet tries to close the socket it first flushes it:
https://github.com/eventlet/eventlet/blob/master/eventlet/wsgi.py#L631
https://hg.python.org/cpython/file/v2.7.11/Lib/SocketServer.py#l711
The problem is that if the client is not consuming the socket buffer
then that flush will wait forever; it's trying to write on a socket
that just threw a timeout trying to write! The flush write is not
protected by any timeout.
As far as I can tell, this WRITE_TIMEOUT does nothing:
https://github.com/openstack/swift/blob/master/swift/common/wsgi.py#L407
wsgi.server() takes a socket_timeout that might be what we're after?
https://github.com/openstack/swift/blob/master/swift/common/wsgi.py#L437-L440
Even with socket_timeout, eventlet needs to be patched. This should be in a finally block:
https://github.com/eventlet/eventlet/blob/master/eventlet/wsgi.py#L636-L637
All of this is probably mitigated by most operators setting an idle
timeout in their load balancers, but I wanted to report it. Going
directly to a proxy I was able to hold sockets open for long periods
of time.
I did the initial research on version 2.2.2 but I was able to
reproduce on 2.7.0. I'm trying to translate links to master branch on
github. I apologize in advance if it's not quite right.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ossa/+bug/1572719/+subscriptions