Open Stack

On 17/04/2013 20:30, "Kurt Seifried" <kseifried at redhat.com> wrote:

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>On 04/17/2013 09:02 PM, Clark, Robert Graham wrote:
>> 
>> On 17/04/2013 19:57, "Kurt Seifried" <kseifried at redhat.com> wrote:
>> 
>> On 04/17/2013 05:03 PM, Clark, Robert Graham wrote:
>>>>> All, below is our draft security note for bug
>>>>> https://bugs.launchpad.net/keystone/+bug/1098177 please
>>>>> review before I release it on the general OpenStack ML.
>> 
>> So normally you guys send the finished draft to distros@ and I
>> assign it a CVE there. If you want I can start assigning the CVE
>> here and now. That sound ok?
>> 
>>>>> Thanks!
>>>>> 
>>>>> -Rob
>>>>> 
>>>>> 
>>>>> Requests with large POST body can crash Pre-Grizzly Keystone
>>>>> or underlying services. -----
>>>>> 
>>>>> ### Summary ### Concurrent Keystone POST requests with large
>>>>> body messages are held in memory without filtering or rate
>>>>> limiting, this can lead to resource exhaustion on the
>>>>> Keystone server.
>>>>> 
>>>>> ### Affected Services / Software ### Keystone, Databases
>>>>> 
>>>>> ### Discussion ### Keystone stores POST messages in memory
>>>>> before validation, concurrent submission of multiple large
>>>>> POST messages can cause the Keystone process to be killed due
>>>>> to memory exhaustion, resulting in a remote Denial of
>>>>> Service.
>>>>> 
>>>>> In many cases Keystone will be deployed behind a
>>>>> load-balancer or proxy that can rate limit POST messages
>>>>> inbound to Keystone. Grizzly is protected against that
>>>>> through the sizelimit middleware.
>>>>> 
>>>>> ### Recommended Actions ### If you are in a situation where
>>>>> Keystone is directly exposed to incoming POST messages and
>>>>> not protected by the sizelimit middleware there are a number
>>>>> of load-balancing/proxy options, we suggest you consider one
>>>>> of the following:
>
>
>> Hi Kurt,
>> 
>> This isn't being considered as an 'OpenStack Vulnerability' as
>> suchŠ
>> 
>> OpenStack Security Notes exist to guide users and implementers of
>> OpenStack through various security 'pain-points'. Security Notes do
>> not directly address vulnerabilities in OpenStack. OSNs provide
>> guidance to ensure secure use of OpenStack and will often provide
>> work arounds or advice for 3rd party libraries and services used in
>> conjunction with OpenStack.
>> 
>> These notes are a product of the OSSG. You should probably reach
>> out to the VMT if you believe that a CVE is required. I've sent
>> this around for comments on -security this evening and I'll publish
>> it (with any changes) tomorrow morning (west-coast).
>> 
>> -Rob
>
>Ok but this sounds like a classic web DoS (send some big requests to
>server, servers falls over/stays busy for a long time).
>
>"Concurrent Keystone POST requests with large body messages are held
>in memory without filtering or rate limiting, this can lead to
>resource exhaustion on the Keystone server. "
>
>If this was brought up to me internally at Red Hat I would have 1)
>assigned a CVE and then 2) notified upstream, this definitely is a
>security flaw.
>
>- -- 
>Kurt Seifried Red Hat Security Response Team (SRT)
>PGP: 0x5E267993 A90B F995 7350 148F 66BF 7554 160D 4553 5E26 7993
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.4.13 (GNU/Linux)
>
>iQIcBAEBAgAGBQJRb2jqAAoJEBYNRVNeJnmT/MQP/0uWYezwNki3Q/d7Y9+NXucc
>i2RYTtuTPvZaT87ilRIIJxYTFkuza7m/bmlayI+jft8wI+NswRWlXA5MtnKdl8Kp
>9htyzM4QSrlqWjsBYT1mAvK/wgYwb6dDi+DsEzfAQIbiFP0IJX5ZOvX7thCqt2vX
>40DcUFANhEJu+78S0MgNgVdBJxtXWSNbizZdDEgWWaZqUVT0uigFBUWnz4razHcL
>aCjsJWDVGkORjmXLAea/P+gmA5/CO8tF9tTElwwVbtsNK/XN+LVBptC2k6/06Er9
>YSF42kUPRUDnnxF4tjjPW+vBiSOcu5XPDy2geELVo8tTB0SIq6r7rmCnpx31XbiI
>xA0VjUOtL60is/iDzaVK/U1Jv+j0lpv8vTkJNLPZGt5IZqmt4+Zf5SOaemp7WOMP
>IiZNoKK4Xp21orAD013cGOZ4vCDndZHzTS9X6hInrw4e3Iz2fm+ab0cQyroyY3Ox
>WjZPeV/JLpPEYWO10jfL12jBlkeBpzufri96iyI01bzc6XVzuY/IX3ZiYncaNM+1
>UO4WG2b4qOU/O2YtOVk4hvV/E2AMFTkHsmmlE9GWXJPWUe+dlSlDQMuXJ2vad+M+
>8fYa/gniheeZ0DCdlRx1aSPtMGQvoOYAoTkSBtBqb526zTLYnNQGYjS9S7orypeA
>1qBCWow304un3wDfDAan
>=e+cJ
>-----END PGP SIGNATURE-----

I agree with you. I'm not currently responsible for how OpenStack handles
issues and wether they're considered as 'vulnerabilities' though the OSSG
will be assisting with that process in the near future.

There's discussion of the issue here:
https://bugs.launchpad.net/keystone/+bug/1098177 I believe the request for
us to cut a OSN in response to this was due to the fact that it doesn't
affect Grizzly and most people who would have an vulnerable attack surface
(web facing etc) would already be running Keystone behind Nginx, WAFs,
LB's etc.

I can hold the draft while you create a CVE and we can reference that in
the released OSN, you should probably approach the VMT about the CVE or
comment on the bug perhaps?

-Rob