Open Stack

Thanks for the comments Nathan.

I wanted to check if references to 3rd party tools are allowed in the
reference section.


On Sat, Jan 4, 2014 at 10:11 AM, Nathan Kinder <nkinder at redhat.com> wrote:

> I would slightly reword the "Summary" section to be a bit more clear.
> I'd suggest something like this:
>
> -------------
> There is currently no limitation on the number of VNC sessions that can be
> established for a single user's VNC token.  This enables one to cause a
> Denial of Service (DoS) style attack by establishing many VNC sessions
> against a single instance through a noVNC proxy.  This can cause timeouts
> for other users who are trying to access the same instance through VNC.
> -------------
>
> In the "Discussion" section, I don't think the first sentence is needed
> ("NoVNC Proxy is explained here").  There are also some areas where I
> would suggest slight rewording:
>
> -------------
> Once a user gets a token to access an instance's VNC console, there is no
> restriction on the frequency that the user can attempt to connect to the
> instance's VNC console using that token.  There is also no restriction on
> the number of simultaneous VNC sessions that the user can establish against
> the instance using a single token.  If many connection requests are made,
> any subsequent connection requests made by other users may time out.  This
> could also impact other user's currently established VNC sessions to the
> instance.  The overall responsiveness of other Nova services running on the
> noVNC host.
>
> By taking advantage of the lack of any VNC connection limiting, a single
> user could cause the noVNC proxy endpoint to be non-responsive or
> unreachable.  This results in a DoS attack.  It should be noted that there
> is no amplification effect.
> -------------
>
> I think the OSSN needs to mention Havana as well.  It currently only
> indicates that Grizzly is affected in the "Affected Services" and
> "Recommended Actions" sections.
>
> You have a few spots that use "NoVNC" or "novnc".  These should be
> "noVNC".
>
> In the "Recommended Actions" section, you start to make a reference to
> some best practices but there is no reference.  Were you intending to
> refer to the Security Guide?
>
> --
> You received this bug notification because you are a member of OpenStack
> Security Group, which is subscribed to OpenStack.
> https://bugs.launchpad.net/bugs/1227575
>
> Title:
>   DoS style attack on noVNC server can lead to service interruption or
>   disruption
>
> Status in OpenStack Compute (Nova):
>   In Progress
> Status in OpenStack Security Notes:
>   New
>
> Bug description:
>   There is no limiting on the number of VNC sessions that can be created
> for a single user's VNC token.
>   Any attempt to create multiple (say hundreds or thousands) of websocket
> connections to the VNC server
>   results in many connection timeouts. Due to these connection timeout
> error, other users trying to access their
>   VM's VNC console cannot do so.
>
>   A sample script that tries to create 100,000 connections to Nova noVNC
> proxy, shows timeout errors
>   Script: http://paste.openstack.org/show/47254/
>
>   Script output.... connections get timed out after a while
>   -------------------
>   ....
>   ..
>
>   Creating Connection
>   Receiving...
>   Received 'RFB 003.008
>   '
>   Creating Connection
>   Receiving...
>   Received 'RFB 003.008
>   '
>   Creating Connection
>   Receiving...
>   Received 'RFB 003.008
>   '
>   Creating Connection
>   Receiving...
>   Received 'RFB 003.008
>   '
>   Creating Connection
>   Receiving...
>   Received 'RFB 003.008
>   '
>   Creating Connection
>   Receiving...
>   Received 'RFB 003.008
>   '
>   Creating Connection
>   Receiving...
>   timed out
>   Creating Connection
>   Receiving...
>   timed out
>   Creating Connection
>   Receiving...
>   timed out
>   Creating Connection
>   Receiving...
>   timed out
>   Creating Connection
>   Receiving...
>   timed out
>   --------------------
>
>   Impact:
>   1. Many of the sessions timeout. Any attempt to open other sessions also
> intermittently fail.
>   This can cause serious problems to users already having a running VNC
> session or trying to create new sessions.
>
>   2. The overall performance and response times of other nova services
> running on the novnc host, using tcp protocol
>   also gets affected after the connection timeout errors.
>
>   For example:
>   Before running the sumulate thousands of connections program:
>   $ time nova get-vnc-console c1b093a3-f53b-4282-b89c-e68f0fa1b6e5  novnc
>
> +-------+---------------------------------------------------------------------------------+
>   | Type  | Url
>                   |
>
> +-------+---------------------------------------------------------------------------------+
>   | novnc |
> http://10.2.3.102:6080/vnc_auto.html?token=e776dd33-422f-4b56-9f98-e317410d0212|
>
> +-------+---------------------------------------------------------------------------------+
>
>   real    0m0.751s
>   user    0m0.376s
>   sys     0m0.084s
>
>   rohit at precise-dev-102:~/tools/websocket-client-0.7.0$
>
>   After running the program, the response time is quite high:
>   $ time nova get-vnc-console c1b093a3-f53b-4282-b89c-e68f0fa1b6e5  novnc
>
>
> +-------+---------------------------------------------------------------------------------+
>   | Type  | Url
>                   |
>
> +-------+---------------------------------------------------------------------------------+
>   | novnc |
> http://10.2.3.102:6080/vnc_auto.html?token=6865d675-d852-478b-b1ee-457b092f11b9|
>
> +-------+---------------------------------------------------------------------------------+
>
>   real    3m9.231s
>   user    0m0.424s
>   sys     0m0.108s
>
>
>   Possible solutions:
>   1. Allow just 1 session per instance, and raise a new exception, say,
> VNCSessionAlreadyExists to reject multiple
>   connections for the same token, and return an error code to the user.
>   2. Make the number of sessions allowed per instance configurable,
> limited by some count of sessions.
>
>   However, both of these solutions may need to override and modify the
> do_proxy() method of websockify's WebSocketProxy class,
>   which can lead to maintenance issues.
>
>   Another possible solution would be to implement some kind of callback
> function in websockify, to which we can pass the token
>   for reconnection. This would first need contribution to the websockify
> project code, and then update Nova.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nova/+bug/1227575/+subscriptions
>
> _______________________________________________
> Openstack-security mailing list
> Openstack-security at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-security
>


-- 
Thanks,
-Sriram

-- 
You received this bug notification because you are a member of OpenStack
Security Group, which is subscribed to OpenStack.
https://bugs.launchpad.net/bugs/1227575

Title:
  DoS style attack on noVNC server can lead to service interruption or
  disruption

Status in OpenStack Compute (Nova):
  In Progress
Status in OpenStack Security Notes:
  New

Bug description:
  There is no limiting on the number of VNC sessions that can be created for a single user's VNC token.
  Any attempt to create multiple (say hundreds or thousands) of websocket connections to the VNC server
  results in many connection timeouts. Due to these connection timeout error, other users trying to access their 
  VM's VNC console cannot do so.

  A sample script that tries to create 100,000 connections to Nova noVNC proxy, shows timeout errors
  Script: http://paste.openstack.org/show/47254/

  Script output.... connections get timed out after a while
  -------------------
  ....
  ..

  Creating Connection
  Receiving...
  Received 'RFB 003.008
  '
  Creating Connection
  Receiving...
  Received 'RFB 003.008
  '
  Creating Connection
  Receiving...
  Received 'RFB 003.008
  '
  Creating Connection
  Receiving...
  Received 'RFB 003.008
  '
  Creating Connection
  Receiving...
  Received 'RFB 003.008
  '
  Creating Connection
  Receiving...
  Received 'RFB 003.008
  '
  Creating Connection
  Receiving...
  timed out
  Creating Connection
  Receiving...
  timed out
  Creating Connection
  Receiving...
  timed out
  Creating Connection
  Receiving...
  timed out
  Creating Connection
  Receiving...
  timed out
  --------------------

  Impact:
  1. Many of the sessions timeout. Any attempt to open other sessions also intermittently fail.
  This can cause serious problems to users already having a running VNC session or trying to create new sessions.

  2. The overall performance and response times of other nova services running on the novnc host, using tcp protocol
  also gets affected after the connection timeout errors.

  For example:
  Before running the sumulate thousands of connections program:
  $ time nova get-vnc-console c1b093a3-f53b-4282-b89c-e68f0fa1b6e5  novnc
  +-------+---------------------------------------------------------------------------------+
  | Type  | Url                                                                             |
  +-------+---------------------------------------------------------------------------------+
  | novnc | http://10.2.3.102:6080/vnc_auto.html?token=e776dd33-422f-4b56-9f98-e317410d0212 |
  +-------+---------------------------------------------------------------------------------+

  real    0m0.751s
  user    0m0.376s
  sys     0m0.084s

  rohit at precise-dev-102:~/tools/websocket-client-0.7.0$

  After running the program, the response time is quite high:
  $ time nova get-vnc-console c1b093a3-f53b-4282-b89c-e68f0fa1b6e5  novnc

  +-------+---------------------------------------------------------------------------------+
  | Type  | Url                                                                             |
  +-------+---------------------------------------------------------------------------------+
  | novnc | http://10.2.3.102:6080/vnc_auto.html?token=6865d675-d852-478b-b1ee-457b092f11b9 |
  +-------+---------------------------------------------------------------------------------+

  real    3m9.231s
  user    0m0.424s
  sys     0m0.108s

  
  Possible solutions:
  1. Allow just 1 session per instance, and raise a new exception, say, VNCSessionAlreadyExists to reject multiple
  connections for the same token, and return an error code to the user.
  2. Make the number of sessions allowed per instance configurable, limited by some count of sessions.

  However, both of these solutions may need to override and modify the do_proxy() method of websockify's WebSocketProxy class, 
  which can lead to maintenance issues.

  Another possible solution would be to implement some kind of callback function in websockify, to which we can pass the token
  for reconnection. This would first need contribution to the websockify project code, and then update Nova.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1227575/+subscriptions