<HTML xmlns="http://www.w3.org/TR/REC-html40" xmlns:v = "urn:schemas-microsoft-com:vml" xmlns:o = "urn:schemas-microsoft-com:office:office" xmlns:w = "urn:schemas-microsoft-com:office:word" xmlns:m = "http://schemas.microsoft.com/office/2004/12/omml"><HEAD>
<META content="text/html; charset=utf-8" http-equiv=Content-Type>
<META name=Generator content="Microsoft Word 15 (filtered medium)">
<STYLE>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</STYLE>
<STYLE><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></STYLE>
</HEAD>
<BODY lang=EN-GB link=blue vLink=purple>
<DIV>
<DIV style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif">I'll make those corrections, thanks Rob.</DIV></DIV>
<DIV dir=ltr>
<HR>
<SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif; FONT-WEIGHT: bold">From: </SPAN><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif"><A href="mailto:robert.clark@hp.com">Clark, Robert Graham</A></SPAN><BR><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif; FONT-WEIGHT: bold">Sent: </SPAN><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif">3/8/2014 12:51 PM</SPAN><BR><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif; FONT-WEIGHT: bold">To: </SPAN><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif"><A href="mailto:sriram@sriramhere.com">Sriram Subramanian</A>; <A href="mailto:1227575@bugs.launchpad.net">Bug 1227575</A>; <A href="mailto:openstack-security@lists.openstack.org">openstack-security@lists.openstack.org</A></SPAN><BR><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif; FONT-WEIGHT: bold">Subject: </SPAN><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri,sans-serif">RE: [Openstack-security] [Bug 1227575] Re: DoS style attack onnoVNCserver can lead to service interruption or disruption</SPAN><BR><BR></DIV>
<DIV class=WordSection1>
<P class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; mso-fareast-language: EN-US">The OSSN looks great, but I think perhaps the summary could be tweaked to be a little shorter and maybe flow a little better…<o:p></o:p></SPAN></P>
<P class=MsoNormal><o:p> </o:p></P>
<P class=MsoNormal>There is currently no limit to the number of noVNC or SPICE console sessions that can be established by a single user. The console host has limited resources and an attacker launching many sessions may be able to exhaust the available resources, resulting in a Denial of Service (DoS) condition. <SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; mso-fareast-language: EN-US"><o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; mso-fareast-language: EN-US"><o:p> </o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; mso-fareast-language: EN-US">Other than that it’s hot-to-trot, as they say.<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; mso-fareast-language: EN-US"><o:p> </o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; mso-fareast-language: EN-US">-Rob<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; mso-fareast-language: EN-US"><o:p> </o:p></SPAN></P>
<DIV>
<DIV style="BORDER-TOP: #e1e1e1 1pt solid; BORDER-RIGHT: medium none; BORDER-BOTTOM: medium none; PADDING-BOTTOM: 0cm; PADDING-TOP: 3pt; PADDING-LEFT: 0cm; BORDER-LEFT: medium none; PADDING-RIGHT: 0cm">
<P class=MsoNormal><B><SPAN lang=EN-US style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'">From:</SPAN></B><SPAN lang=EN-US style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'"> Sriram Subramanian [mailto:sriram@sriramhere.com] <BR><B>Sent:</B> 08 March 2014 06:03<BR><B>To:</B> Bug 1227575; openstack-security@lists.openstack.org<BR><B>Subject:</B> Re: [Openstack-security] [Bug 1227575] Re: DoS style attack on noVNCserver can lead to service interruption or disruption<o:p></o:p></SPAN></P></DIV></DIV>
<P class=MsoNormal><o:p> </o:p></P>
<DIV>
<DIV>
<P class=MsoNormal><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'">Thanks Nathan, good to publish!<o:p></o:p></SPAN></P></DIV></DIV>
<DIV>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center>
<HR align=center SIZE=2 width="100%">
</DIV>
<P class=MsoNormal style="MARGIN-BOTTOM: 12pt"><B><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'">From: </SPAN></B><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'"><A href="mailto:nkinder@redhat.com">Nathan Kinder</A></SPAN><BR><B><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'">Sent: </SPAN></B><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'">3/7/2014 8:33 PM</SPAN><BR><B><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'">To: </SPAN></B><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'"><A href="mailto:openstack-security@lists.openstack.org">openstack-security@lists.openstack.org</A></SPAN><BR><B><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'">Subject: </SPAN></B><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'">[Openstack-security] [Bug 1227575] Re: DoS style attack on noVNCserver can lead to service interruption or disruption</SPAN><o:p></o:p></P></DIV>
<P class=MsoNormal>Here's an updated OSSN draft that covers all of the issues mentioned in<BR>my previous comment (in the appropriate e-mail format). I also removed<BR>the reference to rate-limiting in the Security Guide, as it doesn't<BR>actually contain any information on how to do rate limiting. It only<BR>mentioned that QOSaaS plans to offer rate limiting, which isn't<BR>applicable here.<BR><BR>If this looks fine to everyone else, I'll go ahead and publish it.<BR><BR>-----------------------------------------------------------------------------------------<BR>DoS style attack on noVNC server can lead to service interruption or disruption<BR>---<BR><BR>### Summary ###<BR>There is currently no limit on the number of noVNC or SPICE console<BR>sessions that can be established against a single user token. This<BR>enables launching a Denial of Service (DoS) style attack by establishing<BR>many console sessions against a single virtual machine instance through<BR>the console proxy. This can cause instance access timeouts and general<BR>service response degradation on the console host.<BR><BR>### Affected Services / Software ###<BR>Horizon, Nova, noVNC proxy, SPICE console, Grizzly, Havana<BR><BR>### Discussion ###<BR>Currently with a single user token, no restrictions are enforced on the<BR>number or frequency of noVNC or SPICE console sessions that may be<BR>established. While a user can only access their own virtual machine<BR>instances, resources can be exhausted on the console proxy host by<BR>creating an excessive number of simultaneous console sessions. This can<BR>result in timeouts for subsequent connection requests to instances using<BR>the same console proxy. Not only would this prevent the user from<BR>accessing their own instances, but other legitimate users would also be<BR>deprived of console access. Further, other services running on the<BR>noVNC proxy and Compute hosts may degrade in responsiveness.<BR><BR>By taking advantage of this lack of restrictions around noVNC or SPICE<BR>console connections, a single user could cause the console proxy<BR>endpoint to become unresponsive, resulting in a Denial Of Service (DoS)<BR>style attack. It should be noted that there is no amplification effect.<BR><BR>### Recommended Actions ###<BR>For current stable OpenStack releases (Grizzly, Havana), users need to<BR>workaround this vulnerability by using rate-limiting proxies to cover<BR>access to the noVNC proxy service. Rate-limiting is a common mechanism<BR>to prevent DoS and Brute-Force attacks.<BR><BR>For example, if you are using a proxy such as Repose, enable the rate<BR>limiting feature by following these steps:<BR><BR> <A href="https://repose.atlassian.net/wiki/display/REPOSE/Rate+Limiting+Filter">https://repose.atlassian.net/wiki/display/REPOSE/Rate+Limiting+Filter</A><BR><BR>Future OpenStack releases are looking to add the ability to restrict<BR>noVNC and SPICE console connections.<BR><BR>### Contacts / References ###<BR>This OSSN : <A href="https://wiki.openstack.org/wiki/OSSN/OSSN-0008">https://wiki.openstack.org/wiki/OSSN/OSSN-0008</A><BR>Original LaunchPad Bug : <A href="https://bugs.launchpad.net/nova/+bug/1227575">https://bugs.launchpad.net/nova/+bug/1227575</A><BR>OpenStack Security ML : <A href="mailto:openstack-security@lists.openstack.org">openstack-security@lists.openstack.org</A><BR>OpenStack Security Group : <A href="https://launchpad.net/~openstack-ossg">https://launchpad.net/~openstack-ossg</A><BR><BR>-- <BR>You received this bug notification because you are a member of OpenStack<BR>Security Group, which is subscribed to OpenStack.<BR><A href="https://bugs.launchpad.net/bugs/1227575">https://bugs.launchpad.net/bugs/1227575</A><BR><BR>Title:<BR> DoS style attack on noVNC server can lead to service interruption or<BR> disruption<BR><BR>Status in OpenStack Compute (Nova):<BR> In Progress<BR>Status in OpenStack Security Notes:<BR> New<BR><BR>Bug description:<BR> There is no limiting on the number of VNC sessions that can be created for a single user's VNC token.<BR> Any attempt to create multiple (say hundreds or thousands) of websocket connections to the VNC server<BR> results in many connection timeouts. Due to these connection timeout error, other users trying to access their <BR> VM's VNC console cannot do so.<BR><BR> A sample script that tries to create 100,000 connections to Nova noVNC proxy, shows timeout errors<BR> Script: <A href="http://paste.openstack.org/show/47254/">http://paste.openstack.org/show/47254/</A><BR><BR> Script output.... connections get timed out after a while<BR> -------------------<BR> ....<BR> ..<BR><BR> Creating Connection<BR> Receiving...<BR> Received 'RFB 003.008<BR> '<BR> Creating Connection<BR> Receiving...<BR> Received 'RFB 003.008<BR> '<BR> Creating Connection<BR> Receiving...<BR> Received 'RFB 003.008<BR> '<BR> Creating Connection<BR> Receiving...<BR> Received 'RFB 003.008<BR> '<BR> Creating Connection<BR> Receiving...<BR> Received 'RFB 003.008<BR> '<BR> Creating Connection<BR> Receiving...<BR> Received 'RFB 003.008<BR> '<BR> Creating Connection<BR> Receiving...<BR> timed out<BR> Creating Connection<BR> Receiving...<BR> timed out<BR> Creating Connection<BR> Receiving...<BR> timed out<BR> Creating Connection<BR> Receiving...<BR> timed out<BR> Creating Connection<BR> Receiving...<BR> timed out<BR> --------------------<BR><BR> Impact:<BR> 1. Many of the sessions timeout. Any attempt to open other sessions also intermittently fail.<BR> This can cause serious problems to users already having a running VNC session or trying to create new sessions.<BR><BR> 2. The overall performance and response times of other nova services running on the novnc host, using tcp protocol<BR> also gets affected after the connection timeout errors.<BR><BR> For example:<BR> Before running the sumulate thousands of connections program:<BR> $ time nova get-vnc-console c1b093a3-f53b-4282-b89c-e68f0fa1b6e5 novnc<BR> +-------+---------------------------------------------------------------------------------+<BR> | Type | Url |<BR> +-------+---------------------------------------------------------------------------------+<BR> | novnc | <A href="http://10.2.3.102:6080/vnc_auto.html?token=e776dd33-422f-4b56-9f98-e317410d0212">http://10.2.3.102:6080/vnc_auto.html?token=e776dd33-422f-4b56-9f98-e317410d0212</A> |<BR> +-------+---------------------------------------------------------------------------------+<BR><BR> real 0m0.751s<BR> user 0m0.376s<BR> sys 0m0.084s<BR><BR> <A href="mailto:rohit@precise-dev-102:~/tools/websocket-client-0.7.0$">rohit@precise-dev-102:~/tools/websocket-client-0.7.0$</A><BR><BR> After running the program, the response time is quite high:<BR> $ time nova get-vnc-console c1b093a3-f53b-4282-b89c-e68f0fa1b6e5 novnc<BR><BR> +-------+---------------------------------------------------------------------------------+<BR> | Type | Url |<BR> +-------+---------------------------------------------------------------------------------+<BR> | novnc | <A href="http://10.2.3.102:6080/vnc_auto.html?token=6865d675-d852-478b-b1ee-457b092f11b9">http://10.2.3.102:6080/vnc_auto.html?token=6865d675-d852-478b-b1ee-457b092f11b9</A> |<BR> +-------+---------------------------------------------------------------------------------+<BR><BR> real</P></DIV><BR>
<DIV>[The entire original message is not included.]</DIV></BODY></HTML>