Open Stack

Adam:
       I agree with your point on QOS being used to address quality rather than protection,  but the lines surely blur.  Especially for this use case as we are trying to protect against excessive traffic.   Only the volume of this traffic is harmful to other tenants and the network,  not the traffic itself.   For example, when QoS is used to restrict the traffic from a tenant to ensure the network doesn’t become congested, that ensure network quality AND protect against excessive traffic.  That said, I am not hung up that this must be a QoS feature.   We just want a way to protect the network, this tenant and other tenants.
     If FWaaS handles the traffic restriction in a centralized manner at a centralized point, then I don’t see how this will properly handle the use case.   If a single VM on a compute host is excessive in creating connections this has the potential to  degrade all VMs on that host unless the restriction is on that VM’s port.   So I am not in favor of handling this via FWaaS unless the implementation will allow the granularity to be finer than the router level.
   If the rules and rates are configured against the router but effectively applied against all ports on the networks that connect to the router then this might be workable.  I guess I should study the FWaaS plans a bit more completely.
John

From: Adam Lawson [mailto:alawson at aqorn.com]
Sent: Monday, June 29, 2015 12:56 AM
To: OpenStack Development Mailing List (not for usage questions)
Cc: lionel.zerbib at huawei.com; Derek Chamorro (dechamor); Eran Gampel
Subject: Re: [openstack-dev] [Neutron] [QOS] Request for Additional QoS capabilities


If fwaas code is at the router level, it would seem that null routing might be one method of handling ddos, making fwaas a potentially suitable program. Qos seems to address quality issues rather than focusing on protection from unauthorized or malicious traffic whether that traffic originates from the inside or externally. That seems again, to me, as a fwaas-centric function.

Just my two cents. ;)
On Jun 28, 2015 6:19 PM, "John Joyce (joycej)" <joycej at cisco.com<mailto:joycej at cisco.com>> wrote:
Gal:
     I am also slow to jump between this and other work so I think I should be the one apologizing.  I think we are receptive to any of the approaches QoS, FWaaS or Security Groups.  I am not an expert on FWaaS but from a quick look it seemed like the FWaaS granularity was at the router level.  We would want this per neutron port (e.g. per VM port although don’t want to limit the possibility for this be per container or per bare metal port).   Allowing an aggregate across all ports of the VM would be very nice,  but not a strict requirement.   Do you see this as an issue going the FWaaS route?   Have you been making any headway getting it in there?
     One detailed comment after looking through the reviews.   Would there be any issue in adding a “reject-with-tcp-reset” option?
    The DDOS coming from a VM could be due to a virus within the VM our maybe just an overly aggressive tenant.  I know the team that runs our cloud offering has experienced an excessive connection requests coming from a VM.   I can try to get the exact scenario that triggered this.   The net is that all tenants on that host can be affected,  especially with an OVS based vswitch.
         John





From: Gal Sagie [mailto:gal.sagie at gmail.com<mailto:gal.sagie at gmail.com>]
Sent: Tuesday, June 23, 2015 2:43 PM
To: OpenStack Development Mailing List (not for usage questions)
Cc: lionel.zerbib at huawei.com<mailto:lionel.zerbib at huawei.com>; Derek Chamorro (dechamor); Eran Gampel
Subject: Re: [openstack-dev] [Neutron] [QOS] Request for Additional QoS capabilities

Hi John,

Sorry for the delayed response as i was on vacation with no internet connection (you don't know how much
you miss it until you don't have it).

The work in terms of coding is pretty much done for the reference implementation.
We initially tried to push it as a security group extension but there is a strong objection
to change the security group API, so FWaaS can be next best candidate if we can find support
or other uses of this (like your use case)
(Of course that work will need to be added for supporting the connection limit, we tried
to  tackle brute force prevention which i personally see as a more concerning attack vector internally)

Out of curiosity can you describe scenarios of DDoS attacking from an internal VM ?
I would assume most DDoS will happen from external traffic or a combine attack from various internal
VM's (and then this might no longer fit as a QoS)

But if you feel this belongs in QoS this can certainly be added on top of the framework as Miguel suggested.

Thanks
Gal.

On Fri, Jun 19, 2015 at 12:39 AM, John Joyce (joycej) <joycej at cisco.com<mailto:joycej at cisco.com>> wrote:
Gal:
      I had seen the brute force blueprint and noticed how close the use case was.  Can you tell me the current status of the work?  Do you feel confident it can get into Liberty?  Ideally, we think this fits better with QoS.  Also I don’t think of it as providing FWaaS as we see that all VMs would be protected by this when enabled, but maybe that is just terminology.   We think these protections are critical so we are more concerned with having the ability to protect against these cases than the specific API or service it falls under.  Yes we would be interested in working together to get this pushed through.
                                                                                                                                John

From: Gal Sagie [mailto:gal.sagie at gmail.com<mailto:gal.sagie at gmail.com>]
Sent: Wednesday, June 17, 2015 12:45 PM
To: OpenStack Development Mailing List (not for usage questions)
Cc: lionel.zerbib at huawei.com<mailto:lionel.zerbib at huawei.com>; Derek Chamorro (dechamor); Eran Gampel
Subject: Re: [openstack-dev] [Neutron] [QOS] Request for Additional QoS capabilities

Hi John,

We were trying to push a very similar spec to enhance the security group API, we covered both DDoS case
and another use case for brute force prevention (We did a research to identify common protocols login behaviour
in order to identify brute force attacks using iptables) and even some UI work

You can view the specs and implementations here:
1) https://review.openstack.org/#/c/184243/
2) https://review.openstack.org/#/c/154535/
3) https://review.openstack.org/#/c/151247/
4) https://review.openstack.org/#/c/161207/

The spec didn't got approved as there is a strong opinion to keep the security group API compatible with Amazon.
I think this and your request fits much more into the FWaaS and if this is something you would like to work together on and push
i can help and align the above code to use FWaaS.

Feel free to contact me if you have any question

Thanks
Gal.



On Wed, Jun 17, 2015 at 6:58 PM, John Joyce (joycej) <joycej at cisco.com<mailto:joycej at cisco.com>> wrote:
Hello everyone:
      I would like to test the waters on some new functionality we think is needed to protect OpenStack deployments from some overload situations due to an excessive user or DDOS scenario.   We wrote this up in the style of an RFE.   Please let us know your thoughts and we can proceed with a formal RFE with more detail if there are no concerns raised.


*What is being requested*
This request is to extend the QOS APIs to include the ability to provide connection rate limiting
*Why is it being requested*
There are many scenarios where a VM may be intentionally malicious or become harmful to the network due to its rate of initializing TCP connections.   The reverse direction of a VM being attacked with an excessive amount of TCP connection requests either intentionally or due to overload is also problematic.
*Implementation Choices
   There might be a number of ways to implement this,  but one of the easiest would appear to be to extend the APIs being developed under:  https://review.openstack.org/#/c/187513/. An additional rule type “connections per-second” could be added.
The dataplane implementation itself may be realized with netfilter and conntrack.
*Alternatives
It would be possible to extend the security groups in a similar fashion,  but due to the addition of rate limiting, QoS seems a more nature fit.
*Who needs it*
Cloud operators have experienced this issue in real deployments in a number of cases.


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Best Regards ,

The G.

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Best Regards ,

The G.

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150629/d0356436/attachment.html>