[Openstack-operators] Small openstack

George Shuklin george.shuklin at gmail.com
Mon Jan 12 18:00:39 UTC 2015

Wow. Real problem. I check it - it allows one tenant to interrupt 
traffic on other. I was not able to intercept TCP traffic, victim lost 
connection and TCP  was struggling with retransmissions, but it was not 

But idea of 'no routers in software' is too appealing. I think I'll 
stick with this idea (and wait for fix). Plus, I think, OVS can provide 
some kind of excellent antispoofing mechanism (OVS can use nw_src/nw_dst 
for arp filtering) - if no fix will be available, I'll do some crude fix 
on neutron/nova with OVS (I don't really like etables).

On 01/09/2015 09:25 PM, Kris G. Lindgren wrote:
> Also, If you are running this configuration you should be aware of the
> following bug:
> https://bugs.launchpad.net/neutron/+bug/1274034
> And the corresponding fix: https://review.openstack.org/#/c/141130/
> Basically - Neutron security group rules do nothing to protect against arp
> spoofing/poisoning from vm's.  So its possible under a shared network
> configuration for a vm to arp for another vm's ip address and temporarily
> knock that vm offline.  The above commit - which is still a WIP adds
> ebtable rules to allow neutron to filter protocols other than IP (eg arp).
> ____________________________________________
> Kris Lindgren
> Senior Linux Systems Engineer
> GoDaddy, LLC.
> On 1/8/15, 9:50 AM, "Kris G. Lindgren" <klindgren at godaddy.com> wrote:
>> On 1/8/15, 4:34 AM, "Antonio Messina" <antonio.s.messina at gmail.com> wrote:
>>> On Thu, Jan 8, 2015 at 12:12 PM, gustavo panizzo (gfa) <gfa at zumbi.com.ar>
>>> wrote:
>>>> On 01/08/2015 07:01 PM, Antonio Messina wrote:
>>>>> On Thu, Jan 8, 2015 at 11:53 AM, gustavo panizzo (gfa)
>>>>> <gfa at zumbi.com.ar>
>>>>> wrote:
>>>>>> i may be wrong as i haven't tested that on juno, but in icehouse and
>>>>>> havana
>>>>>> i've setup external/provider networks one for each tenant
>>>>> Ah, ok, this is the point. What I would like to have instead is
>>>>> 1) one big external network with routable, private IPs, to be used by
>>>>> *any*
>>>>>      tenant (where any tenant can plug ports)
>>>> neutron net-create --shared should do the trick
>>> I guess the problem is that I was creating *external* _and_ *shared*
>>> network, but if I don't want to use floating IPs from that network I
>>> probably don't need the network to be external, right?
>> Correct.  We have our "external networks" (private ip) only configured as
>> shared so any vm from any tenant can create a port on it.  External means
>> it's meant to be plugged into a router/l3 agent.
>>>>> 3) small external networks dedicated to a tenant
>>>> neutron net-create --tenant-id XXXXX-XXXXXX
>>>> i've made that mix (also add tenantn networks) in my lab running
>>>> icehouse
>>>> (2014.1.2) worked fine, i've upgraded it to juno but i haven't test
>>>> that yet
>>> Thank you, I will test it further.
>>>> do you run more than one l3 agent? are you floating ips configured on
>>>> br-ex?
>>>> i think if you fix the policy.json on nova you should get it working
>>> I currently have only 1 l3-agent running, but it's suboptimal. I would
>>> like to have L3-HA for the floating IPs and DVR for the shared and
>>> tenant networks, but as far as I know it's not currently supported. I
>>> plan to deploy the production cloud after Kilo release, and crossing
>>> my fingers...
>>> Floating IPs are currently configured on br-ex.
>> You have floating ips working under this configuration?
>> If so then that means you are using the neutron router as the gateway for
>> all of your vm's and not the gateway provided by your network device.  As
>> soon as you created a router and attached the the shared network to it the
>> router got configured with the gateway address that you configured in your
>> network.  You may want to make sure that you don¹t have traffic flapping
>> between your real network gateway and your neutron gateway on your l3
>> router.
>> The reason why floating ip's won't work under this configuration (using a
>> real network device for the gateway) is the fact that the floating ip is
>> applied at the router as a nat so the traffic flows like: Client (
>> -> floating ip ( -> external port on router -> The router changes
>> the destination IP from the floating ip to the private IP of the vm
>> ( and send the traffic out to the vm via the routers
>> connection on the shared network.  The VM responds to the client traffic
>> which is not in its subnet directly to the default gateway (
>> The gateway (if it is a neutron gateway) will then undo the nat and send
>> the traffic back to the client.
>> Where this breaks down with a physical network gateway is when the traffic
> >from the VM is sent to the real network gateway - the gateway doesn't know
>> anything about the nat and does not re-map the vm ip to the floating ip.
>> IE The client ip never gets changed to the floating ip and sent back to
>> the client IP.  The Response would be seen as coming directly from
>>, instead of  So you will never establish a tcp
>> connection.
>> If you have this working somehow - please let me know.  When we attempted
>> to run this configuration we ran into the above issues with floating ips
>> when using a real gateway on a network device.
>> Note: this is also why you need to create your shared network with a
>> specific router to the metadata service.  Most likely on your real network
>> gateway you haven't added a route for to a specific ip.
>>> .a.
>>> -- 
>>> antonio.s.messina at gmail.com
>>> antonio.messina at uzh.ch                     +41 (0)44 635 42 22
>>> S3IT: Service and Support for Science IT   http://www.s3it.uzh.ch/
>>> University of Zurich
>>> Winterthurerstrasse 190
>>> CH-8057 Zurich Switzerland
>>> _______________________________________________
>>> OpenStack-operators mailing list
>>> OpenStack-operators at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

More information about the OpenStack-operators mailing list