[Openstack-operators] Multiple floating IPs mapped to multiple vNICs (multi-homing)

Paul Browne pfb29 at cam.ac.uk
Thu Dec 1 14:18:35 UTC 2016


Hello Saverio,

Many thanks for the reply, I'll answer your queries below;

On 01/12/16 12:49, Saverio Proto wrote:
> Hello,
>
> while the problem is in place, you should share the output of
>
> ip rule show
> ip route show table 1
>
> It could be just a problem in your ruleset

Of course, these are those outputs ;

root at test1:~# ip rule show
0:      from all lookup local
32764:  from all to 10.0.16.11 lookup rt2
32765:  from 10.0.16.11 lookup rt2
32766:  from all lookup main
32767:  from all lookup default

root at test1:~# ip route show table 1
default via 10.0.16.1 dev eth1
10.0.16.0/24 dev eth1  scope link  src 10.0.16.11


>
> and, which one is your webserver ? can you tcpdump to make sure reply
> packets get out on the NIC with src address 10.0.16.11 ?
>
> Saverio

The instance has its two vNICs with source addresses 10.0.0.11 & 
10.0.16.11, and the web-server is listening on both.

The HTTP packets do seem to be getting out from 10.0.16.11 as source, 
but are stopped elsewhere upstream.

I've attached two pcaps showing HTTP reply packets, one from 10.0.0.11 
(first vNIC; HTTP request and reply works to a remote client) and one 
from 10.0.16.11 (second vNIC; HTTP request is sent, reply not received 
by remote client). In the latter case, the server starts to make 
retransmissions to the remote client.

Kind regards,
Paul Browne



>
>
> 2016-12-01 13:08 GMT+01:00 Paul Browne <pfb29 at cam.ac.uk>:
>> Hello Operators,
>>
>> For reasons not yet amenable to persuasion otherwise, a customer of our
>> ML2+OVS classic implemented OpenStack would like to map two floating IPs
>> pulled from two separate external network floating IP pools, to two
>> different vNICs on his instances.
>>
>> The floating IP pools correspond to one pool routable from the external
>> Internet and another, RFC1918 pool routable from internal University
>> networks.
>>
>> The tenant private networks are arranged as two RFC1918 VXLANs, each with a
>> router to one of the two external networks.
>>
>> 10.0.0.0/24 -> route to -> 128.232.226.0/23
>>
>> 10.0.16.0/24 -> route to -> 172.24.46.0/23
>>
>>
>> Mapping two floating IPs to instances isn't possible in Horizon, but is
>> possible from command-line. This doesn't immediately work, however, as the
>> return traffic from the instance needs to be sent back through the correct
>> router gateway interface and not the instance default gateway.
>>
>> I'd initially thought this would be possible by placing a second routing
>> table on the instances to handle the return traffic;
>>
>> debian at test1:/etc/iproute2$ less rt_tables
>> #
>> # reserved values
>> #
>> 255     local
>> 254     main
>> 253     default
>> 0       unspec
>> #
>> # local
>> #
>> #1      inr.ruhep
>> 1 rt2
>>
>> debian at test1:/etc/network$ less interfaces
>> # The loopback network interface
>> auto lo
>> iface lo inet loopback
>>
>> # The first vNIC, eth0
>> auto eth0
>> iface eth0 inet dhcp
>>
>> # The second vNIC, eth1
>> auto eth1
>> iface eth1 inet static
>>          address 10.0.16.11
>>          netmask 255.255.255.0
>>          post-up ip route add 10.0.16.0/24 dev eth1 src 10.0.16.11 table rt2
>>          post-up ip route add default via 10.0.16.1 dev eth1 table rt2
>>          post-up ip rule add from 10.0.16.11/32 table rt2
>>          post-up ip rule add to 10.0.16.11/32 table rt2
>>
>> And this works well for SSH and ICMP, but curiously not for HTTP traffic.
>>
>>
>> Requests to a web-server listening on all vNICs are sent but replies not
>> received when the requests are sent to the second mapped floating IP (HTTP
>> requests and replies work as expected when sent to the first mapped floating
>> IP). The requests are logged in both cases however, so traffic is making it
>> to the instance in both cases.
>>
>> I'd say this is clearly an unusual (and possibly un-natural) arrangement,
>> but I was wondering whether anyone else on Operators had come across a
>> similar situation in trying to map floating IPs from two different external
>> networks to an instance?
>>
>> Kind regards,
>>
>> Paul Browne
>>
>> --
>> *******************
>> Paul Browne
>> Research Computing Platforms
>> University Information Services
>> Roger Needham Building
>> JJ Thompson Avenue
>> University of Cambridge
>> Cambridge
>> United Kingdom
>> E-Mail: pfb29 at cam.ac.uk
>> Tel: 0044-1223-46548
>> *******************
>>
>>
>> _______________________________________________
>> OpenStack-operators mailing list
>> OpenStack-operators at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>

-- 
*******************
Paul Browne
Research Computing Platforms
University Information Services
Roger Needham Building
JJ Thompson Avenue
University of Cambridge
Cambridge
United Kingdom
E-Mail: pfb29 at cam.ac.uk
Tel: 0044-1223-46548
*******************

-------------- next part --------------
A non-text attachment was scrubbed...
Name: eth0.pcap
Type: application/vnd.tcpdump.pcap
Size: 5189 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20161201/a220ce47/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: eth1.pcap
Type: application/vnd.tcpdump.pcap
Size: 11927 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20161201/a220ce47/attachment-0001.bin>


More information about the OpenStack-operators mailing list