[openstack-dev] [tricircle] Easy Way to Test Tricircle North-South L3 Networking
Shinobu Kinjo
shinobu.kj at gmail.com
Fri May 6 04:59:56 UTC 2016
Hi Team,
If a main problem is a number of network interfaces (I guess it's a
problem), here is more eco solution.
In side your linux box, you can add virtual network interfaces with
multiple subnets.
It's not related to this project. But it may be better to show the
followings for someone who are interested in this project.
0. Create VMs and add virtual interfaces no matter what you want.
1. Create configuration files.
* A number of files are up to your requirement.
* In this scenario, ens9 and ens10 are new.
[root at bottom01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
ONBOOT=yes
HWADDR=52:54:00:7D:1A:6B
TYPE=Ethernet
BOOTPROTO=dhcp
[root at bottom01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens10
DEVICE=ens10
ONBOOT=yes
TYPE=Ethernet
BOOTPROTO=none
IPADDR=1.0.1.1
NETMASK=255.255.255.0
[root at bottom01 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens11
DEVICE=ens11
ONBOOT=yes
TYPE=Ethernet
BOOTPROTO=none
IPADDR=1.0.2.1
NETMASK=255.255.255.0
2. Modify routing table
[root at localhost ~]# cat /etc/iproute2/rt_tables
[snip]
100 t1
101 t2
[snip]
3. Set default gateway for additional interfaces
[root at localhost ~]# cat /etc/rc.local
[snip]
# It's up to above settings
ip route add 1.0.1.0/24 dev ens10 src 1.0.1.1 table t1
ip route add table t1 default via 172.16.0.254 dev ens10
ip route add 1.0.2.0/24 dev ens11 src 1.0.2.1 table t2
ip route add table t2 default via 172.16.0.254 dev ens11
ip rule add table t1 from 1.0.1.1
ip rule add table t2 from 1.0.2.1
4. Set arp settings for additional interfaces
[root at localhost ~]# cat /etc/sysctl.d/arp.conf
net.ipv4.conf.all.filter = 1
net.ipv4.conf.default.filter = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_announce = 2
5. Check ips
[root at bottom01 ~]# ip a | egrep 'eth|ens' | grep inet
inet 1.0.1.1/24 brd 1.0.1.255 scope global ens10
inet 1.0.2.1/24 brd 1.0.2.255 scope global ens11
inet 172.16.0.79/24 brd 172.16.0.255 scope global dynamic eth0
# Repeat 1 ~ 4, if necessary
6. Make sure network connectivity between hosts on additional network address
* bottom01
[root at bottom01 ~]# ip a | egrep 'eth|ens' | grep inet
inet 1.0.1.1/24 brd 1.0.1.255 scope global ens10
inet 1.0.2.1/24 brd 1.0.2.255 scope global ens11
inet 172.16.0.79/24 brd 172.16.0.255 scope global dynamic eth0
* bottom02
[root at bottom02 ~]# ip a | egrep 'eth|ens' | grep inet
inet 1.0.1.2/24 brd 1.0.1.255 scope global ens9
inet 1.0.2.2/24 brd 1.0.2.255 scope global ens10
inet 172.16.0.78/24 brd 172.16.0.255 scope global dynamic eth0
[root at bottom02 ~]# ping 1.0.1.1
PING 1.0.1.1 (1.0.1.1) 56(84) bytes of data.
64 bytes from 1.0.1.1: icmp_seq=1 ttl=64 time=0.502 ms
64 bytes from 1.0.1.1: icmp_seq=2 ttl=64 time=0.267 ms
64 bytes from 1.0.1.1: icmp_seq=3 ttl=64 time=0.366 ms
64 bytes from 1.0.1.1: icmp_seq=4 ttl=64 time=0.390 ms
64 bytes from 1.0.1.1: icmp_seq=5 ttl=64 time=0.335 ms
[root at bottom01 ~]# tcpdump -i ens10
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens10, link-type EN10MB (Ethernet), capture size 65535 bytes
00:53:59.873858 IP 1.0.1.2 > bottom01: ICMP echo request, id 2909, seq
7, length 64
00:53:59.873886 IP bottom01 > 1.0.1.2: ICMP echo reply, id 2909, seq
7, length 64
00:54:00.835425 STP 802.1d, Config, Flags [none], bridge-id
8000.8c:dc:d4:32:63:ad.8005, length 43
00:54:00.873897 IP 1.0.1.2 > bottom01: ICMP echo request, id 2909, seq
8, length 64
00:54:00.873926 IP bottom01 > 1.0.1.2: ICMP echo reply, id 2909, seq
8, length 64
00:54:01.873828 IP 1.0.1.2 > bottom01: ICMP echo request, id 2909, seq
9, length 64
00:54:01.873855 IP bottom01 > 1.0.1.2: ICMP echo reply, id 2909, seq
9, length 64
00:54:02.835464 STP 802.1d, Config, Flags [none], bridge-id
8000.8c:dc:d4:32:63:ad.8005, length 43
00:54:02.873849 IP 1.0.1.2 > bottom01: ICMP echo request, id 2909, seq
10, length 64
00:54:02.873879 IP bottom01 > 1.0.1.2: ICMP echo reply, id 2909, seq
10, length 64
00:54:03.873843 IP 1.0.1.2 > bottom01: ICMP echo request, id 2909, seq
11, length 64
00:54:03.873871 IP bottom01 > 1.0.1.2: ICMP echo reply, id 2909, seq
11, length 64
00:54:04.835573 STP 802.1d, Config, Flags [none], bridge-id
8000.8c:dc:d4:32:63:ad.8005, length 43
00:54:04.873870 IP 1.0.1.2 > bottom01: ICMP echo request, id 2909, seq
12, length 64
00:54:04.873899 IP bottom01 > 1.0.1.2: ICMP echo reply, id 2909, seq
12, length 64
Cheers,
S
On Wed, May 4, 2016 at 11:17 AM, joehuang <joehuang at huawei.com> wrote:
> Hi, Shinobu,
>
> Correct, this is not the normal deployment scenario and the way of testbed setup.
>
> Cheers
>
> BR
> Chaoyi Huang ( joehuang )
>
> ________________________________________
> From: Shinobu Kinjo [shinobu.kj at gmail.com]
> Sent: 04 May 2016 9:38
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [tricircle] Easy Way to Test Tricircle North-South L3 Networking
>
> Hi Chaoyi,
>
> I didn't consider Ronghui's environment which I have no idea about.
>
>> That's why Zhiyuan proposed hacking way to do it.
>
> Considering such a limited situation, I understood this solution is
> for particular situation which is not usual for cascaded stack
> environment.
> Is it same of what you are implying in your message?
>
> I would like to avoid any misunderstanding between members as much as possible.
>
> Cheers,
> Shinobu
>
> On Wed, May 4, 2016 at 10:25 AM, joehuang <joehuang at huawei.com> wrote:
>> Hi, Shinobu,
>>
>> I think Zhiyuan's suggestion is mainly for Ronghui's environment, for his environment has very limited network infterfaces, it's difficult to experiment N-S feature. It would be recommended to use VMs for setting up Tricircle test bed with two bottom pods, so it's much more easier to manage networking plane for different purpose. But Ronghui's machine also have very limited vCPU and memory, so booting serveral VMs to establish the tricircle and two bottom pods test bed also not possible. That's why Zhiyuan proposed hacking way to do it.
>>
>> Best Regards
>> Chaoyi Huang ( joehuang )
>>
>> ________________________________________
>> From: Shinobu Kinjo [shinobu.kj at gmail.com]
>> Sent: 04 May 2016 6:58
>> To: OpenStack Development Mailing List (not for usage questions)
>> Subject: Re: [openstack-dev] [tricircle] Easy Way to Test Tricircle North-South L3 Networking
>>
>> Vega,
>>
>> On Tue, May 3, 2016 at 5:49 PM, Vega Cai <luckyvega.g at gmail.com> wrote:
>>> Hi all,
>>>
>>> Just would like to share a way to test Tricircle north-south L3 networking
>>> without requiring the third interface.
>>>
>>> In the Tricircle readme, it is said that you need to add an interface in
>>> your host to br-ext bridge. One interface to access the host, one interface
>>> for east-west networking and one interface for north-south networking, so
>>> all together three interfaces are required.
>>>
>>> What if your host only have two interfaces? Here is another deployment
>>> choice.
>>>
>>> First, change your external network type to flat type. If you are using the
>>> DevStack script provided by Tricircle, do the following changes in node2
>>> local.conf then run DevStack in node2.
>>>
>>> (1) change Q_ML2_PLUGIN_VLAN_TYPE_OPTIONS
>>> from (network_vlan_ranges=bridge:2001:3000,extern:3001:4000)
>>> to (network_vlan_ranges=bridge:2001:3000)
>>> (since we going to use flat external network, no need to configure VLAN
>>> range for extern)
>>> (2) add PHYSICAL_NETWORK=extern
>>> (3) keep OVS_BRIDGE_MAPPINGS=bridge:br-bridge,extern:br-ext
>>
>> Good point.
>>
>>>
>>> Second, specify flat type when creating external network.
>>>
>>> curl -X POST http://127.0.0.1:9696/v2.0/networks
>>> -H "Content-Type: application/json" \
>>> -H "X-Auth-Token: $token" \
>>> -d '{"network": {"name": "ext-net", "admin_state_up": true,
>>> "router:external": true, "provider:network_type": "flat",
>>> "provider:physical_network": "extern", "availability_zone_hints":
>>> ["Pod2"]}}'
>>
>> Understood.
>>
>>>
>>> Third, configure IP address of br-ext.
>>>
>>> sudo ifconfig br-ext 163.3.124.1 netmask 255.255.255.0
>>>
>>> Here 163.3.124.1 is your external network gateway IP, set net mask
>>> according to your CIDR.
>>>
>>> After the above steps, you can access your VM via floating IP in node2. Also
>>> your VM can ping the external gateway.
>>>
>>> Would like your VM to access the Internet?(Of course node2 should be able to
>>> access the Internet) Two more steps to follow:
>>> (1) Enable packet forward in node2
>>>
>>> sudo bash
>>> echo 1 >/proc/sys/net/ipv4/ip_forward
>>>
>>> (2) Configure SNAT in node2
>>>
>>> sudo iptables -t nat -I POSTROUTING -s 163.3.124.0/24 -o eth1 -j SNAT
>>> --to-source 10.250.201.21
>>>
>>> 163.3.124.0/24 is your external network CIDR, eth1 is the interface
>>> associated with your default route in node2 and 10.250.201.21 is the IP of
>>> eth1.
>>
>> I would like to avoid this kind of hackery way as much as possible.
>> I would like to see your further recommendation so that we easily and
>> quickly build cascaded stack system including top.
>>
>>>
>>> Hope this information helps.
>>>
>>> BR
>>> Zhiyuan
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>> --
>> Email:
>> shinobu at linux.com
>> GitHub:
>> shinobu-x
>> Blog:
>> Life with Distributed Computational System based on OpenSource
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Email:
> shinobu at linux.com
> shinobu at redhat.com
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Email:
shinobu at linux.com
shinobu at redhat.com
More information about the OpenStack-dev
mailing list