[Openstack] Help with ipv6 route configuration and problem to traverse virtual router.
Jorge Luiz Correa
correajl at gmail.com
Tue Aug 30 18:53:44 UTC 2016
Thank you Tomas and Brian!
Here they are (just replace my ipv6 prefix with 2001:DB8). But, I think the
problem is with firewall rules (see bellow).
root at dataexp-network:/# ip netns exec
qrouter-eb42f197-8969-4744-b226-49653ed2bf48
ifconfig
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
*qg-69fbbe1a-ee* Link encap:Ethernet HWaddr fa:16:3e:d5:c5:f8
inet addr:<IPV4 Address> Bcast:<IPV4 Address>
Mask:255.255.255.192
inet6 addr: *fe80::f816:3eff:fed5:c5f8*/64 Scope:Link
inet6 addr: fe80::f816:3eff:fe47:364/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:673930 errors:0 dropped:0 overruns:0 frame:0
TX packets:69034 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:45132120 (45.1 MB) TX bytes:7887310 (7.8 MB)
*qr-1ee33f03-23 *Link encap:Ethernet HWaddr fa:16:3e:47:03:64
inet6 addr: 2001:DB8:1400:c539::1/64 Scope:Global
inet6 addr: fe80::f816:3eff:fe47:364/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:99 errors:0 dropped:2 overruns:0 frame:0
TX packets:350 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:9416 (9.4 KB) TX bytes:40556 (40.5 KB)
qr-9f742219-78 Link encap:Ethernet HWaddr fa:16:3e:6a:5e:b3
inet addr:192.168.0.1 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::f816:3eff:fe6a:5eb3/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:7567 errors:0 dropped:2 overruns:0 frame:0
TX packets:2481 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:808083 (808.0 KB) TX bytes:243573 (243.5 KB)
root at dataexp-network:/# ip netns exec
qrouter-eb42f197-8969-4744-b226-49653ed2bf48
ip -6 route show
*2001:DB8:1400:c539::/64 dev qr-1ee33f03-23* proto kernel metric 256
pref medium
fe80::/64 dev qg-69fbbe1a-ee proto kernel metric 256 pref medium
fe80::/64 dev qr-9f742219-78 proto kernel metric 256 pref medium
fe80::/64 dev qr-1ee33f03-23 proto kernel metric 256 pref medium
*default via fe80::215:17ff:fea0:211d* dev qg-69fbbe1a-ee metric 1024
pref medium
fe80::215:17ff:fea0:211d is my firewall/router and this route was learned
via RA.
At this moment my firewall/router has one route to 2001:DB8:1400::1/52 via
fe80::f816:3eff:fed5:c5f8 (the path is firewall/router -> br-ex -> br-int
-> qg-69fbbe1a-ee). The packets go up to qg-69fbbe1a-ee.
I think these setting are ok!
---------------
Now, I found something with iptables. See the rules in qrouter namespace:
root at dataexp-network:~# ip netns exec
qrouter-eb42f197-8969-4744-b226-49653ed2bf48
ip6tables -L -n -v
Chain INPUT (policy ACCEPT 26 packets, 3263 bytes)
pkts bytes target prot opt in out source
destination
26 3263 neutron-l3-agent-INPUT all * *
::/0 ::/0
*Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)*
pkts bytes target prot opt in out source
destination
78 4368 neutron-filter-top all * *
::/0 ::/0
* 78 4368 neutron-l3-agent-FORWARD all * *
::/0 ::/0 *
Chain OUTPUT (policy ACCEPT 16 packets, 1367 bytes)
pkts bytes target prot opt in out source
destination
16 1367 neutron-filter-top all * *
::/0 ::/0
16 1367 neutron-l3-agent-OUTPUT all * *
::/0 ::/0
Chain neutron-filter-top (2 references)
pkts bytes target prot opt in out source
destination
94 5735 neutron-l3-agent-local all * *
::/0 ::/0
*Chain neutron-l3-agent-FORWARD (1 references)*
pkts bytes target prot opt in out source
destination
78 4368 *neutron-l3-agent-scope* all * *
::/0 ::/0
Chain neutron-l3-agent-INPUT (1 references)
pkts bytes target prot opt in out source
destination
Chain neutron-l3-agent-OUTPUT (1 references)
pkts bytes target prot opt in out source
destination
Chain neutron-l3-agent-local (1 references)
pkts bytes target prot opt in out source
destination
*Chain neutron-l3-agent-scope (1 references)*
pkts bytes target prot opt in out source
destination
78 4368 *DROP* all * qr-1ee33f03-23
::/0 ::/0 mark match ! 0x4000000/0xffff0000
Packets pass in chain FORWARD -> neutron-filter-top ->
neutron-l3-agent-local -> back to FORWARD -> neutron-l3-agent-FORWARD ->
neutron-l3-agent-scope -> DROP.
My security group is:
Direction Ether Type IP Protocol Port Range Remote IP Prefix
Remote Security Group Actions
Egress IPv6 Any Any ::/0
-
Ingress IPv6 Any Any -
default
Egress IPv4 Any Any 0.0.0.0/0
-
Ingress IPv4 Any Any -
default
Ingress IPv6 ICMP Any ::/0
-
Ingress IPv4 TCP 22 (SSH) 0.0.0.0/0
-
IPv4 rules is very similar but works. Ipv6 is blocking for some reason.
I've noted that if I keep a ping and restart the services on network node,
1 icmp echo reply pass, maybe during the time when iptables is being
reconfigured.
So, I don't know why is dropping when using ipv6.
> That external router is giving you the prefix via PD, right? I would
have thought it would have added a route for your /64 when it did that.
No, I've tried to do this form but I decided to implement a scenario like
described in documentation (the first attempt to use one external PD server
didn't work). So, I'm using Dibbler running on network node because the
default neutron driver is the dibbler driver.
This is not working as says documentation, because dibbler should run a
script to create one route (for the delegated prefix) and this route is not
created. But, as I said, the static route make the packets arrive on qg
interface. So, next step would be or try an external PD server or fix this
problem with dibbler script.
Tks.
- JLC
On Tue, Aug 30, 2016 at 11:44 AM, Tomáš Vondra <vondra at homeatcloud.cz>
wrote:
> Hi!
>
> To debug this, it would be nice to see the interfaces of configured on
> your Virtual Router and its routing table.
>
> Tomas
>
>
>
> *From:* Jorge Luiz Correa [mailto:correajl at gmail.com]
> *Sent:* Tuesday, August 30, 2016 3:55 PM
> *To:* openstack at lists.openstack.org
> *Subject:* [Openstack] Help with ipv6 route configuration and problem to
> traverse virtual router.
>
>
>
> Hi! I need some help to understand and configure my network node to
> provide network access using a dual stack configuration. I've a scenario
> with one controller, one network node and a lot of compute nodes. The
> version is Mitaka on Ubuntu 16.04 LTS, Kernel 4.4.0-36.
>
> The IPv4 is working fine. Instances can get IPv4 inside tenant networks, I
> can configure floating IPs, access external hosts etc.
>
> The IPv6 has some features working, but I still didn't got the traffic
> pass between internal and the external networks.
>
> I'm using prefix delegation with dibbler as described here:
>
> http://docs.openstack.org/mitaka/networking-guide/config-ipv6.html
>
> I can create IPv6 tenant subnets, they can get a prefix from dibbler and
> instances on this subnets can configure IPv6 normally.
>
> I've a default security group with rules passing any IPv4 and IPv6 traffic
> and any ICMP.
>
> The problem is that the packages from and to instances don't pass through
> virtual router. The virtual router has one external interface named qg-
> (connected to br-int -> br-ex) and one internal interface named qr-
> connected to tenant network (br-int -> int-br-vlan). When testing
> connectivity I can see packages (with tcpdump) on my external
> router/firewall and on qg- interface. For example, when I try to ping my
> external router/firewall from an instance, echo requests pass to the
> external network (through the virtual router) but echo reply die on virtual
> router (last seen on qg-).
>
> ## echo request:
>
> Instance A
> |
> |
> v
> br-int
> |
> |
> v
> qr- interface
> VIRTUAL ROUTER
> qg- interface
> |
> |
> v
> br-int
> |
> |
> v
> br-ex
> |
> |
> v
> Router/Firewall (I can see here with tcpdump)
>
>
> ## echo reply:
>
> Instance A
> x
> x
> x
> qr- interface (I CAN'T SEE HERE, LOST)
> VIRTUAL ROUTER
> qg- interface (I can see here with tcpdump)
> ^
> |
> |
> br-int (ovs bridge, can't do tcpdump, but ok)
> ^
> |
> |
> br-ex (I can see here with tcpdump)
> ^
> |
> |
> Router/Firewall
>
> Question 1) Where can I start to debug this problem?
>
> I'm thinking that can be something with ipv6 packet forwarding
> (configurable with sysctl). Using 'ip6tables -v' I can't see droppings.
>
> Chain neutron-openvswi-sg-fallback (0 references)
> pkts bytes target prot opt in out source
> destination
> 0 0 DROP all * * ::/0
> ::/0 /* Default drop rule for unmatched traffic. */
>
> Another thing I would like to understand is about how I should configure
> my router/firewall to send IPv6 packets to Openstack network node. For
> example, if I have the network 2001:DB8::/52 to use on Openstack. Each
> project will get a 2001:DB8::/64 range from prefix delegation. When one
> project get its prefix, the virtual router knows how to send traffic to
> external world because my router/firewall sends RA. But, on my
> router/firewall I need to configure a route to 2001:DB8::/52. To do this, I
> need to inform one next-hop. I'm using de LLA (fe80::...) of br-ex as
> next-hop. So, all traffic destinated to any network inside 2001:DB8::/52
> will be send to br-ex (that is on network node). This configuration seems
> to work because packets arrive on virtual router as described above.
>
> Question 2) Is this the right way?
>
> Thanks for any help!
>
> - JLC
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20160830/a23b8541/attachment.html>
More information about the Openstack
mailing list