[neutron] DevStack with IPv6
Hi folks, I'm having troubles to ping6 a VM running over DevStack from its hypervisor. Could you please help me troubleshooting it?
I deployed DevStack with NEUTRON_CREATE_INITIAL_NETWORKS=False, and manually created the networks, subnets and router. Following is my router:
$ openstack router show router1 -c external_gateway_info -c interfaces_info +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
| +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | external_gateway_info | {"network_id": "b87048ed-1be9-4f31-8d7e-fe74921aeec4", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "28a00bc3-b30b-456f-b26a-44b50d37183f", "ip_address": "10.2.0.199"}, {"subnet_id": "a9729beb-b297-4fec-8ec3-7703f7f6f4bc", "ip_address": "fd12:67:1::3c"}]} | | interfaces_info | [{"subnet_id": "081e8508-4ceb-4aaf-bf91-36a1e22a768c", "ip_address": "fd12:67:1:1::1", "port_id": "75391abd-8ac8-41f8-acf8-3dfaf2a6b08f"}]
| +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
I'm trying to ping6 the following VM:
$ openstack server list +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | 938854d0-80e9-45b2-bc29-8fe7651ffa93 | manila1 | ACTIVE | private1=fd12:67:1:1:f816:3eff:fe0e:17c3 | manila | manila | +--------------------------------------+---------+--------+------------------------------------------+--------+--------+
I intend to reach it via br-ex interface of the hypervisor:
$ ip a show dev br-ex 9: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 0e:82:a1:ba:77:4c brd ff:ff:ff:ff:ff:ff inet6 fd12:67:1::1/64 scope global valid_lft forever preferred_lft forever inet6 fe80::c82:a1ff:feba:774c/64 scope link valid_lft forever preferred_lft forever
The hypervisor has the following routes:
$ ip -6 route fd12:67:1:1::/64 via fd12:67:1::3c dev br-ex metric 1024 pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium fe80::/64 dev br-ex proto kernel metric 256 pref medium fe80::/64 dev br-int proto kernel metric 256 pref medium fe80::/64 dev tapa5cf4799-9f proto kernel metric 256 pref medium
And within the VM has the following routes:
root@ubuntu:~# ip -6 route root@ubuntu:~# ip -6 route fd12:67:1::/64 via fd12:67:1:1::1 dev ens3 metric 1024 pref medium fd12:67:1:1::/64 dev ens3 proto kernel metric 256 expires 86360sec pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium default via fe80::f816:3eff:feb3:bd56 dev ens3 proto ra metric 1024 expires 260sec hoplimit 64 pref medium
Though the ping6 from VM to hypervisor doesn't work: root@ubuntu:~# ping6 fd12:67:1::1 -c4 PING fd12:67:1::1 (fd12:67:1::1): 56 data bytes --- fd12:67:1::1 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss
I'm able to tcpdump inside the router1 netns and see that request packet is passing there, but can't see any reply packets:
$ sudo ip netns exec qrouter-5172472c-bbe7-4907-832a-e2239c8badb4 tcpdump -l -i any icmp6 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes 21:29:29.351358 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 0, length 64 21:29:30.033316 IP6 fe80::f816:3eff:feb3:bd56 > fe80::f816:3eff:fe0e:17c3: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe0e:17c3, length 32 21:29:30.035807 IP6 fe80::f816:3eff:fe0e:17c3 > fe80::f816:3eff:feb3:bd56: ICMP6, neighbor advertisement, tgt is fe80::f816:3eff:fe0e:17c3, length 24 21:29:30.353646 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 1, length 64 21:29:31.355410 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 2, length 64 21:29:32.357239 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 3, length 64
The same happens from hypervisor to VM. I only acan see the request packets, but no reply packets.
Thanks in advance, Lucio Seki
Security group rules?
Donny Davis c: 805 814 6800
On Thu, Sep 12, 2019, 5:53 PM Lucio Seki lucioseki@gmail.com wrote:
Hi folks, I'm having troubles to ping6 a VM running over DevStack from its hypervisor. Could you please help me troubleshooting it?
I deployed DevStack with NEUTRON_CREATE_INITIAL_NETWORKS=False, and manually created the networks, subnets and router. Following is my router:
$ openstack router show router1 -c external_gateway_info -c interfaces_info
+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | external_gateway_info | {"network_id": "b87048ed-1be9-4f31-8d7e-fe74921aeec4", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "28a00bc3-b30b-456f-b26a-44b50d37183f", "ip_address": "10.2.0.199"}, {"subnet_id": "a9729beb-b297-4fec-8ec3-7703f7f6f4bc", "ip_address": "fd12:67:1::3c"}]} | | interfaces_info | [{"subnet_id": "081e8508-4ceb-4aaf-bf91-36a1e22a768c", "ip_address": "fd12:67:1:1::1", "port_id": "75391abd-8ac8-41f8-acf8-3dfaf2a6b08f"}]
|+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
I'm trying to ping6 the following VM:
$ openstack server list
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | 938854d0-80e9-45b2-bc29-8fe7651ffa93 | manila1 | ACTIVE | private1=fd12:67:1:1:f816:3eff:fe0e:17c3 | manila | manila |
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+
I intend to reach it via br-ex interface of the hypervisor:
$ ip a show dev br-ex 9: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 0e:82:a1:ba:77:4c brd ff:ff:ff:ff:ff:ff inet6 fd12:67:1::1/64 scope global valid_lft forever preferred_lft forever inet6 fe80::c82:a1ff:feba:774c/64 scope link valid_lft forever preferred_lft forever
The hypervisor has the following routes:
$ ip -6 route fd12:67:1:1::/64 via fd12:67:1::3c dev br-ex metric 1024 pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium fe80::/64 dev br-ex proto kernel metric 256 pref medium fe80::/64 dev br-int proto kernel metric 256 pref medium fe80::/64 dev tapa5cf4799-9f proto kernel metric 256 pref medium
And within the VM has the following routes:
root@ubuntu:~# ip -6 route root@ubuntu:~# ip -6 route fd12:67:1::/64 via fd12:67:1:1::1 dev ens3 metric 1024 pref medium fd12:67:1:1::/64 dev ens3 proto kernel metric 256 expires 86360sec pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium default via fe80::f816:3eff:feb3:bd56 dev ens3 proto ra metric 1024 expires 260sec hoplimit 64 pref medium
Though the ping6 from VM to hypervisor doesn't work: root@ubuntu:~# ping6 fd12:67:1::1 -c4 PING fd12:67:1::1 (fd12:67:1::1): 56 data bytes --- fd12:67:1::1 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss
I'm able to tcpdump inside the router1 netns and see that request packet is passing there, but can't see any reply packets:
$ sudo ip netns exec qrouter-5172472c-bbe7-4907-832a-e2239c8badb4 tcpdump -l -i any icmp6 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes 21:29:29.351358 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 0, length 64 21:29:30.033316 IP6 fe80::f816:3eff:feb3:bd56 > fe80::f816:3eff:fe0e:17c3: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe0e:17c3, length 32 21:29:30.035807 IP6 fe80::f816:3eff:fe0e:17c3 > fe80::f816:3eff:feb3:bd56: ICMP6, neighbor advertisement, tgt is fe80::f816:3eff:fe0e:17c3, length 24 21:29:30.353646 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 1, length 64 21:29:31.355410 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 2, length 64 21:29:32.357239 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 3, length 64
The same happens from hypervisor to VM. I only acan see the request packets, but no reply packets.
Thanks in advance, Lucio Seki
Hi Donny, following are the rules:
$ openstack security group list --project admin +--------------------------------------+---------+------------------------+----------------------------------+------+ | ID | Name | Description | Project | Tags | +--------------------------------------+---------+------------------------+----------------------------------+------+ | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | default | Default security group | 68e3942285a24fb5bd1aed30e166aaee | [] | +--------------------------------------+---------+------------------------+----------------------------------+------+
$ openstack security group rule list d0136b0e-ee51-461c-afa0-c5adb88dd0dd +--------------------------------------+-------------+----------+------------+--------------------------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group | +--------------------------------------+-------------+----------+------------+--------------------------------------+ | 38394345-3e44-4284-a519-cdd8af020f30 | tcp | ::/0 | 22:22 | None | | 40881f76-c87f-4685-b3af-c3497dd44837 | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 56d4ae52-195e-48df-871e-dc70b899b7ba | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 759edd06-b698-45ca-94cd-44e0cc2cc848 | ipv6-icmp | None | | None | | 762effae-b8e5-42ac-ba99-e85a7bc42455 | tcp | ::/0 | 22:22 | None | | 81f3588d-4159-4af2-ad50-ff6b76add9cf | ipv6-icmp | None | | None | +--------------------------------------+-------------+----------+------------+--------------------------------------+
$ openstack security group rule show 759edd06-b698-45ca-94cd-44e0cc2cc848 +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
| +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:41Z
| | description |
| | direction | egress
| | ether_type | IPv6
| | id | 759edd06-b698-45ca-94cd-44e0cc2cc848
| | location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None
| | port_range_max | None
| | port_range_min | None
| | project_id | 68e3942285a24fb5bd1aed30e166aaee
| | protocol | ipv6-icmp
| | remote_group_id | None
| | remote_ip_prefix | None
| | revision_number | 0
| | security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
| | tags | []
| | updated_at | 2019-09-03T16:51:41Z
| +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
$ openstack security group rule show 81f3588d-4159-4af2-ad50-ff6b76add9cf +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
| +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:30Z
| | description |
| | direction | ingress
| | ether_type | IPv6
| | id | 81f3588d-4159-4af2-ad50-ff6b76add9cf
| | location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None
| | port_range_max | None
| | port_range_min | None
| | project_id | 68e3942285a24fb5bd1aed30e166aaee
| | protocol | ipv6-icmp
| | remote_group_id | None
| | remote_ip_prefix | None
| | revision_number | 0
| | security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
| | tags | []
| | updated_at | 2019-09-03T16:51:30Z
| +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 10:16 AM Donny Davis donny@fortnebula.com wrote:
Security group rules?
Donny Davis c: 805 814 6800
On Thu, Sep 12, 2019, 5:53 PM Lucio Seki lucioseki@gmail.com wrote:
Hi folks, I'm having troubles to ping6 a VM running over DevStack from its hypervisor. Could you please help me troubleshooting it?
I deployed DevStack with NEUTRON_CREATE_INITIAL_NETWORKS=False, and manually created the networks, subnets and router. Following is my router:
$ openstack router show router1 -c external_gateway_info -c interfaces_info
+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | external_gateway_info | {"network_id": "b87048ed-1be9-4f31-8d7e-fe74921aeec4", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "28a00bc3-b30b-456f-b26a-44b50d37183f", "ip_address": "10.2.0.199"}, {"subnet_id": "a9729beb-b297-4fec-8ec3-7703f7f6f4bc", "ip_address": "fd12:67:1::3c"}]} | | interfaces_info | [{"subnet_id": "081e8508-4ceb-4aaf-bf91-36a1e22a768c", "ip_address": "fd12:67:1:1::1", "port_id": "75391abd-8ac8-41f8-acf8-3dfaf2a6b08f"}]
|+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
I'm trying to ping6 the following VM:
$ openstack server list
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | 938854d0-80e9-45b2-bc29-8fe7651ffa93 | manila1 | ACTIVE | private1=fd12:67:1:1:f816:3eff:fe0e:17c3 | manila | manila |
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+
I intend to reach it via br-ex interface of the hypervisor:
$ ip a show dev br-ex 9: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 0e:82:a1:ba:77:4c brd ff:ff:ff:ff:ff:ff inet6 fd12:67:1::1/64 scope global valid_lft forever preferred_lft forever inet6 fe80::c82:a1ff:feba:774c/64 scope link valid_lft forever preferred_lft forever
The hypervisor has the following routes:
$ ip -6 route fd12:67:1:1::/64 via fd12:67:1::3c dev br-ex metric 1024 pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium fe80::/64 dev br-ex proto kernel metric 256 pref medium fe80::/64 dev br-int proto kernel metric 256 pref medium fe80::/64 dev tapa5cf4799-9f proto kernel metric 256 pref medium
And within the VM has the following routes:
root@ubuntu:~# ip -6 route root@ubuntu:~# ip -6 route fd12:67:1::/64 via fd12:67:1:1::1 dev ens3 metric 1024 pref medium fd12:67:1:1::/64 dev ens3 proto kernel metric 256 expires 86360sec pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium default via fe80::f816:3eff:feb3:bd56 dev ens3 proto ra metric 1024 expires 260sec hoplimit 64 pref medium
Though the ping6 from VM to hypervisor doesn't work: root@ubuntu:~# ping6 fd12:67:1::1 -c4 PING fd12:67:1::1 (fd12:67:1::1): 56 data bytes --- fd12:67:1::1 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss
I'm able to tcpdump inside the router1 netns and see that request packet is passing there, but can't see any reply packets:
$ sudo ip netns exec qrouter-5172472c-bbe7-4907-832a-e2239c8badb4 tcpdump -l -i any icmp6 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes 21:29:29.351358 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 0, length 64 21:29:30.033316 IP6 fe80::f816:3eff:feb3:bd56 > fe80::f816:3eff:fe0e:17c3: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe0e:17c3, length 32 21:29:30.035807 IP6 fe80::f816:3eff:fe0e:17c3 > fe80::f816:3eff:feb3:bd56: ICMP6, neighbor advertisement, tgt is fe80::f816:3eff:fe0e:17c3, length 24 21:29:30.353646 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 1, length 64 21:29:31.355410 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 2, length 64 21:29:32.357239 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 3, length 64
The same happens from hypervisor to VM. I only acan see the request packets, but no reply packets.
Thanks in advance, Lucio Seki
Well here is the output from my rule list that is in prod right now with ipv6 +--------------------------------------+-------------+-----------+------------+-----------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group | +--------------------------------------+-------------+-----------+------------+-----------------------+ | 9ab00b6f-2bc2-4554-818d-eff6e0570943 | None | 0.0.0.0/0 | | None | | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa | icmp | ::/0 | | None | | e7fd4840-5fbd-4709-b918-f80eac5cb6da | None | ::/0 | | None | | e9968d53-7efe-4a9e-ad42-1092ffaf52e7 | None | None | | None | | ec1ea961-9025-4229-92cf-618026a1851b | None | None | | None | +--------------------------------------+-------------+-----------+------------+-----------------------+
+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
| +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-07-30T00:50:25Z
| | description |
| | direction | ingress
| | ether_type | IPv6
| | id | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa
| | location | Munch({'cloud': '', 'region_name': 'regionOne', 'zone': None, 'project': Munch({'id': 'e8fd161dc34c421a979a9e6421f823e9', 'name': 'openstackzuul', 'domain_id': None, 'domain_name': 'Default'})}) | | name | None
| | port_range_max | None
| | port_range_min | None
| | project_id | e8fd161dc34c421a979a9e6421f823e9
| | protocol | icmp
| | remote_group_id | None
| | remote_ip_prefix | ::/0
| | revision_number | 0
| | security_group_id | bcedc0e0-e2e8-41fc-aeaa-afd2e10c7ab6
| | tags | []
| | updated_at | 2019-07-30T00:50:25Z
| +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 9:24 AM Lucio Seki lucioseki@gmail.com wrote:
Hi Donny, following are the rules:
$ openstack security group list --project admin
+--------------------------------------+---------+------------------------+----------------------------------+------+ | ID | Name | Description | Project | Tags |
+--------------------------------------+---------+------------------------+----------------------------------+------+ | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | default | Default security group | 68e3942285a24fb5bd1aed30e166aaee | [] |
+--------------------------------------+---------+------------------------+----------------------------------+------+
$ openstack security group rule list d0136b0e-ee51-461c-afa0-c5adb88dd0dd
+--------------------------------------+-------------+----------+------------+--------------------------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group |
+--------------------------------------+-------------+----------+------------+--------------------------------------+ | 38394345-3e44-4284-a519-cdd8af020f30 | tcp | ::/0 | 22:22 | None | | 40881f76-c87f-4685-b3af-c3497dd44837 | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 56d4ae52-195e-48df-871e-dc70b899b7ba | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 759edd06-b698-45ca-94cd-44e0cc2cc848 | ipv6-icmp | None | | None | | 762effae-b8e5-42ac-ba99-e85a7bc42455 | tcp | ::/0 | 22:22 | None | | 81f3588d-4159-4af2-ad50-ff6b76add9cf | ipv6-icmp | None | | None |
+--------------------------------------+-------------+----------+------------+--------------------------------------+
$ openstack security group rule show 759edd06-b698-45ca-94cd-44e0cc2cc848
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:41Z
|| description |
|| direction | egress
|| ether_type | IPv6
|| id | 759edd06-b698-45ca-94cd-44e0cc2cc848
|| location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | 68e3942285a24fb5bd1aed30e166aaee
|| protocol | ipv6-icmp
|| remote_group_id | None
|| remote_ip_prefix | None
|| revision_number | 0
|| security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
|| tags | []
|| updated_at | 2019-09-03T16:51:41Z
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
$ openstack security group rule show 81f3588d-4159-4af2-ad50-ff6b76add9cf
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:30Z
|| description |
|| direction | ingress
|| ether_type | IPv6
|| id | 81f3588d-4159-4af2-ad50-ff6b76add9cf
|| location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | 68e3942285a24fb5bd1aed30e166aaee
|| protocol | ipv6-icmp
|| remote_group_id | None
|| remote_ip_prefix | None
|| revision_number | 0
|| security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
|| tags | []
|| updated_at | 2019-09-03T16:51:30Z
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 10:16 AM Donny Davis donny@fortnebula.com wrote:
Security group rules?
Donny Davis c: 805 814 6800
On Thu, Sep 12, 2019, 5:53 PM Lucio Seki lucioseki@gmail.com wrote:
Hi folks, I'm having troubles to ping6 a VM running over DevStack from its hypervisor. Could you please help me troubleshooting it?
I deployed DevStack with NEUTRON_CREATE_INITIAL_NETWORKS=False, and manually created the networks, subnets and router. Following is my router:
$ openstack router show router1 -c external_gateway_info -c interfaces_info
+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | external_gateway_info | {"network_id": "b87048ed-1be9-4f31-8d7e-fe74921aeec4", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "28a00bc3-b30b-456f-b26a-44b50d37183f", "ip_address": "10.2.0.199"}, {"subnet_id": "a9729beb-b297-4fec-8ec3-7703f7f6f4bc", "ip_address": "fd12:67:1::3c"}]} | | interfaces_info | [{"subnet_id": "081e8508-4ceb-4aaf-bf91-36a1e22a768c", "ip_address": "fd12:67:1:1::1", "port_id": "75391abd-8ac8-41f8-acf8-3dfaf2a6b08f"}]
|+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
I'm trying to ping6 the following VM:
$ openstack server list
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | 938854d0-80e9-45b2-bc29-8fe7651ffa93 | manila1 | ACTIVE | private1=fd12:67:1:1:f816:3eff:fe0e:17c3 | manila | manila |
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+
I intend to reach it via br-ex interface of the hypervisor:
$ ip a show dev br-ex 9: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 0e:82:a1:ba:77:4c brd ff:ff:ff:ff:ff:ff inet6 fd12:67:1::1/64 scope global valid_lft forever preferred_lft forever inet6 fe80::c82:a1ff:feba:774c/64 scope link valid_lft forever preferred_lft forever
The hypervisor has the following routes:
$ ip -6 route fd12:67:1:1::/64 via fd12:67:1::3c dev br-ex metric 1024 pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium fe80::/64 dev br-ex proto kernel metric 256 pref medium fe80::/64 dev br-int proto kernel metric 256 pref medium fe80::/64 dev tapa5cf4799-9f proto kernel metric 256 pref medium
And within the VM has the following routes:
root@ubuntu:~# ip -6 route root@ubuntu:~# ip -6 route fd12:67:1::/64 via fd12:67:1:1::1 dev ens3 metric 1024 pref medium fd12:67:1:1::/64 dev ens3 proto kernel metric 256 expires 86360sec pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium default via fe80::f816:3eff:feb3:bd56 dev ens3 proto ra metric 1024 expires 260sec hoplimit 64 pref medium
Though the ping6 from VM to hypervisor doesn't work: root@ubuntu:~# ping6 fd12:67:1::1 -c4 PING fd12:67:1::1 (fd12:67:1::1): 56 data bytes --- fd12:67:1::1 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss
I'm able to tcpdump inside the router1 netns and see that request packet is passing there, but can't see any reply packets:
$ sudo ip netns exec qrouter-5172472c-bbe7-4907-832a-e2239c8badb4 tcpdump -l -i any icmp6 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes 21:29:29.351358 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 0, length 64 21:29:30.033316 IP6 fe80::f816:3eff:feb3:bd56 > fe80::f816:3eff:fe0e:17c3: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe0e:17c3, length 32 21:29:30.035807 IP6 fe80::f816:3eff:fe0e:17c3 > fe80::f816:3eff:feb3:bd56: ICMP6, neighbor advertisement, tgt is fe80::f816:3eff:fe0e:17c3, length 24 21:29:30.353646 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 1, length 64 21:29:31.355410 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 2, length 64 21:29:32.357239 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 3, length 64
The same happens from hypervisor to VM. I only acan see the request packets, but no reply packets.
Thanks in advance, Lucio Seki
Also I have no v6 address on my br-ex
On Fri, Sep 13, 2019 at 1:22 PM Donny Davis donny@fortnebula.com wrote:
Well here is the output from my rule list that is in prod right now with ipv6
+--------------------------------------+-------------+-----------+------------+-----------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group |
+--------------------------------------+-------------+-----------+------------+-----------------------+ | 9ab00b6f-2bc2-4554-818d-eff6e0570943 | None | 0.0.0.0/0 | | None | | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa | icmp | ::/0 | | None | | e7fd4840-5fbd-4709-b918-f80eac5cb6da | None | ::/0 | | None | | e9968d53-7efe-4a9e-ad42-1092ffaf52e7 | None | None | | None | | ec1ea961-9025-4229-92cf-618026a1851b | None | None | | None |
+--------------------------------------+-------------+-----------+------------+-----------------------+
+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-07-30T00:50:25Z
|| description |
|| direction | ingress
|| ether_type | IPv6
|| id | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa
|| location | Munch({'cloud': '', 'region_name': 'regionOne', 'zone': None, 'project': Munch({'id': 'e8fd161dc34c421a979a9e6421f823e9', 'name': 'openstackzuul', 'domain_id': None, 'domain_name': 'Default'})}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | e8fd161dc34c421a979a9e6421f823e9
|| protocol | icmp
|| remote_group_id | None
|| remote_ip_prefix | ::/0
|| revision_number | 0
|| security_group_id | bcedc0e0-e2e8-41fc-aeaa-afd2e10c7ab6
|| tags | []
|| updated_at | 2019-07-30T00:50:25Z
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 9:24 AM Lucio Seki lucioseki@gmail.com wrote:
Hi Donny, following are the rules:
$ openstack security group list --project admin
+--------------------------------------+---------+------------------------+----------------------------------+------+ | ID | Name | Description | Project | Tags |
+--------------------------------------+---------+------------------------+----------------------------------+------+ | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | default | Default security group | 68e3942285a24fb5bd1aed30e166aaee | [] |
+--------------------------------------+---------+------------------------+----------------------------------+------+
$ openstack security group rule list d0136b0e-ee51-461c-afa0-c5adb88dd0dd
+--------------------------------------+-------------+----------+------------+--------------------------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group |
+--------------------------------------+-------------+----------+------------+--------------------------------------+ | 38394345-3e44-4284-a519-cdd8af020f30 | tcp | ::/0 | 22:22 | None | | 40881f76-c87f-4685-b3af-c3497dd44837 | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 56d4ae52-195e-48df-871e-dc70b899b7ba | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 759edd06-b698-45ca-94cd-44e0cc2cc848 | ipv6-icmp | None | | None | | 762effae-b8e5-42ac-ba99-e85a7bc42455 | tcp | ::/0 | 22:22 | None | | 81f3588d-4159-4af2-ad50-ff6b76add9cf | ipv6-icmp | None | | None |
+--------------------------------------+-------------+----------+------------+--------------------------------------+
$ openstack security group rule show 759edd06-b698-45ca-94cd-44e0cc2cc848
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:41Z
|| description |
|| direction | egress
|| ether_type | IPv6
|| id | 759edd06-b698-45ca-94cd-44e0cc2cc848
|| location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | 68e3942285a24fb5bd1aed30e166aaee
|| protocol | ipv6-icmp
|| remote_group_id | None
|| remote_ip_prefix | None
|| revision_number | 0
|| security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
|| tags | []
|| updated_at | 2019-09-03T16:51:41Z
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
$ openstack security group rule show 81f3588d-4159-4af2-ad50-ff6b76add9cf
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:30Z
|| description |
|| direction | ingress
|| ether_type | IPv6
|| id | 81f3588d-4159-4af2-ad50-ff6b76add9cf
|| location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | 68e3942285a24fb5bd1aed30e166aaee
|| protocol | ipv6-icmp
|| remote_group_id | None
|| remote_ip_prefix | None
|| revision_number | 0
|| security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
|| tags | []
|| updated_at | 2019-09-03T16:51:30Z
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 10:16 AM Donny Davis donny@fortnebula.com wrote:
Security group rules?
Donny Davis c: 805 814 6800
On Thu, Sep 12, 2019, 5:53 PM Lucio Seki lucioseki@gmail.com wrote:
Hi folks, I'm having troubles to ping6 a VM running over DevStack from its hypervisor. Could you please help me troubleshooting it?
I deployed DevStack with NEUTRON_CREATE_INITIAL_NETWORKS=False, and manually created the networks, subnets and router. Following is my router:
$ openstack router show router1 -c external_gateway_info -c interfaces_info
+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | external_gateway_info | {"network_id": "b87048ed-1be9-4f31-8d7e-fe74921aeec4", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "28a00bc3-b30b-456f-b26a-44b50d37183f", "ip_address": "10.2.0.199"}, {"subnet_id": "a9729beb-b297-4fec-8ec3-7703f7f6f4bc", "ip_address": "fd12:67:1::3c"}]} | | interfaces_info | [{"subnet_id": "081e8508-4ceb-4aaf-bf91-36a1e22a768c", "ip_address": "fd12:67:1:1::1", "port_id": "75391abd-8ac8-41f8-acf8-3dfaf2a6b08f"}]
|+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
I'm trying to ping6 the following VM:
$ openstack server list
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | 938854d0-80e9-45b2-bc29-8fe7651ffa93 | manila1 | ACTIVE | private1=fd12:67:1:1:f816:3eff:fe0e:17c3 | manila | manila |
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+
I intend to reach it via br-ex interface of the hypervisor:
$ ip a show dev br-ex 9: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 0e:82:a1:ba:77:4c brd ff:ff:ff:ff:ff:ff inet6 fd12:67:1::1/64 scope global valid_lft forever preferred_lft forever inet6 fe80::c82:a1ff:feba:774c/64 scope link valid_lft forever preferred_lft forever
The hypervisor has the following routes:
$ ip -6 route fd12:67:1:1::/64 via fd12:67:1::3c dev br-ex metric 1024 pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium fe80::/64 dev br-ex proto kernel metric 256 pref medium fe80::/64 dev br-int proto kernel metric 256 pref medium fe80::/64 dev tapa5cf4799-9f proto kernel metric 256 pref medium
And within the VM has the following routes:
root@ubuntu:~# ip -6 route root@ubuntu:~# ip -6 route fd12:67:1::/64 via fd12:67:1:1::1 dev ens3 metric 1024 pref medium fd12:67:1:1::/64 dev ens3 proto kernel metric 256 expires 86360sec pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium default via fe80::f816:3eff:feb3:bd56 dev ens3 proto ra metric 1024 expires 260sec hoplimit 64 pref medium
Though the ping6 from VM to hypervisor doesn't work: root@ubuntu:~# ping6 fd12:67:1::1 -c4 PING fd12:67:1::1 (fd12:67:1::1): 56 data bytes --- fd12:67:1::1 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss
I'm able to tcpdump inside the router1 netns and see that request packet is passing there, but can't see any reply packets:
$ sudo ip netns exec qrouter-5172472c-bbe7-4907-832a-e2239c8badb4 tcpdump -l -i any icmp6 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes 21:29:29.351358 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 0, length 64 21:29:30.033316 IP6 fe80::f816:3eff:feb3:bd56 > fe80::f816:3eff:fe0e:17c3: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe0e:17c3, length 32 21:29:30.035807 IP6 fe80::f816:3eff:fe0e:17c3 > fe80::f816:3eff:feb3:bd56: ICMP6, neighbor advertisement, tgt is fe80::f816:3eff:fe0e:17c3, length 24 21:29:30.353646 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 1, length 64 21:29:31.355410 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 2, length 64 21:29:32.357239 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 3, length 64
The same happens from hypervisor to VM. I only acan see the request packets, but no reply packets.
Thanks in advance, Lucio Seki
Hmm OK, I'll try to figure out what hacking create_neutron_initial_network does...
BTW, I noticed that I can ping6 the router interface at private subnet from the DevStack host:
$ ping6 fd12:67:1:1::1 PING fd12:67:1:1::1(fd12:67:1:1::1) 56 data bytes 64 bytes from fd12:67:1:1::1: icmp_seq=1 ttl=64 time=0.646 ms 64 bytes from fd12:67:1:1::1: icmp_seq=2 ttl=64 time=0.095 ms 64 bytes from fd12:67:1:1::1: icmp_seq=3 ttl=64 time=0.106 ms 64 bytes from fd12:67:1:1::1: icmp_seq=4 ttl=64 time=0.129 ms
And also I can ping6 the public subnet interface from the VM:
root@ubuntu:~# ping6 fd12:67:1::3c PING fd12:67:1::3c (fd12:67:1::3c): 56 data bytes ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=0 ttl=64 time=2.079 ms ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=1 ttl=64 time=1.385 ms ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=2 ttl=64 time=0.881 ms
Not sure if it means that there's something missing within the router itself...
On Fri, Sep 13, 2019 at 2:24 PM Donny Davis donny@fortnebula.com wrote:
Also I have no v6 address on my br-ex
On Fri, Sep 13, 2019 at 1:22 PM Donny Davis donny@fortnebula.com wrote:
Well here is the output from my rule list that is in prod right now with ipv6
+--------------------------------------+-------------+-----------+------------+-----------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group |
+--------------------------------------+-------------+-----------+------------+-----------------------+ | 9ab00b6f-2bc2-4554-818d-eff6e0570943 | None | 0.0.0.0/0 | | None | | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa | icmp | ::/0 | | None | | e7fd4840-5fbd-4709-b918-f80eac5cb6da | None | ::/0 | | None | | e9968d53-7efe-4a9e-ad42-1092ffaf52e7 | None | None | | None | | ec1ea961-9025-4229-92cf-618026a1851b | None | None | | None |
+--------------------------------------+-------------+-----------+------------+-----------------------+
+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-07-30T00:50:25Z
|| description |
|| direction | ingress
|| ether_type | IPv6
|| id | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa
|| location | Munch({'cloud': '', 'region_name': 'regionOne', 'zone': None, 'project': Munch({'id': 'e8fd161dc34c421a979a9e6421f823e9', 'name': 'openstackzuul', 'domain_id': None, 'domain_name': 'Default'})}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | e8fd161dc34c421a979a9e6421f823e9
|| protocol | icmp
|| remote_group_id | None
|| remote_ip_prefix | ::/0
|| revision_number | 0
|| security_group_id | bcedc0e0-e2e8-41fc-aeaa-afd2e10c7ab6
|| tags | []
|| updated_at | 2019-07-30T00:50:25Z
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 9:24 AM Lucio Seki lucioseki@gmail.com wrote:
Hi Donny, following are the rules:
$ openstack security group list --project admin
+--------------------------------------+---------+------------------------+----------------------------------+------+ | ID | Name | Description | Project | Tags |
+--------------------------------------+---------+------------------------+----------------------------------+------+ | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | default | Default security group | 68e3942285a24fb5bd1aed30e166aaee | [] |
+--------------------------------------+---------+------------------------+----------------------------------+------+
$ openstack security group rule list d0136b0e-ee51-461c-afa0-c5adb88dd0dd
+--------------------------------------+-------------+----------+------------+--------------------------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group |
+--------------------------------------+-------------+----------+------------+--------------------------------------+ | 38394345-3e44-4284-a519-cdd8af020f30 | tcp | ::/0 | 22:22 | None | | 40881f76-c87f-4685-b3af-c3497dd44837 | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 56d4ae52-195e-48df-871e-dc70b899b7ba | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 759edd06-b698-45ca-94cd-44e0cc2cc848 | ipv6-icmp | None | | None | | 762effae-b8e5-42ac-ba99-e85a7bc42455 | tcp | ::/0 | 22:22 | None | | 81f3588d-4159-4af2-ad50-ff6b76add9cf | ipv6-icmp | None | | None |
+--------------------------------------+-------------+----------+------------+--------------------------------------+
$ openstack security group rule show 759edd06-b698-45ca-94cd-44e0cc2cc848
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:41Z
|| description |
|| direction | egress
|| ether_type | IPv6
|| id | 759edd06-b698-45ca-94cd-44e0cc2cc848
|| location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | 68e3942285a24fb5bd1aed30e166aaee
|| protocol | ipv6-icmp
|| remote_group_id | None
|| remote_ip_prefix | None
|| revision_number | 0
|| security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
|| tags | []
|| updated_at | 2019-09-03T16:51:41Z
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
$ openstack security group rule show 81f3588d-4159-4af2-ad50-ff6b76add9cf
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:30Z
|| description |
|| direction | ingress
|| ether_type | IPv6
|| id | 81f3588d-4159-4af2-ad50-ff6b76add9cf
|| location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | 68e3942285a24fb5bd1aed30e166aaee
|| protocol | ipv6-icmp
|| remote_group_id | None
|| remote_ip_prefix | None
|| revision_number | 0
|| security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
|| tags | []
|| updated_at | 2019-09-03T16:51:30Z
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 10:16 AM Donny Davis donny@fortnebula.com wrote:
Security group rules?
Donny Davis c: 805 814 6800
On Thu, Sep 12, 2019, 5:53 PM Lucio Seki lucioseki@gmail.com wrote:
Hi folks, I'm having troubles to ping6 a VM running over DevStack from its hypervisor. Could you please help me troubleshooting it?
I deployed DevStack with NEUTRON_CREATE_INITIAL_NETWORKS=False, and manually created the networks, subnets and router. Following is my router:
$ openstack router show router1 -c external_gateway_info -c interfaces_info
+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | external_gateway_info | {"network_id": "b87048ed-1be9-4f31-8d7e-fe74921aeec4", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "28a00bc3-b30b-456f-b26a-44b50d37183f", "ip_address": "10.2.0.199"}, {"subnet_id": "a9729beb-b297-4fec-8ec3-7703f7f6f4bc", "ip_address": "fd12:67:1::3c"}]} | | interfaces_info | [{"subnet_id": "081e8508-4ceb-4aaf-bf91-36a1e22a768c", "ip_address": "fd12:67:1:1::1", "port_id": "75391abd-8ac8-41f8-acf8-3dfaf2a6b08f"}]
|+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
I'm trying to ping6 the following VM:
$ openstack server list
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | 938854d0-80e9-45b2-bc29-8fe7651ffa93 | manila1 | ACTIVE | private1=fd12:67:1:1:f816:3eff:fe0e:17c3 | manila | manila |
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+
I intend to reach it via br-ex interface of the hypervisor:
$ ip a show dev br-ex 9: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 0e:82:a1:ba:77:4c brd ff:ff:ff:ff:ff:ff inet6 fd12:67:1::1/64 scope global valid_lft forever preferred_lft forever inet6 fe80::c82:a1ff:feba:774c/64 scope link valid_lft forever preferred_lft forever
The hypervisor has the following routes:
$ ip -6 route fd12:67:1:1::/64 via fd12:67:1::3c dev br-ex metric 1024 pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium fe80::/64 dev br-ex proto kernel metric 256 pref medium fe80::/64 dev br-int proto kernel metric 256 pref medium fe80::/64 dev tapa5cf4799-9f proto kernel metric 256 pref medium
And within the VM has the following routes:
root@ubuntu:~# ip -6 route root@ubuntu:~# ip -6 route fd12:67:1::/64 via fd12:67:1:1::1 dev ens3 metric 1024 pref medium fd12:67:1:1::/64 dev ens3 proto kernel metric 256 expires 86360sec pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium default via fe80::f816:3eff:feb3:bd56 dev ens3 proto ra metric 1024 expires 260sec hoplimit 64 pref medium
Though the ping6 from VM to hypervisor doesn't work: root@ubuntu:~# ping6 fd12:67:1::1 -c4 PING fd12:67:1::1 (fd12:67:1::1): 56 data bytes --- fd12:67:1::1 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss
I'm able to tcpdump inside the router1 netns and see that request packet is passing there, but can't see any reply packets:
$ sudo ip netns exec qrouter-5172472c-bbe7-4907-832a-e2239c8badb4 tcpdump -l -i any icmp6 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes 21:29:29.351358 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 0, length 64 21:29:30.033316 IP6 fe80::f816:3eff:feb3:bd56 > fe80::f816:3eff:fe0e:17c3: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe0e:17c3, length 32 21:29:30.035807 IP6 fe80::f816:3eff:fe0e:17c3 > fe80::f816:3eff:feb3:bd56: ICMP6, neighbor advertisement, tgt is fe80::f816:3eff:fe0e:17c3, length 24 21:29:30.353646 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 1, length 64 21:29:31.355410 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 2, length 64 21:29:32.357239 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 3, length 64
The same happens from hypervisor to VM. I only acan see the request packets, but no reply packets.
Thanks in advance, Lucio Seki
So outbound traffic works, but inbound traffic doesn't?
Here is my icmp security group rule for ipv6. +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
| +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-07-30T00:50:25Z
| | description |
| | direction | ingress
| | ether_type | IPv6
| | id | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa
| | location | Munch({'cloud': '', 'region_name': 'regionOne', 'zone': None, 'project': Munch({'id': 'e8fd161dc34c421a979a9e6421f823e9', 'name': 'openstackzuul', 'domain_id': None, 'domain_name': 'Default'})}) | | name | None
| | port_range_max | None
| | port_range_min | None
| | project_id | e8fd161dc34c421a979a9e6421f823e9
| | protocol | icmp
| | remote_group_id | None
| | remote_ip_prefix | ::/0
| | revision_number | 0
| | security_group_id | bcedc0e0-e2e8-41fc-aeaa-afd2e10c7ab6
| | tags | []
| | updated_at | 2019-07-30T00:50:25Z
| +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 2:48 PM Lucio Seki lucioseki@gmail.com wrote:
Hmm OK, I'll try to figure out what hacking create_neutron_initial_network does...
BTW, I noticed that I can ping6 the router interface at private subnet from the DevStack host:
$ ping6 fd12:67:1:1::1 PING fd12:67:1:1::1(fd12:67:1:1::1) 56 data bytes 64 bytes from fd12:67:1:1::1: icmp_seq=1 ttl=64 time=0.646 ms 64 bytes from fd12:67:1:1::1: icmp_seq=2 ttl=64 time=0.095 ms 64 bytes from fd12:67:1:1::1: icmp_seq=3 ttl=64 time=0.106 ms 64 bytes from fd12:67:1:1::1: icmp_seq=4 ttl=64 time=0.129 ms
And also I can ping6 the public subnet interface from the VM:
root@ubuntu:~# ping6 fd12:67:1::3c PING fd12:67:1::3c (fd12:67:1::3c): 56 data bytes ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=0 ttl=64 time=2.079 ms ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=1 ttl=64 time=1.385 ms ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=2 ttl=64 time=0.881 ms
Not sure if it means that there's something missing within the router itself...
On Fri, Sep 13, 2019 at 2:24 PM Donny Davis donny@fortnebula.com wrote:
Also I have no v6 address on my br-ex
On Fri, Sep 13, 2019 at 1:22 PM Donny Davis donny@fortnebula.com wrote:
Well here is the output from my rule list that is in prod right now with ipv6
+--------------------------------------+-------------+-----------+------------+-----------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group |
+--------------------------------------+-------------+-----------+------------+-----------------------+ | 9ab00b6f-2bc2-4554-818d-eff6e0570943 | None | 0.0.0.0/0 | | None | | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa | icmp | ::/0 | | None | | e7fd4840-5fbd-4709-b918-f80eac5cb6da | None | ::/0 | | None | | e9968d53-7efe-4a9e-ad42-1092ffaf52e7 | None | None | | None | | ec1ea961-9025-4229-92cf-618026a1851b | None | None | | None |
+--------------------------------------+-------------+-----------+------------+-----------------------+
+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-07-30T00:50:25Z
|| description |
|| direction | ingress
|| ether_type | IPv6
|| id | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa
|| location | Munch({'cloud': '', 'region_name': 'regionOne', 'zone': None, 'project': Munch({'id': 'e8fd161dc34c421a979a9e6421f823e9', 'name': 'openstackzuul', 'domain_id': None, 'domain_name': 'Default'})}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | e8fd161dc34c421a979a9e6421f823e9
|| protocol | icmp
|| remote_group_id | None
|| remote_ip_prefix | ::/0
|| revision_number | 0
|| security_group_id | bcedc0e0-e2e8-41fc-aeaa-afd2e10c7ab6
|| tags | []
|| updated_at | 2019-07-30T00:50:25Z
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 9:24 AM Lucio Seki lucioseki@gmail.com wrote:
Hi Donny, following are the rules:
$ openstack security group list --project admin
+--------------------------------------+---------+------------------------+----------------------------------+------+ | ID | Name | Description | Project | Tags |
+--------------------------------------+---------+------------------------+----------------------------------+------+ | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | default | Default security group | 68e3942285a24fb5bd1aed30e166aaee | [] |
+--------------------------------------+---------+------------------------+----------------------------------+------+
$ openstack security group rule list d0136b0e-ee51-461c-afa0-c5adb88dd0dd
+--------------------------------------+-------------+----------+------------+--------------------------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group |
+--------------------------------------+-------------+----------+------------+--------------------------------------+ | 38394345-3e44-4284-a519-cdd8af020f30 | tcp | ::/0 | 22:22 | None | | 40881f76-c87f-4685-b3af-c3497dd44837 | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 56d4ae52-195e-48df-871e-dc70b899b7ba | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 759edd06-b698-45ca-94cd-44e0cc2cc848 | ipv6-icmp | None | | None | | 762effae-b8e5-42ac-ba99-e85a7bc42455 | tcp | ::/0 | 22:22 | None | | 81f3588d-4159-4af2-ad50-ff6b76add9cf | ipv6-icmp | None | | None |
+--------------------------------------+-------------+----------+------------+--------------------------------------+
$ openstack security group rule show 759edd06-b698-45ca-94cd-44e0cc2cc848
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:41Z
|| description |
|| direction | egress
|| ether_type | IPv6
|| id | 759edd06-b698-45ca-94cd-44e0cc2cc848
|| location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | 68e3942285a24fb5bd1aed30e166aaee
|| protocol | ipv6-icmp
|| remote_group_id | None
|| remote_ip_prefix | None
|| revision_number | 0
|| security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
|| tags | []
|| updated_at | 2019-09-03T16:51:41Z
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
$ openstack security group rule show 81f3588d-4159-4af2-ad50-ff6b76add9cf
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:30Z
|| description |
|| direction | ingress
|| ether_type | IPv6
|| id | 81f3588d-4159-4af2-ad50-ff6b76add9cf
|| location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | 68e3942285a24fb5bd1aed30e166aaee
|| protocol | ipv6-icmp
|| remote_group_id | None
|| remote_ip_prefix | None
|| revision_number | 0
|| security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
|| tags | []
|| updated_at | 2019-09-03T16:51:30Z
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 10:16 AM Donny Davis donny@fortnebula.com wrote:
Security group rules?
Donny Davis c: 805 814 6800
On Thu, Sep 12, 2019, 5:53 PM Lucio Seki lucioseki@gmail.com wrote:
Hi folks, I'm having troubles to ping6 a VM running over DevStack from its hypervisor. Could you please help me troubleshooting it?
I deployed DevStack with NEUTRON_CREATE_INITIAL_NETWORKS=False, and manually created the networks, subnets and router. Following is my router:
$ openstack router show router1 -c external_gateway_info -c interfaces_info
+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | external_gateway_info | {"network_id": "b87048ed-1be9-4f31-8d7e-fe74921aeec4", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "28a00bc3-b30b-456f-b26a-44b50d37183f", "ip_address": "10.2.0.199"}, {"subnet_id": "a9729beb-b297-4fec-8ec3-7703f7f6f4bc", "ip_address": "fd12:67:1::3c"}]} | | interfaces_info | [{"subnet_id": "081e8508-4ceb-4aaf-bf91-36a1e22a768c", "ip_address": "fd12:67:1:1::1", "port_id": "75391abd-8ac8-41f8-acf8-3dfaf2a6b08f"}]
|+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
I'm trying to ping6 the following VM:
$ openstack server list
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | 938854d0-80e9-45b2-bc29-8fe7651ffa93 | manila1 | ACTIVE | private1=fd12:67:1:1:f816:3eff:fe0e:17c3 | manila | manila |
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+
I intend to reach it via br-ex interface of the hypervisor:
$ ip a show dev br-ex 9: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 0e:82:a1:ba:77:4c brd ff:ff:ff:ff:ff:ff inet6 fd12:67:1::1/64 scope global valid_lft forever preferred_lft forever inet6 fe80::c82:a1ff:feba:774c/64 scope link valid_lft forever preferred_lft forever
The hypervisor has the following routes:
$ ip -6 route fd12:67:1:1::/64 via fd12:67:1::3c dev br-ex metric 1024 pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium fe80::/64 dev br-ex proto kernel metric 256 pref medium fe80::/64 dev br-int proto kernel metric 256 pref medium fe80::/64 dev tapa5cf4799-9f proto kernel metric 256 pref medium
And within the VM has the following routes:
root@ubuntu:~# ip -6 route root@ubuntu:~# ip -6 route fd12:67:1::/64 via fd12:67:1:1::1 dev ens3 metric 1024 pref medium fd12:67:1:1::/64 dev ens3 proto kernel metric 256 expires 86360sec pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium default via fe80::f816:3eff:feb3:bd56 dev ens3 proto ra metric 1024 expires 260sec hoplimit 64 pref medium
Though the ping6 from VM to hypervisor doesn't work: root@ubuntu:~# ping6 fd12:67:1::1 -c4 PING fd12:67:1::1 (fd12:67:1::1): 56 data bytes --- fd12:67:1::1 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss
I'm able to tcpdump inside the router1 netns and see that request packet is passing there, but can't see any reply packets:
$ sudo ip netns exec qrouter-5172472c-bbe7-4907-832a-e2239c8badb4 tcpdump -l -i any icmp6 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes 21:29:29.351358 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 0, length 64 21:29:30.033316 IP6 fe80::f816:3eff:feb3:bd56 > fe80::f816:3eff:fe0e:17c3: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe0e:17c3, length 32 21:29:30.035807 IP6 fe80::f816:3eff:fe0e:17c3 > fe80::f816:3eff:feb3:bd56: ICMP6, neighbor advertisement, tgt is fe80::f816:3eff:fe0e:17c3, length 24 21:29:30.353646 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 1, length 64 21:29:31.355410 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 2, length 64 21:29:32.357239 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 3, length 64
The same happens from hypervisor to VM. I only acan see the request packets, but no reply packets.
Thanks in advance, Lucio Seki
I drawed the environment I have [1]. Also attached it as an image.
Currently I have the interfaces 1 pinging 3, and 4 pinging 2. When I attempt to make 1 ping 4, I can only see the request packets at 2. When I attempt to make 4 ping 1, I can only see the request packets at 3.
[1] https://docs.google.com/drawings/d/1zhgN9TCINrVIlQpZT9hlCrHxWrQerjIo62oRmTGx...
On Fri, Sep 13, 2019 at 3:55 PM Donny Davis donny@fortnebula.com wrote:
So outbound traffic works, but inbound traffic doesn't?
Here is my icmp security group rule for ipv6.
+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-07-30T00:50:25Z
|| description |
|| direction | ingress
|| ether_type | IPv6
|| id | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa
|| location | Munch({'cloud': '', 'region_name': 'regionOne', 'zone': None, 'project': Munch({'id': 'e8fd161dc34c421a979a9e6421f823e9', 'name': 'openstackzuul', 'domain_id': None, 'domain_name': 'Default'})}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | e8fd161dc34c421a979a9e6421f823e9
|| protocol | icmp
|| remote_group_id | None
|| remote_ip_prefix | ::/0
|| revision_number | 0
|| security_group_id | bcedc0e0-e2e8-41fc-aeaa-afd2e10c7ab6
|| tags | []
|| updated_at | 2019-07-30T00:50:25Z
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 2:48 PM Lucio Seki lucioseki@gmail.com wrote:
Hmm OK, I'll try to figure out what hacking create_neutron_initial_network does...
BTW, I noticed that I can ping6 the router interface at private subnet from the DevStack host:
$ ping6 fd12:67:1:1::1 PING fd12:67:1:1::1(fd12:67:1:1::1) 56 data bytes 64 bytes from fd12:67:1:1::1: icmp_seq=1 ttl=64 time=0.646 ms 64 bytes from fd12:67:1:1::1: icmp_seq=2 ttl=64 time=0.095 ms 64 bytes from fd12:67:1:1::1: icmp_seq=3 ttl=64 time=0.106 ms 64 bytes from fd12:67:1:1::1: icmp_seq=4 ttl=64 time=0.129 ms
And also I can ping6 the public subnet interface from the VM:
root@ubuntu:~# ping6 fd12:67:1::3c PING fd12:67:1::3c (fd12:67:1::3c): 56 data bytes ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=0 ttl=64 time=2.079 ms ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=1 ttl=64 time=1.385 ms ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=2 ttl=64 time=0.881 ms
Not sure if it means that there's something missing within the router itself...
On Fri, Sep 13, 2019 at 2:24 PM Donny Davis donny@fortnebula.com wrote:
Also I have no v6 address on my br-ex
On Fri, Sep 13, 2019 at 1:22 PM Donny Davis donny@fortnebula.com wrote:
Well here is the output from my rule list that is in prod right now with ipv6
+--------------------------------------+-------------+-----------+------------+-----------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group |
+--------------------------------------+-------------+-----------+------------+-----------------------+ | 9ab00b6f-2bc2-4554-818d-eff6e0570943 | None | 0.0.0.0/0 | | None | | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa | icmp | ::/0 | | None | | e7fd4840-5fbd-4709-b918-f80eac5cb6da | None | ::/0 | | None | | e9968d53-7efe-4a9e-ad42-1092ffaf52e7 | None | None | | None | | ec1ea961-9025-4229-92cf-618026a1851b | None | None | | None |
+--------------------------------------+-------------+-----------+------------+-----------------------+
+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-07-30T00:50:25Z
|| description |
|| direction | ingress
|| ether_type | IPv6
|| id | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa
|| location | Munch({'cloud': '', 'region_name': 'regionOne', 'zone': None, 'project': Munch({'id': 'e8fd161dc34c421a979a9e6421f823e9', 'name': 'openstackzuul', 'domain_id': None, 'domain_name': 'Default'})}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | e8fd161dc34c421a979a9e6421f823e9
|| protocol | icmp
|| remote_group_id | None
|| remote_ip_prefix | ::/0
|| revision_number | 0
|| security_group_id | bcedc0e0-e2e8-41fc-aeaa-afd2e10c7ab6
|| tags | []
|| updated_at | 2019-07-30T00:50:25Z
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 9:24 AM Lucio Seki lucioseki@gmail.com wrote:
Hi Donny, following are the rules:
$ openstack security group list --project admin
+--------------------------------------+---------+------------------------+----------------------------------+------+ | ID | Name | Description | Project | Tags |
+--------------------------------------+---------+------------------------+----------------------------------+------+ | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | default | Default security group | 68e3942285a24fb5bd1aed30e166aaee | [] |
+--------------------------------------+---------+------------------------+----------------------------------+------+
$ openstack security group rule list d0136b0e-ee51-461c-afa0-c5adb88dd0dd
+--------------------------------------+-------------+----------+------------+--------------------------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group |
+--------------------------------------+-------------+----------+------------+--------------------------------------+ | 38394345-3e44-4284-a519-cdd8af020f30 | tcp | ::/0 | 22:22 | None | | 40881f76-c87f-4685-b3af-c3497dd44837 | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 56d4ae52-195e-48df-871e-dc70b899b7ba | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 759edd06-b698-45ca-94cd-44e0cc2cc848 | ipv6-icmp | None | | None | | 762effae-b8e5-42ac-ba99-e85a7bc42455 | tcp | ::/0 | 22:22 | None | | 81f3588d-4159-4af2-ad50-ff6b76add9cf | ipv6-icmp | None | | None |
+--------------------------------------+-------------+----------+------------+--------------------------------------+
$ openstack security group rule show 759edd06-b698-45ca-94cd-44e0cc2cc848
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:41Z
|| description |
|| direction | egress
|| ether_type | IPv6
|| id | 759edd06-b698-45ca-94cd-44e0cc2cc848
|| location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | 68e3942285a24fb5bd1aed30e166aaee
|| protocol | ipv6-icmp
|| remote_group_id | None
|| remote_ip_prefix | None
|| revision_number | 0
|| security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
|| tags | []
|| updated_at | 2019-09-03T16:51:41Z
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
$ openstack security group rule show 81f3588d-4159-4af2-ad50-ff6b76add9cf
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:30Z
|| description |
|| direction | ingress
|| ether_type | IPv6
|| id | 81f3588d-4159-4af2-ad50-ff6b76add9cf
|| location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | 68e3942285a24fb5bd1aed30e166aaee
|| protocol | ipv6-icmp
|| remote_group_id | None
|| remote_ip_prefix | None
|| revision_number | 0
|| security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
|| tags | []
|| updated_at | 2019-09-03T16:51:30Z
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 10:16 AM Donny Davis donny@fortnebula.com wrote:
Security group rules?
Donny Davis c: 805 814 6800
On Thu, Sep 12, 2019, 5:53 PM Lucio Seki lucioseki@gmail.com wrote:
> Hi folks, I'm having troubles to ping6 a VM running over DevStack > from its hypervisor. > Could you please help me troubleshooting it? > > I deployed DevStack with NEUTRON_CREATE_INITIAL_NETWORKS=False, > and manually created the networks, subnets and router. Following is > my router: > > $ openstack router show router1 -c external_gateway_info -c > interfaces_info > > +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > | Field | Value > > > | > > +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > | external_gateway_info | {"network_id": > "b87048ed-1be9-4f31-8d7e-fe74921aeec4", "enable_snat": true, > "external_fixed_ips": [{"subnet_id": > "28a00bc3-b30b-456f-b26a-44b50d37183f", "ip_address": "10.2.0.199"}, > {"subnet_id": "a9729beb-b297-4fec-8ec3-7703f7f6f4bc", "ip_address": > "fd12:67:1::3c"}]} | > | interfaces_info | [{"subnet_id": > "081e8508-4ceb-4aaf-bf91-36a1e22a768c", "ip_address": "fd12:67:1:1::1", > "port_id": "75391abd-8ac8-41f8-acf8-3dfaf2a6b08f"}] > > | > > +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > > I'm trying to ping6 the following VM: > > $ openstack server list > > +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ > | ID | Name | Status | Networks > | Image | Flavor | > > +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ > | 938854d0-80e9-45b2-bc29-8fe7651ffa93 | manila1 | ACTIVE | > private1=fd12:67:1:1:f816:3eff:fe0e:17c3 | manila | manila | > > +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ > > I intend to reach it via br-ex interface of the hypervisor: > > $ ip a show dev br-ex > 9: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue > state UNKNOWN group default qlen 1000 > link/ether 0e:82:a1:ba:77:4c brd ff:ff:ff:ff:ff:ff > inet6 fd12:67:1::1/64 scope global > valid_lft forever preferred_lft forever > inet6 fe80::c82:a1ff:feba:774c/64 scope link > valid_lft forever preferred_lft forever > > The hypervisor has the following routes: > > $ ip -6 route > fd12:67:1:1::/64 via fd12:67:1::3c dev br-ex metric 1024 pref medium > fe80::/64 dev ens3 proto kernel metric 256 pref medium > fe80::/64 dev br-ex proto kernel metric 256 pref medium > fe80::/64 dev br-int proto kernel metric 256 pref medium > fe80::/64 dev tapa5cf4799-9f proto kernel metric 256 pref medium > > And within the VM has the following routes: > > root@ubuntu:~# ip -6 route > root@ubuntu:~# ip -6 route > fd12:67:1::/64 via fd12:67:1:1::1 dev ens3 metric 1024 pref medium > fd12:67:1:1::/64 dev ens3 proto kernel metric 256 expires 86360sec > pref medium > fe80::/64 dev ens3 proto kernel metric 256 pref medium > default via fe80::f816:3eff:feb3:bd56 dev ens3 proto ra metric 1024 > expires 260sec hoplimit 64 pref medium > > Though the ping6 from VM to hypervisor doesn't work: > root@ubuntu:~# ping6 fd12:67:1::1 -c4 > PING fd12:67:1::1 (fd12:67:1::1): 56 data bytes > --- fd12:67:1::1 ping statistics --- > 4 packets transmitted, 0 packets received, 100% packet loss > > I'm able to tcpdump inside the router1 netns and see that request > packet is passing there, but can't see any reply packets: > > $ sudo ip netns exec qrouter-5172472c-bbe7-4907-832a-e2239c8badb4 > tcpdump -l -i any icmp6 > tcpdump: verbose output suppressed, use -v or -vv for full protocol > decode > listening on any, link-type LINUX_SLL (Linux cooked), capture size > 262144 bytes > 21:29:29.351358 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: > ICMP6, echo request, seq 0, length 64 > 21:29:30.033316 IP6 fe80::f816:3eff:feb3:bd56 > > fe80::f816:3eff:fe0e:17c3: ICMP6, neighbor solicitation, who has > fe80::f816:3eff:fe0e:17c3, length 32 > 21:29:30.035807 IP6 fe80::f816:3eff:fe0e:17c3 > > fe80::f816:3eff:feb3:bd56: ICMP6, neighbor advertisement, tgt is > fe80::f816:3eff:fe0e:17c3, length 24 > 21:29:30.353646 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: > ICMP6, echo request, seq 1, length 64 > 21:29:31.355410 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: > ICMP6, echo request, seq 2, length 64 > 21:29:32.357239 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: > ICMP6, echo request, seq 3, length 64 > > The same happens from hypervisor to VM. I only acan see the request > packets, but no reply packets. > > Thanks in advance, > Lucio Seki >
I recreated my security group rules, to set remote_ip_prefix to ::/0 instead of None as in Donny's environment, but made no difference. :-(
On Fri, Sep 13, 2019 at 3:55 PM Donny Davis donny@fortnebula.com wrote:
So outbound traffic works, but inbound traffic doesn't?
Here is my icmp security group rule for ipv6.
+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-07-30T00:50:25Z
|| description |
|| direction | ingress
|| ether_type | IPv6
|| id | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa
|| location | Munch({'cloud': '', 'region_name': 'regionOne', 'zone': None, 'project': Munch({'id': 'e8fd161dc34c421a979a9e6421f823e9', 'name': 'openstackzuul', 'domain_id': None, 'domain_name': 'Default'})}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | e8fd161dc34c421a979a9e6421f823e9
|| protocol | icmp
|| remote_group_id | None
|| remote_ip_prefix | ::/0
|| revision_number | 0
|| security_group_id | bcedc0e0-e2e8-41fc-aeaa-afd2e10c7ab6
|| tags | []
|| updated_at | 2019-07-30T00:50:25Z
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 2:48 PM Lucio Seki lucioseki@gmail.com wrote:
Hmm OK, I'll try to figure out what hacking create_neutron_initial_network does...
BTW, I noticed that I can ping6 the router interface at private subnet from the DevStack host:
$ ping6 fd12:67:1:1::1 PING fd12:67:1:1::1(fd12:67:1:1::1) 56 data bytes 64 bytes from fd12:67:1:1::1: icmp_seq=1 ttl=64 time=0.646 ms 64 bytes from fd12:67:1:1::1: icmp_seq=2 ttl=64 time=0.095 ms 64 bytes from fd12:67:1:1::1: icmp_seq=3 ttl=64 time=0.106 ms 64 bytes from fd12:67:1:1::1: icmp_seq=4 ttl=64 time=0.129 ms
And also I can ping6 the public subnet interface from the VM:
root@ubuntu:~# ping6 fd12:67:1::3c PING fd12:67:1::3c (fd12:67:1::3c): 56 data bytes ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=0 ttl=64 time=2.079 ms ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=1 ttl=64 time=1.385 ms ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=2 ttl=64 time=0.881 ms
Not sure if it means that there's something missing within the router itself...
On Fri, Sep 13, 2019 at 2:24 PM Donny Davis donny@fortnebula.com wrote:
Also I have no v6 address on my br-ex
On Fri, Sep 13, 2019 at 1:22 PM Donny Davis donny@fortnebula.com wrote:
Well here is the output from my rule list that is in prod right now with ipv6
+--------------------------------------+-------------+-----------+------------+-----------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group |
+--------------------------------------+-------------+-----------+------------+-----------------------+ | 9ab00b6f-2bc2-4554-818d-eff6e0570943 | None | 0.0.0.0/0 | | None | | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa | icmp | ::/0 | | None | | e7fd4840-5fbd-4709-b918-f80eac5cb6da | None | ::/0 | | None | | e9968d53-7efe-4a9e-ad42-1092ffaf52e7 | None | None | | None | | ec1ea961-9025-4229-92cf-618026a1851b | None | None | | None |
+--------------------------------------+-------------+-----------+------------+-----------------------+
+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-07-30T00:50:25Z
|| description |
|| direction | ingress
|| ether_type | IPv6
|| id | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa
|| location | Munch({'cloud': '', 'region_name': 'regionOne', 'zone': None, 'project': Munch({'id': 'e8fd161dc34c421a979a9e6421f823e9', 'name': 'openstackzuul', 'domain_id': None, 'domain_name': 'Default'})}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | e8fd161dc34c421a979a9e6421f823e9
|| protocol | icmp
|| remote_group_id | None
|| remote_ip_prefix | ::/0
|| revision_number | 0
|| security_group_id | bcedc0e0-e2e8-41fc-aeaa-afd2e10c7ab6
|| tags | []
|| updated_at | 2019-07-30T00:50:25Z
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 9:24 AM Lucio Seki lucioseki@gmail.com wrote:
Hi Donny, following are the rules:
$ openstack security group list --project admin
+--------------------------------------+---------+------------------------+----------------------------------+------+ | ID | Name | Description | Project | Tags |
+--------------------------------------+---------+------------------------+----------------------------------+------+ | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | default | Default security group | 68e3942285a24fb5bd1aed30e166aaee | [] |
+--------------------------------------+---------+------------------------+----------------------------------+------+
$ openstack security group rule list d0136b0e-ee51-461c-afa0-c5adb88dd0dd
+--------------------------------------+-------------+----------+------------+--------------------------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group |
+--------------------------------------+-------------+----------+------------+--------------------------------------+ | 38394345-3e44-4284-a519-cdd8af020f30 | tcp | ::/0 | 22:22 | None | | 40881f76-c87f-4685-b3af-c3497dd44837 | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 56d4ae52-195e-48df-871e-dc70b899b7ba | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 759edd06-b698-45ca-94cd-44e0cc2cc848 | ipv6-icmp | None | | None | | 762effae-b8e5-42ac-ba99-e85a7bc42455 | tcp | ::/0 | 22:22 | None | | 81f3588d-4159-4af2-ad50-ff6b76add9cf | ipv6-icmp | None | | None |
+--------------------------------------+-------------+----------+------------+--------------------------------------+
$ openstack security group rule show 759edd06-b698-45ca-94cd-44e0cc2cc848
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:41Z
|| description |
|| direction | egress
|| ether_type | IPv6
|| id | 759edd06-b698-45ca-94cd-44e0cc2cc848
|| location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | 68e3942285a24fb5bd1aed30e166aaee
|| protocol | ipv6-icmp
|| remote_group_id | None
|| remote_ip_prefix | None
|| revision_number | 0
|| security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
|| tags | []
|| updated_at | 2019-09-03T16:51:41Z
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
$ openstack security group rule show 81f3588d-4159-4af2-ad50-ff6b76add9cf
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:30Z
|| description |
|| direction | ingress
|| ether_type | IPv6
|| id | 81f3588d-4159-4af2-ad50-ff6b76add9cf
|| location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | 68e3942285a24fb5bd1aed30e166aaee
|| protocol | ipv6-icmp
|| remote_group_id | None
|| remote_ip_prefix | None
|| revision_number | 0
|| security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
|| tags | []
|| updated_at | 2019-09-03T16:51:30Z
|+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 10:16 AM Donny Davis donny@fortnebula.com wrote:
Security group rules?
Donny Davis c: 805 814 6800
On Thu, Sep 12, 2019, 5:53 PM Lucio Seki lucioseki@gmail.com wrote:
> Hi folks, I'm having troubles to ping6 a VM running over DevStack > from its hypervisor. > Could you please help me troubleshooting it? > > I deployed DevStack with NEUTRON_CREATE_INITIAL_NETWORKS=False, > and manually created the networks, subnets and router. Following is > my router: > > $ openstack router show router1 -c external_gateway_info -c > interfaces_info > > +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > | Field | Value > > > | > > +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > | external_gateway_info | {"network_id": > "b87048ed-1be9-4f31-8d7e-fe74921aeec4", "enable_snat": true, > "external_fixed_ips": [{"subnet_id": > "28a00bc3-b30b-456f-b26a-44b50d37183f", "ip_address": "10.2.0.199"}, > {"subnet_id": "a9729beb-b297-4fec-8ec3-7703f7f6f4bc", "ip_address": > "fd12:67:1::3c"}]} | > | interfaces_info | [{"subnet_id": > "081e8508-4ceb-4aaf-bf91-36a1e22a768c", "ip_address": "fd12:67:1:1::1", > "port_id": "75391abd-8ac8-41f8-acf8-3dfaf2a6b08f"}] > > | > > +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ > > I'm trying to ping6 the following VM: > > $ openstack server list > > +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ > | ID | Name | Status | Networks > | Image | Flavor | > > +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ > | 938854d0-80e9-45b2-bc29-8fe7651ffa93 | manila1 | ACTIVE | > private1=fd12:67:1:1:f816:3eff:fe0e:17c3 | manila | manila | > > +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ > > I intend to reach it via br-ex interface of the hypervisor: > > $ ip a show dev br-ex > 9: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue > state UNKNOWN group default qlen 1000 > link/ether 0e:82:a1:ba:77:4c brd ff:ff:ff:ff:ff:ff > inet6 fd12:67:1::1/64 scope global > valid_lft forever preferred_lft forever > inet6 fe80::c82:a1ff:feba:774c/64 scope link > valid_lft forever preferred_lft forever > > The hypervisor has the following routes: > > $ ip -6 route > fd12:67:1:1::/64 via fd12:67:1::3c dev br-ex metric 1024 pref medium > fe80::/64 dev ens3 proto kernel metric 256 pref medium > fe80::/64 dev br-ex proto kernel metric 256 pref medium > fe80::/64 dev br-int proto kernel metric 256 pref medium > fe80::/64 dev tapa5cf4799-9f proto kernel metric 256 pref medium > > And within the VM has the following routes: > > root@ubuntu:~# ip -6 route > root@ubuntu:~# ip -6 route > fd12:67:1::/64 via fd12:67:1:1::1 dev ens3 metric 1024 pref medium > fd12:67:1:1::/64 dev ens3 proto kernel metric 256 expires 86360sec > pref medium > fe80::/64 dev ens3 proto kernel metric 256 pref medium > default via fe80::f816:3eff:feb3:bd56 dev ens3 proto ra metric 1024 > expires 260sec hoplimit 64 pref medium > > Though the ping6 from VM to hypervisor doesn't work: > root@ubuntu:~# ping6 fd12:67:1::1 -c4 > PING fd12:67:1::1 (fd12:67:1::1): 56 data bytes > --- fd12:67:1::1 ping statistics --- > 4 packets transmitted, 0 packets received, 100% packet loss > > I'm able to tcpdump inside the router1 netns and see that request > packet is passing there, but can't see any reply packets: > > $ sudo ip netns exec qrouter-5172472c-bbe7-4907-832a-e2239c8badb4 > tcpdump -l -i any icmp6 > tcpdump: verbose output suppressed, use -v or -vv for full protocol > decode > listening on any, link-type LINUX_SLL (Linux cooked), capture size > 262144 bytes > 21:29:29.351358 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: > ICMP6, echo request, seq 0, length 64 > 21:29:30.033316 IP6 fe80::f816:3eff:feb3:bd56 > > fe80::f816:3eff:fe0e:17c3: ICMP6, neighbor solicitation, who has > fe80::f816:3eff:fe0e:17c3, length 32 > 21:29:30.035807 IP6 fe80::f816:3eff:fe0e:17c3 > > fe80::f816:3eff:feb3:bd56: ICMP6, neighbor advertisement, tgt is > fe80::f816:3eff:fe0e:17c3, length 24 > 21:29:30.353646 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: > ICMP6, echo request, seq 1, length 64 > 21:29:31.355410 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: > ICMP6, echo request, seq 2, length 64 > 21:29:32.357239 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: > ICMP6, echo request, seq 3, length 64 > > The same happens from hypervisor to VM. I only acan see the request > packets, but no reply packets. > > Thanks in advance, > Lucio Seki >
Can you check if ipv6 forwarding is enabled in the router namespace?
net.ipv6.conf.all.forwarding=1
On Sat, 14 Sep 2019 at 02:13, Lucio Seki lucioseki@gmail.com wrote:
I recreated my security group rules, to set remote_ip_prefix to ::/0 instead of None as in Donny's environment, but made no difference. :-(
On Fri, Sep 13, 2019 at 3:55 PM Donny Davis donny@fortnebula.com wrote:
So outbound traffic works, but inbound traffic doesn't?
Here is my icmp security group rule for ipv6. +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-07-30T00:50:25Z | | description | | | direction | ingress | | ether_type | IPv6 | | id | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa | | location | Munch({'cloud': '', 'region_name': 'regionOne', 'zone': None, 'project': Munch({'id': 'e8fd161dc34c421a979a9e6421f823e9', 'name': 'openstackzuul', 'domain_id': None, 'domain_name': 'Default'})}) | | name | None | | port_range_max | None | | port_range_min | None | | project_id | e8fd161dc34c421a979a9e6421f823e9 | | protocol | icmp | | remote_group_id | None | | remote_ip_prefix | ::/0 | | revision_number | 0 | | security_group_id | bcedc0e0-e2e8-41fc-aeaa-afd2e10c7ab6 | | tags | [] | | updated_at | 2019-07-30T00:50:25Z | +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 2:48 PM Lucio Seki lucioseki@gmail.com wrote:
Hmm OK, I'll try to figure out what hacking create_neutron_initial_network does...
BTW, I noticed that I can ping6 the router interface at private subnet from the DevStack host:
$ ping6 fd12:67:1:1::1 PING fd12:67:1:1::1(fd12:67:1:1::1) 56 data bytes 64 bytes from fd12:67:1:1::1: icmp_seq=1 ttl=64 time=0.646 ms 64 bytes from fd12:67:1:1::1: icmp_seq=2 ttl=64 time=0.095 ms 64 bytes from fd12:67:1:1::1: icmp_seq=3 ttl=64 time=0.106 ms 64 bytes from fd12:67:1:1::1: icmp_seq=4 ttl=64 time=0.129 ms
And also I can ping6 the public subnet interface from the VM:
root@ubuntu:~# ping6 fd12:67:1::3c PING fd12:67:1::3c (fd12:67:1::3c): 56 data bytes ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=0 ttl=64 time=2.079 ms ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=1 ttl=64 time=1.385 ms ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=2 ttl=64 time=0.881 ms
Not sure if it means that there's something missing within the router itself...
On Fri, Sep 13, 2019 at 2:24 PM Donny Davis donny@fortnebula.com wrote:
Also I have no v6 address on my br-ex
On Fri, Sep 13, 2019 at 1:22 PM Donny Davis donny@fortnebula.com wrote:
Well here is the output from my rule list that is in prod right now with ipv6 +--------------------------------------+-------------+-----------+------------+-----------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group | +--------------------------------------+-------------+-----------+------------+-----------------------+ | 9ab00b6f-2bc2-4554-818d-eff6e0570943 | None | 0.0.0.0/0 | | None | | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa | icmp | ::/0 | | None | | e7fd4840-5fbd-4709-b918-f80eac5cb6da | None | ::/0 | | None | | e9968d53-7efe-4a9e-ad42-1092ffaf52e7 | None | None | | None | | ec1ea961-9025-4229-92cf-618026a1851b | None | None | | None | +--------------------------------------+-------------+-----------+------------+-----------------------+
+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-07-30T00:50:25Z | | description | | | direction | ingress | | ether_type | IPv6 | | id | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa | | location | Munch({'cloud': '', 'region_name': 'regionOne', 'zone': None, 'project': Munch({'id': 'e8fd161dc34c421a979a9e6421f823e9', 'name': 'openstackzuul', 'domain_id': None, 'domain_name': 'Default'})}) | | name | None | | port_range_max | None | | port_range_min | None | | project_id | e8fd161dc34c421a979a9e6421f823e9 | | protocol | icmp | | remote_group_id | None | | remote_ip_prefix | ::/0 | | revision_number | 0 | | security_group_id | bcedc0e0-e2e8-41fc-aeaa-afd2e10c7ab6 | | tags | [] | | updated_at | 2019-07-30T00:50:25Z | +-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 9:24 AM Lucio Seki lucioseki@gmail.com wrote:
Hi Donny, following are the rules:
$ openstack security group list --project admin +--------------------------------------+---------+------------------------+----------------------------------+------+ | ID | Name | Description | Project | Tags | +--------------------------------------+---------+------------------------+----------------------------------+------+ | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | default | Default security group | 68e3942285a24fb5bd1aed30e166aaee | [] | +--------------------------------------+---------+------------------------+----------------------------------+------+
$ openstack security group rule list d0136b0e-ee51-461c-afa0-c5adb88dd0dd +--------------------------------------+-------------+----------+------------+--------------------------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group | +--------------------------------------+-------------+----------+------------+--------------------------------------+ | 38394345-3e44-4284-a519-cdd8af020f30 | tcp | ::/0 | 22:22 | None | | 40881f76-c87f-4685-b3af-c3497dd44837 | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 56d4ae52-195e-48df-871e-dc70b899b7ba | None | None | | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | 759edd06-b698-45ca-94cd-44e0cc2cc848 | ipv6-icmp | None | | None | | 762effae-b8e5-42ac-ba99-e85a7bc42455 | tcp | ::/0 | 22:22 | None | | 81f3588d-4159-4af2-ad50-ff6b76add9cf | ipv6-icmp | None | | None | +--------------------------------------+-------------+----------+------------+--------------------------------------+
$ openstack security group rule show 759edd06-b698-45ca-94cd-44e0cc2cc848 +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:41Z | | description | | | direction | egress | | ether_type | IPv6 | | id | 759edd06-b698-45ca-94cd-44e0cc2cc848 | | location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None | | port_range_max | None | | port_range_min | None | | project_id | 68e3942285a24fb5bd1aed30e166aaee | | protocol | ipv6-icmp | | remote_group_id | None | | remote_ip_prefix | None | | revision_number | 0 | | security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | tags | [] | | updated_at | 2019-09-03T16:51:41Z | +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
$ openstack security group rule show 81f3588d-4159-4af2-ad50-ff6b76add9cf +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | created_at | 2019-09-03T16:51:30Z | | description | | | direction | ingress | | ether_type | IPv6 | | id | 81f3588d-4159-4af2-ad50-ff6b76add9cf | | location | Munch({'project': Munch({'domain_id': 'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) | | name | None | | port_range_max | None | | port_range_min | None | | project_id | 68e3942285a24fb5bd1aed30e166aaee | | protocol | ipv6-icmp | | remote_group_id | None | | remote_ip_prefix | None | | revision_number | 0 | | security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | | tags | [] | | updated_at | 2019-09-03T16:51:30Z | +-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 10:16 AM Donny Davis donny@fortnebula.com wrote: > > Security group rules? > > Donny Davis > c: 805 814 6800 > > On Thu, Sep 12, 2019, 5:53 PM Lucio Seki lucioseki@gmail.com wrote: >> >> Hi folks, I'm having troubles to ping6 a VM running over DevStack from its hypervisor. >> Could you please help me troubleshooting it? >> >> I deployed DevStack with NEUTRON_CREATE_INITIAL_NETWORKS=False, >> and manually created the networks, subnets and router. Following is my router: >> >> $ openstack router show router1 -c external_gateway_info -c interfaces_info >> +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ >> | Field | Value | >> +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ >> | external_gateway_info | {"network_id": "b87048ed-1be9-4f31-8d7e-fe74921aeec4", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "28a00bc3-b30b-456f-b26a-44b50d37183f", "ip_address": "10.2.0.199"}, {"subnet_id": "a9729beb-b297-4fec-8ec3-7703f7f6f4bc", "ip_address": "fd12:67:1::3c"}]} | >> | interfaces_info | [{"subnet_id": "081e8508-4ceb-4aaf-bf91-36a1e22a768c", "ip_address": "fd12:67:1:1::1", "port_id": "75391abd-8ac8-41f8-acf8-3dfaf2a6b08f"}] | >> +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ >> >> I'm trying to ping6 the following VM: >> >> $ openstack server list >> +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ >> | ID | Name | Status | Networks | Image | Flavor | >> +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ >> | 938854d0-80e9-45b2-bc29-8fe7651ffa93 | manila1 | ACTIVE | private1=fd12:67:1:1:f816:3eff:fe0e:17c3 | manila | manila | >> +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ >> >> I intend to reach it via br-ex interface of the hypervisor: >> >> $ ip a show dev br-ex >> 9: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 >> link/ether 0e:82:a1:ba:77:4c brd ff:ff:ff:ff:ff:ff >> inet6 fd12:67:1::1/64 scope global >> valid_lft forever preferred_lft forever >> inet6 fe80::c82:a1ff:feba:774c/64 scope link >> valid_lft forever preferred_lft forever >> >> The hypervisor has the following routes: >> >> $ ip -6 route >> fd12:67:1:1::/64 via fd12:67:1::3c dev br-ex metric 1024 pref medium >> fe80::/64 dev ens3 proto kernel metric 256 pref medium >> fe80::/64 dev br-ex proto kernel metric 256 pref medium >> fe80::/64 dev br-int proto kernel metric 256 pref medium >> fe80::/64 dev tapa5cf4799-9f proto kernel metric 256 pref medium >> >> And within the VM has the following routes: >> >> root@ubuntu:~# ip -6 route >> root@ubuntu:~# ip -6 route >> fd12:67:1::/64 via fd12:67:1:1::1 dev ens3 metric 1024 pref medium >> fd12:67:1:1::/64 dev ens3 proto kernel metric 256 expires 86360sec pref medium >> fe80::/64 dev ens3 proto kernel metric 256 pref medium >> default via fe80::f816:3eff:feb3:bd56 dev ens3 proto ra metric 1024 expires 260sec hoplimit 64 pref medium >> >> Though the ping6 from VM to hypervisor doesn't work: >> root@ubuntu:~# ping6 fd12:67:1::1 -c4 >> PING fd12:67:1::1 (fd12:67:1::1): 56 data bytes >> --- fd12:67:1::1 ping statistics --- >> 4 packets transmitted, 0 packets received, 100% packet loss >> >> I'm able to tcpdump inside the router1 netns and see that request packet is passing there, but can't see any reply packets: >> >> $ sudo ip netns exec qrouter-5172472c-bbe7-4907-832a-e2239c8badb4 tcpdump -l -i any icmp6 >> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode >> listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes >> 21:29:29.351358 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 0, length 64 >> 21:29:30.033316 IP6 fe80::f816:3eff:feb3:bd56 > fe80::f816:3eff:fe0e:17c3: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe0e:17c3, length 32 >> 21:29:30.035807 IP6 fe80::f816:3eff:fe0e:17c3 > fe80::f816:3eff:feb3:bd56: ICMP6, neighbor advertisement, tgt is fe80::f816:3eff:fe0e:17c3, length 24 >> 21:29:30.353646 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 1, length 64 >> 21:29:31.355410 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 2, length 64 >> 21:29:32.357239 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 3, length 64 >> >> The same happens from hypervisor to VM. I only acan see the request packets, but no reply packets. >> >> Thanks in advance, >> Lucio Seki
Hi Antonio. Yes, it is
$ sysctl net.ipv6.conf.all.forwarding net.ipv6.conf.all.forwarding = 1
On Sat, Sep 14, 2019 at 6:02 AM Antonio Ojea antonio.ojea.garcia@gmail.com wrote:
Can you check if ipv6 forwarding is enabled in the router namespace?
net.ipv6.conf.all.forwarding=1
On Sat, 14 Sep 2019 at 02:13, Lucio Seki lucioseki@gmail.com wrote:
I recreated my security group rules, to set remote_ip_prefix to ::/0
instead of None as in Donny's environment, but made no difference. :-(
On Fri, Sep 13, 2019 at 3:55 PM Donny Davis donny@fortnebula.com
wrote:
So outbound traffic works, but inbound traffic doesn't?
Here is my icmp security group rule for ipv6.
+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| created_at | 2019-07-30T00:50:25Z
|| description |
|| direction | ingress
|| ether_type | IPv6
|| id | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa
|| location | Munch({'cloud': '', 'region_name': 'regionOne',
'zone': None, 'project': Munch({'id': 'e8fd161dc34c421a979a9e6421f823e9', 'name': 'openstackzuul', 'domain_id': None, 'domain_name': 'Default'})}) |
| name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | e8fd161dc34c421a979a9e6421f823e9
|| protocol | icmp
|| remote_group_id | None
|| remote_ip_prefix | ::/0
|| revision_number | 0
|| security_group_id | bcedc0e0-e2e8-41fc-aeaa-afd2e10c7ab6
|| tags | []
|| updated_at | 2019-07-30T00:50:25Z
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 2:48 PM Lucio Seki lucioseki@gmail.com wrote:
Hmm OK, I'll try to figure out what hacking
create_neutron_initial_network does...
BTW, I noticed that I can ping6 the router interface at private subnet
from the DevStack host:
$ ping6 fd12:67:1:1::1 PING fd12:67:1:1::1(fd12:67:1:1::1) 56 data bytes 64 bytes from fd12:67:1:1::1: icmp_seq=1 ttl=64 time=0.646 ms 64 bytes from fd12:67:1:1::1: icmp_seq=2 ttl=64 time=0.095 ms 64 bytes from fd12:67:1:1::1: icmp_seq=3 ttl=64 time=0.106 ms 64 bytes from fd12:67:1:1::1: icmp_seq=4 ttl=64 time=0.129 ms
And also I can ping6 the public subnet interface from the VM:
root@ubuntu:~# ping6 fd12:67:1::3c PING fd12:67:1::3c (fd12:67:1::3c): 56 data bytes ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=0 ttl=64 time=2.079 ms ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=1 ttl=64 time=1.385 ms ping: getnameinfo: Temporary failure in name resolution 64 bytes from unknown: icmp_seq=2 ttl=64 time=0.881 ms
Not sure if it means that there's something missing within the router
itself...
On Fri, Sep 13, 2019 at 2:24 PM Donny Davis donny@fortnebula.com
wrote:
Also I have no v6 address on my br-ex
On Fri, Sep 13, 2019 at 1:22 PM Donny Davis donny@fortnebula.com
wrote:
Well here is the output from my rule list that is in prod right now
with ipv6
+--------------------------------------+-------------+-----------+------------+-----------------------+
| ID | IP Protocol | IP Range |
Port Range | Remote Security Group |
+--------------------------------------+-------------+-----------+------------+-----------------------+
| 9ab00b6f-2bc2-4554-818d-eff6e0570943 | None | 0.0.0.0/0 |
| None || b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa | icmp | ::/0 |
| None || e7fd4840-5fbd-4709-b918-f80eac5cb6da | None | ::/0 |
| None || e9968d53-7efe-4a9e-ad42-1092ffaf52e7 | None | None |
| None || ec1ea961-9025-4229-92cf-618026a1851b | None | None |
| None |+--------------------------------------+-------------+-----------+------------+-----------------------+
+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| created_at | 2019-07-30T00:50:25Z
|| description |
|| direction | ingress
|| ether_type | IPv6
|| id | b6df5801-8c2c-4ba4-afe1-2cbaa2922dfa
|| location | Munch({'cloud': '', 'region_name':
'regionOne', 'zone': None, 'project': Munch({'id': 'e8fd161dc34c421a979a9e6421f823e9', 'name': 'openstackzuul', 'domain_id': None, 'domain_name': 'Default'})}) |
| name | None
|| port_range_max | None
|| port_range_min | None
|| project_id | e8fd161dc34c421a979a9e6421f823e9
|| protocol | icmp
|| remote_group_id | None
|| remote_ip_prefix | ::/0
|| revision_number | 0
|| security_group_id | bcedc0e0-e2e8-41fc-aeaa-afd2e10c7ab6
|| tags | []
|| updated_at | 2019-07-30T00:50:25Z
|+-------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
On Fri, Sep 13, 2019 at 9:24 AM Lucio Seki lucioseki@gmail.com
wrote:
> > Hi Donny, following are the rules: > > $ openstack security group list --project admin >
+--------------------------------------+---------+------------------------+----------------------------------+------+
> | ID | Name | Description
| Project | Tags |>
+--------------------------------------+---------+------------------------+----------------------------------+------+
> | d0136b0e-ee51-461c-afa0-c5adb88dd0dd | default | Default security
group | 68e3942285a24fb5bd1aed30e166aaee | [] |
>
+--------------------------------------+---------+------------------------+----------------------------------+------+
> > $ openstack security group rule list
d0136b0e-ee51-461c-afa0-c5adb88dd0dd
>
+--------------------------------------+-------------+----------+------------+--------------------------------------+
> | ID | IP Protocol | IP Range |
Port Range | Remote Security Group |
>
+--------------------------------------+-------------+----------+------------+--------------------------------------+
> | 38394345-3e44-4284-a519-cdd8af020f30 | tcp | ::/0 |
22:22 | None |
> | 40881f76-c87f-4685-b3af-c3497dd44837 | None | None |
| d0136b0e-ee51-461c-afa0-c5adb88dd0dd |> | 56d4ae52-195e-48df-871e-dc70b899b7ba | None | None |
| d0136b0e-ee51-461c-afa0-c5adb88dd0dd |> | 759edd06-b698-45ca-94cd-44e0cc2cc848 | ipv6-icmp | None |
| None |> | 762effae-b8e5-42ac-ba99-e85a7bc42455 | tcp | ::/0 |
22:22 | None |
> | 81f3588d-4159-4af2-ad50-ff6b76add9cf | ipv6-icmp | None |
| None |>
+--------------------------------------+-------------+----------+------------+--------------------------------------+
> > $ openstack security group rule show
759edd06-b698-45ca-94cd-44e0cc2cc848
>
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
> | Field | Value
|>
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
> | created_at | 2019-09-03T16:51:41Z
|> | description |
|> | direction | egress
|> | ether_type | IPv6
|> | id | 759edd06-b698-45ca-94cd-44e0cc2cc848
|> | location | Munch({'project': Munch({'domain_id':
'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) |
> | name | None
|> | port_range_max | None
|> | port_range_min | None
|> | project_id | 68e3942285a24fb5bd1aed30e166aaee
|> | protocol | ipv6-icmp
|> | remote_group_id | None
|> | remote_ip_prefix | None
|> | revision_number | 0
|> | security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
|> | tags | []
|> | updated_at | 2019-09-03T16:51:41Z
|>
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
> > $ openstack security group rule show
81f3588d-4159-4af2-ad50-ff6b76add9cf
>
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
> | Field | Value
|>
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
> | created_at | 2019-09-03T16:51:30Z
|> | description |
|> | direction | ingress
|> | ether_type | IPv6
|> | id | 81f3588d-4159-4af2-ad50-ff6b76add9cf
|> | location | Munch({'project': Munch({'domain_id':
'default', 'id': u'68e3942285a24fb5bd1aed30e166aaee', 'name': 'admin', 'domain_name': None}), 'cloud': '', 'region_name': 'RegionOne', 'zone': None}) |
> | name | None
|> | port_range_max | None
|> | port_range_min | None
|> | project_id | 68e3942285a24fb5bd1aed30e166aaee
|> | protocol | ipv6-icmp
|> | remote_group_id | None
|> | remote_ip_prefix | None
|> | revision_number | 0
|> | security_group_id | d0136b0e-ee51-461c-afa0-c5adb88dd0dd
|> | tags | []
|> | updated_at | 2019-09-03T16:51:30Z
|>
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
> > > On Fri, Sep 13, 2019 at 10:16 AM Donny Davis donny@fortnebula.com
wrote:
>> >> Security group rules? >> >> Donny Davis >> c: 805 814 6800 >> >> On Thu, Sep 12, 2019, 5:53 PM Lucio Seki lucioseki@gmail.com
wrote:
>>> >>> Hi folks, I'm having troubles to ping6 a VM running over DevStack
from its hypervisor.
>>> Could you please help me troubleshooting it? >>> >>> I deployed DevStack with NEUTRON_CREATE_INITIAL_NETWORKS=False, >>> and manually created the networks, subnets and router. Following
is my router:
>>> >>> $ openstack router show router1 -c external_gateway_info -c
interfaces_info
>>>
+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
>>> | Field | Value
|
>>>
+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
>>> | external_gateway_info | {"network_id":
"b87048ed-1be9-4f31-8d7e-fe74921aeec4", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "28a00bc3-b30b-456f-b26a-44b50d37183f", "ip_address": "10.2.0.199"}, {"subnet_id": "a9729beb-b297-4fec-8ec3-7703f7f6f4bc", "ip_address": "fd12:67:1::3c"}]} |
>>> | interfaces_info | [{"subnet_id":
"081e8508-4ceb-4aaf-bf91-36a1e22a768c", "ip_address": "fd12:67:1:1::1", "port_id": "75391abd-8ac8-41f8-acf8-3dfaf2a6b08f"}]
|>>>
+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
>>> >>> I'm trying to ping6 the following VM: >>> >>> $ openstack server list >>>
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+
>>> | ID | Name | Status |
Networks | Image | Flavor |
>>>
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+
>>> | 938854d0-80e9-45b2-bc29-8fe7651ffa93 | manila1 | ACTIVE |
private1=fd12:67:1:1:f816:3eff:fe0e:17c3 | manila | manila |
>>>
+--------------------------------------+---------+--------+------------------------------------------+--------+--------+
>>> >>> I intend to reach it via br-ex interface of the hypervisor: >>> >>> $ ip a show dev br-ex >>> 9: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
noqueue state UNKNOWN group default qlen 1000
>>> link/ether 0e:82:a1:ba:77:4c brd ff:ff:ff:ff:ff:ff >>> inet6 fd12:67:1::1/64 scope global >>> valid_lft forever preferred_lft forever >>> inet6 fe80::c82:a1ff:feba:774c/64 scope link >>> valid_lft forever preferred_lft forever >>> >>> The hypervisor has the following routes: >>> >>> $ ip -6 route >>> fd12:67:1:1::/64 via fd12:67:1::3c dev br-ex metric 1024 pref
medium
>>> fe80::/64 dev ens3 proto kernel metric 256 pref medium >>> fe80::/64 dev br-ex proto kernel metric 256 pref medium >>> fe80::/64 dev br-int proto kernel metric 256 pref medium >>> fe80::/64 dev tapa5cf4799-9f proto kernel metric 256 pref medium >>> >>> And within the VM has the following routes: >>> >>> root@ubuntu:~# ip -6 route >>> root@ubuntu:~# ip -6 route >>> fd12:67:1::/64 via fd12:67:1:1::1 dev ens3 metric 1024 pref medium >>> fd12:67:1:1::/64 dev ens3 proto kernel metric 256 expires
86360sec pref medium
>>> fe80::/64 dev ens3 proto kernel metric 256 pref medium >>> default via fe80::f816:3eff:feb3:bd56 dev ens3 proto ra metric
1024 expires 260sec hoplimit 64 pref medium
>>> >>> Though the ping6 from VM to hypervisor doesn't work: >>> root@ubuntu:~# ping6 fd12:67:1::1 -c4 >>> PING fd12:67:1::1 (fd12:67:1::1): 56 data bytes >>> --- fd12:67:1::1 ping statistics --- >>> 4 packets transmitted, 0 packets received, 100% packet loss >>> >>> I'm able to tcpdump inside the router1 netns and see that request
packet is passing there, but can't see any reply packets:
>>> >>> $ sudo ip netns exec qrouter-5172472c-bbe7-4907-832a-e2239c8badb4
tcpdump -l -i any icmp6
>>> tcpdump: verbose output suppressed, use -v or -vv for full
protocol decode
>>> listening on any, link-type LINUX_SLL (Linux cooked), capture
size 262144 bytes
>>> 21:29:29.351358 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 >
fd12:67:1::1: ICMP6, echo request, seq 0, length 64
>>> 21:29:30.033316 IP6 fe80::f816:3eff:feb3:bd56 >
fe80::f816:3eff:fe0e:17c3: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe0e:17c3, length 32
>>> 21:29:30.035807 IP6 fe80::f816:3eff:fe0e:17c3 >
fe80::f816:3eff:feb3:bd56: ICMP6, neighbor advertisement, tgt is fe80::f816:3eff:fe0e:17c3, length 24
>>> 21:29:30.353646 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 >
fd12:67:1::1: ICMP6, echo request, seq 1, length 64
>>> 21:29:31.355410 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 >
fd12:67:1::1: ICMP6, echo request, seq 2, length 64
>>> 21:29:32.357239 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 >
fd12:67:1::1: ICMP6, echo request, seq 3, length 64
>>> >>> The same happens from hypervisor to VM. I only acan see the
request packets, but no reply packets.
>>> >>> Thanks in advance, >>> Lucio Seki
On 9/12/19 5:49 PM, Lucio Seki wrote:
Hi folks, I'm having troubles to ping6 a VM running over DevStack from its hypervisor. Could you please help me troubleshooting it?
I deployed DevStack with NEUTRON_CREATE_INITIAL_NETWORKS=False,
I think this is your problem. When this is set to True, create_neutron_initial_network() is called, which does a little "hacking" by bringing interfaces up, moving addresses and adding routes so that you can communicate with floating IP and IPv6 addresses. You would have to look at that code and do similar things manually.
-Brian
and manually created the networks, subnets and router. Following is my router:
$ openstack router show router1 -c external_gateway_info -c interfaces_info +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value
| +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | external_gateway_info | {"network_id": "b87048ed-1be9-4f31-8d7e-fe74921aeec4", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "28a00bc3-b30b-456f-b26a-44b50d37183f", "ip_address": "10.2.0.199"}, {"subnet_id": "a9729beb-b297-4fec-8ec3-7703f7f6f4bc", "ip_address": "fd12:67:1::3c"}]} | | interfaces_info | [{"subnet_id": "081e8508-4ceb-4aaf-bf91-36a1e22a768c", "ip_address": "fd12:67:1:1::1", "port_id": "75391abd-8ac8-41f8-acf8-3dfaf2a6b08f"}]
| +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
I'm trying to ping6 the following VM:
$ openstack server list +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | 938854d0-80e9-45b2-bc29-8fe7651ffa93 | manila1 | ACTIVE | private1=fd12:67:1:1:f816:3eff:fe0e:17c3 | manila | manila | +--------------------------------------+---------+--------+------------------------------------------+--------+--------+
I intend to reach it via br-ex interface of the hypervisor:
$ ip a show dev br-ex 9: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 0e:82:a1:ba:77:4c brd ff:ff:ff:ff:ff:ff inet6 fd12:67:1::1/64 scope global valid_lft forever preferred_lft forever inet6 fe80::c82:a1ff:feba:774c/64 scope link valid_lft forever preferred_lft forever
The hypervisor has the following routes:
$ ip -6 route fd12:67:1:1::/64 via fd12:67:1::3c dev br-ex metric 1024 pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium fe80::/64 dev br-ex proto kernel metric 256 pref medium fe80::/64 dev br-int proto kernel metric 256 pref medium fe80::/64 dev tapa5cf4799-9f proto kernel metric 256 pref medium
And within the VM has the following routes:
root@ubuntu:~# ip -6 route root@ubuntu:~# ip -6 route fd12:67:1::/64 via fd12:67:1:1::1 dev ens3 metric 1024 pref medium fd12:67:1:1::/64 dev ens3 proto kernel metric 256 expires 86360sec pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium default via fe80::f816:3eff:feb3:bd56 dev ens3 proto ra metric 1024 expires 260sec hoplimit 64 pref medium
Though the ping6 from VM to hypervisor doesn't work: root@ubuntu:~# ping6 fd12:67:1::1 -c4 PING fd12:67:1::1 (fd12:67:1::1): 56 data bytes --- fd12:67:1::1 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss
I'm able to tcpdump inside the router1 netns and see that request packet is passing there, but can't see any reply packets:
$ sudo ip netns exec qrouter-5172472c-bbe7-4907-832a-e2239c8badb4 tcpdump -l -i any icmp6 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes 21:29:29.351358 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 0, length 64 21:29:30.033316 IP6 fe80::f816:3eff:feb3:bd56 > fe80::f816:3eff:fe0e:17c3: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe0e:17c3, length 32 21:29:30.035807 IP6 fe80::f816:3eff:fe0e:17c3 > fe80::f816:3eff:feb3:bd56: ICMP6, neighbor advertisement, tgt is fe80::f816:3eff:fe0e:17c3, length 24 21:29:30.353646 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 1, length 64 21:29:31.355410 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 2, length 64 21:29:32.357239 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 3, length 64
The same happens from hypervisor to VM. I only acan see the request packets, but no reply packets.
Thanks in advance, Lucio Seki
Hi,
On 13 Sep 2019, at 16:10, Brian Haley haleyb.dev@gmail.com wrote:
On 9/12/19 5:49 PM, Lucio Seki wrote:
Hi folks, I'm having troubles to ping6 a VM running over DevStack from its hypervisor. Could you please help me troubleshooting it? I deployed DevStack with NEUTRON_CREATE_INITIAL_NETWORKS=False,
I think this is your problem. When this is set to True, create_neutron_initial_network() is called, which does a little "hacking" by bringing interfaces up, moving addresses and adding routes so that you can communicate with floating IP and IPv6 addresses. You would have to look at that code and do similar things manually.
I agree with Brian. Probably You need to add IP address from same subnet to br-ex interface that Your floating IPs will be reachable via br-ex. That is the way how this is done by Devstack by default IIRC.
-Brian
and manually created the networks, subnets and router. Following is my router: $ openstack router show router1 -c external_gateway_info -c interfaces_info +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | external_gateway_info | {"network_id": "b87048ed-1be9-4f31-8d7e-fe74921aeec4", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "28a00bc3-b30b-456f-b26a-44b50d37183f", "ip_address": "10.2.0.199"}, {"subnet_id": "a9729beb-b297-4fec-8ec3-7703f7f6f4bc", "ip_address": "fd12:67:1::3c"}]} | | interfaces_info | [{"subnet_id": "081e8508-4ceb-4aaf-bf91-36a1e22a768c", "ip_address": "fd12:67:1:1::1", "port_id": "75391abd-8ac8-41f8-acf8-3dfaf2a6b08f"}] | +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ I'm trying to ping6 the following VM: $ openstack server list +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ | 938854d0-80e9-45b2-bc29-8fe7651ffa93 | manila1 | ACTIVE | private1=fd12:67:1:1:f816:3eff:fe0e:17c3 | manila | manila | +--------------------------------------+---------+--------+------------------------------------------+--------+--------+ I intend to reach it via br-ex interface of the hypervisor: $ ip a show dev br-ex 9: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 0e:82:a1:ba:77:4c brd ff:ff:ff:ff:ff:ff inet6 fd12:67:1::1/64 scope global valid_lft forever preferred_lft forever inet6 fe80::c82:a1ff:feba:774c/64 scope link valid_lft forever preferred_lft forever The hypervisor has the following routes: $ ip -6 route fd12:67:1:1::/64 via fd12:67:1::3c dev br-ex metric 1024 pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium fe80::/64 dev br-ex proto kernel metric 256 pref medium fe80::/64 dev br-int proto kernel metric 256 pref medium fe80::/64 dev tapa5cf4799-9f proto kernel metric 256 pref medium And within the VM has the following routes: root@ubuntu:~# ip -6 route root@ubuntu:~# ip -6 route fd12:67:1::/64 via fd12:67:1:1::1 dev ens3 metric 1024 pref medium fd12:67:1:1::/64 dev ens3 proto kernel metric 256 expires 86360sec pref medium fe80::/64 dev ens3 proto kernel metric 256 pref medium default via fe80::f816:3eff:feb3:bd56 dev ens3 proto ra metric 1024 expires 260sec hoplimit 64 pref medium Though the ping6 from VM to hypervisor doesn't work: root@ubuntu:~# ping6 fd12:67:1::1 -c4 PING fd12:67:1::1 (fd12:67:1::1): 56 data bytes --- fd12:67:1::1 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss I'm able to tcpdump inside the router1 netns and see that request packet is passing there, but can't see any reply packets: $ sudo ip netns exec qrouter-5172472c-bbe7-4907-832a-e2239c8badb4 tcpdump -l -i any icmp6 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes 21:29:29.351358 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 0, length 64 21:29:30.033316 IP6 fe80::f816:3eff:feb3:bd56 > fe80::f816:3eff:fe0e:17c3: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe0e:17c3, length 32 21:29:30.035807 IP6 fe80::f816:3eff:fe0e:17c3 > fe80::f816:3eff:feb3:bd56: ICMP6, neighbor advertisement, tgt is fe80::f816:3eff:fe0e:17c3, length 24 21:29:30.353646 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 1, length 64 21:29:31.355410 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 2, length 64 21:29:32.357239 IP6 fd12:67:1:1:f816:3eff:fe0e:17c3 > fd12:67:1::1: ICMP6, echo request, seq 3, length 64 The same happens from hypervisor to VM. I only acan see the request packets, but no reply packets. Thanks in advance, Lucio Seki
— Slawek Kaplonski Senior software engineer Red Hat
participants (5)
-
Antonio Ojea
-
Brian Haley
-
Donny Davis
-
Lucio Seki
-
Slawek Kaplonski