[openstack-dev] [networking-ovn] Re: [ovs-dev] Issue when using ovn with Openstack

Ryan Moats rmoats at us.ibm.com
Wed Jul 20 04:42:04 UTC 2016


"dev" <dev-bounces at openvswitch.org> wrote on 07/19/2016 10:44:27 PM:

> From: Chen Li <lichen.hangzhou at gmail.com>
> To: dev at openvswitch.org
> Date: 07/19/2016 10:44 PM
> Subject: [ovs-dev] Issue when using ovn with Openstack
> Sent by: "dev" <dev-bounces at openvswitch.org>

>
> Hi list,
>
> I have an all-in-one devstack environment with ovn enabled.
> I create a neutron network.
> Create a port A from the network with secgroup A
> Create a vm from the network with secgroup B.
> Secgroup B has both ICMP  and tcp 22 enabled.
>
> Then I try to ping the VM from the dhcp namespace, since the Secgroup B
has
> enabled ICMP,  I suppose this should work. But, unfortunately,  this do
not
> work. And,  the ssh failed too.
>
> Anyone can help me to solve this issue ?
>
> I did some basic checks and looks like flows are missing in table 52.
>
> Here are flows in table 52:
>
> sudo ovs-ofctl dump-flows br-int |grep table=52
>
>  cookie=0x0, duration=7766.195s, table=52, n_packets=0, n_bytes=0,
> idle_age=7766,
priority=65535,icmp6,metadata=0x4,icmp_type=135,icmp_code=0
> actions=resubmit(,53)
>  cookie=0x0, duration=7766.195s, table=52, n_packets=0, n_bytes=0,
> idle_age=7766,
priority=65535,icmp6,metadata=0x4,icmp_type=136,icmp_code=0
> actions=resubmit(,53)
>  cookie=0x0, duration=7766.195s, table=52, n_packets=4, n_bytes=1474,
> idle_age=7744, priority=2002,udp,reg15=0x2,metadata=0x4,nw_src=
> 192.168.0.0/24,tp_src=67,tp_dst=68
> actions=load:0x1->NXM_NX_REG0[1],resubmit(,53)
>  cookie=0x0, duration=7557.209s, table=52, n_packets=2, n_bytes=759,
> idle_age=7548, priority=2002,udp,reg15=0x3,metadata=0x4,nw_src=
> 192.168.0.0/24,tp_src=67,tp_dst=68
> actions=load:0x1->NXM_NX_REG0[1],resubmit(,53)
>  cookie=0x0, duration=7766.195s, table=52, n_packets=0, n_bytes=0,
> idle_age=7766, priority=2001,ipv6,reg15=0x2,metadata=0x4 actions=drop
>  cookie=0x0, duration=7766.195s, table=52, n_packets=2, n_bytes=676,
> idle_age=7548, priority=2001,ip,reg15=0x2,metadata=0x4 actions=drop
>  cookie=0x0, duration=7557.209s, table=52, n_packets=0, n_bytes=0,
> idle_age=7557, priority=2001,ipv6,reg15=0x3,metadata=0x4 actions=drop
>  cookie=0x0, duration=7557.209s, table=52, n_packets=3979,
n_bytes=389774,
> idle_age=413, priority=2001,ip,reg15=0x3,metadata=0x4 actions=drop
>  cookie=0x0, duration=7766.195s, table=52, n_packets=0, n_bytes=0,
> idle_age=7766, priority=1,ipv6,metadata=0x4
> actions=load:0x1->NXM_NX_REG0[1],resubmit(,53)
>  cookie=0x0, duration=7766.195s, table=52, n_packets=8, n_bytes=2733,
> idle_age=7548, priority=1,ip,metadata=0x4
> actions=load:0x1->NXM_NX_REG0[1],resubmit(,53)
>  cookie=0x0, duration=7926.354s, table=52, n_packets=0, n_bytes=0,
> idle_age=7926, priority=0,metadata=0x1 actions=resubmit(,53)
>  cookie=0x0, duration=7790.771s, table=52, n_packets=129, n_bytes=5418,
> idle_age=408, priority=0,metadata=0x4 actions=resubmit(,53)
>
> Here are steps how I find flows are missing in table52:
>
> ovs-dpctl show
>
>     port 0: ovs-system (internal)
>     port 1: br-int (internal)
>     port 2: tap446ef382-f0 (internal)
>     port 3: tapc7c9f581-2d (internal)  => the dhcp port for the testing
> network
>     port 4: o-hm0 (internal)                => the port create from the
> testing network with security group A
>     port 5: tap275a5a25-79               => the port for the vm in the
> testing network with security group B
>
>
> sudo ip netns exec qdhcp-e8586b01-6441-4c3d-a90d-91bb0a54ec80 arp -n
>
> Address                  HWtype  HWaddress           Flags Mask
> Iface
> 192.168.0.6              ether   fa:16:3e:40:85:41   C
> tapc7c9f581-2d
> 192.168.0.12             ether   fa:16:3e:5c:fe:86   C
> tapc7c9f581-2d
>
>
> sudo ip netns exec qdhcp-e8586b01-6441-4c3d-a90d-91bb0a54ec80 ping
> 192.168.0.12  => This is the IP for the VM.
> PING 192.168.0.12 (192.168.0.12) 56(84) bytes of data.
>
>
> ovs-dpctl dump-flows
>
> recirc_id(0),in_port(3),eth(src=fa:16:3e:b6:f6:25,dst=fa:16:3e:
> 5c:fe:86),eth_type(0x0806),arp(sip=192.168.0.2,tip=192.168.0.
> 12,op=1/0xff,sha=fa:16:3e:b6:f6:25,tha=00:00:00:00:00:00),
> packets:0, bytes:0, used:never,
> actions:userspace(pid=4294958325,slow_path(action))
> recirc_id(0),in_port(3),eth(src=00:00:00:00:00:00/01:00:00:00:00:
> 00,dst=fa:16:3e:5c:fe:86),eth_type(0x0800),ipv4(src=
> 192.168.0.0/255.255.255.0,proto=1,frag=no), packets:14, bytes:1372,
> used:0.974s, actions:drop
>
>
> sudo ovs-appctl ofproto/trace
> "recirc_id(0),in_port(3),eth(src=00:00:00:00:00:00/01:00:00:00:00:
> 00,dst=fa:16:3e:5c:fe:86),eth_type(0x0800),ipv4(src=
> 192.168.0.0/255.255.255.0,proto=1,frag=no)"
> => This produce a long output, and here are the end of the output:
>
> OpenFlow actions=resubmit(,52)
>
> Resubmitted flow: unchanged
>
> Resubmitted regs: reg0=0x1 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0
> reg6=0x0 reg7=0x0 reg8=0x0 reg9=0x0 reg10=0x0 reg11=0x0 reg12=0x0
> reg13=0x0 reg14=0x1 reg15=0x3
>
> Resubmitted  odp: drop
>
> Resubmitted megaflow:
>
recirc_id=0,icmp,reg0=0,reg1=0,reg2=0,reg3=0,reg4=0,reg5=0,reg6=0,reg7=0,reg8=0,reg9=0,reg14=0,reg15=0,metadata=0,in_port=4,vlan_tci=0x0000/

> 0x1000
> ,dl_src=00:00:00:00:00:00/01:00:00:00:00:00,dl_dst=fa:16:3e:5c:fe:86,nw_src=

> 192.168.0.0/24,nw_frag=no
>
> Rule: table=52 cookie=0 priority=2001,ip,reg15=0x3,metadata=0x4
>
> OpenFlow actions=drop
> Final flow:
>
icmp,reg0=0x1,reg14=0x1,reg15=0x3,metadata=0x4,in_port=4,vlan_tci=0x0000,dl_src=00:

> 00:00:00:00:00,dl_dst=fa:16:3e:5c:fe:86,nw_src=192.168.0.0,nw_d
> st=0.0.0.0,nw_tos=0,nw_ecn=0,nw_ttl=0,icmp_type=0,icmp_code=0
> Megaflow:
> recirc_id=0,icmp,in_port=4,vlan_tci=0x0000/0x1000,dl_src=00:00:00:
> 00:00:00/01:00:00:00:00:00,dl_dst=fa:16:3e:5c:fe:86,nw_src=
> 192.168.0.0/24,nw_fra
> g=no
> Datapath actions: drop
>
>
> Here are some output from OVN commands:
>
> sudo ovn-nbctl show
>
>     switch 3ce05ec4-f591-4ca7-ba54-dc4fab2ffd1b
> (neutron-e8586b01-6441-4c3d-a90d-91bb0a54ec80)
>         port 2c713237-ffc7-4ff1-9e4c-95c1337545e6
>             addresses: ["fa:16:3e:40:85:41 192.168.0.6"]
>         port c7c9f581-2db9-4b06-86c6-bde2d1aa8ffb
>             addresses: ["fa:16:3e:b6:f6:25 192.168.0.2"]
>         port 275a5a25-794f-47b9-9b04-8a8da053c143
>             addresses: ["fa:16:3e:5c:fe:86 192.168.0.12"]
>
>
>  ovn-nbctl acl-list 3ce05ec4-f591-4ca7-ba54-dc4fab2ffd1b
>
> from-lport  1002 (inport == "275a5a25-794f-47b9-9b04-8a8da053c143" &&
ip4)
> allow-related
> from-lport  1002 (inport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4
> && (ip4.dst == 255.255.255.255 || ip4.dst == 192.168.0.0/24) && udp &&
> udp.src == 68 && udp.dst == 67) allow
> from-lport  1002 (inport == "275a5a25-794f-47b9-9b04-8a8da053c143" &&
ip6)
> allow-related
> from-lport  1002 (inport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" &&
ip4)
> allow-related
> from-lport  1002 (inport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip4
> && (ip4.dst == 255.255.255.255 || ip4.dst == 192.168.0.0/24) && udp &&
> udp.src == 68 && udp.dst == 67) allow
> from-lport  1002 (inport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" &&
ip6)
> allow-related
> from-lport  1001 (inport == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip)
> drop
> from-lport  1001 (inport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip)
> drop
>   to-lport  1002 (outport == "275a5a25-794f-47b9-9b04-8a8da053c143" &&
ip4
> && icmp4) allow-related
>   to-lport  1002 (outport == "275a5a25-794f-47b9-9b04-8a8da053c143" &&
ip4
> && ip4.src == 192.168.0.0/24 && udp && udp.src == 67 && udp.dst == 68)
allow
>   to-lport  1002 (outport == "275a5a25-794f-47b9-9b04-8a8da053c143" &&
ip4
> && tcp && tcp.dst == 22) allow-related
>   to-lport  1002 (outport == "275a5a25-794f-47b9-9b04-8a8da053c143" &&
ip4
> && tcp && tcp.dst == 9443) allow-related
>   to-lport  1002 (outport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" &&
ip4
> && ip4.src == 192.168.0.0/24 && udp && udp.src == 67 && udp.dst == 68)
allow
>   to-lport  1002 (outport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" &&
ip4
> && udp && udp.dst == 5555) allow-related
>   to-lport  1001 (outport == "275a5a25-794f-47b9-9b04-8a8da053c143" &&
ip)
> drop
>   to-lport  1001 (outport == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" &&
ip)
> drop
>
>
> ovn-sbctl lflow-list | grep ls_out_acl
>
>   table=4 (ls_out_acl         ), priority=0    , match=(1), action=
(next;)
>   table=4 (ls_out_acl         ), priority=0    , match=(1), action=
(next;)
>   table=4 (ls_out_acl         ), priority=65535, match=(!ct.est && ct.rel
> && !ct.new && !ct.inv), action=(next;)
>   table=4 (ls_out_acl         ), priority=65535, match=(ct.est && !ct.rel
> && !ct.new && !ct.inv), action=(next;)
>   table=4 (ls_out_acl         ), priority=65535, match=(ct.inv),
> action=(drop;)
>   table=4 (ls_out_acl         ), priority=65535, match=(nd), action=
(next;)
>   table=4 (ls_out_acl         ), priority=2002 , match=(ct.new &&
(outport
> == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 && icmp4)),
> action=(reg0[1] = 1; next;)
>   table=4 (ls_out_acl         ), priority=2002 , match=(ct.new &&
(outport
> == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 && tcp && tcp.dst ==
22)),
> action=(reg0[1] = 1; next;)
>   table=4 (ls_out_acl         ), priority=2002 , match=(ct.new &&
(outport
> == "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 && tcp && tcp.dst ==
> 9443)), action=(reg0[1] = 1; next;)
>   table=4 (ls_out_acl         ), priority=2002 , match=(ct.new &&
(outport
> == "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip4 && udp && udp.dst ==
> 5555)), action=(reg0[1] = 1; next;)
>   table=4 (ls_out_acl         ), priority=2002 , match=(outport ==
> "275a5a25-794f-47b9-9b04-8a8da053c143" && ip4 && ip4.src ==
192.168.0.0/24
> && udp && udp.src == 67 && udp.dst == 68), action=(reg0[1] = 1; next;)
>   table=4 (ls_out_acl         ), priority=2002 , match=(outport ==
> "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip4 && ip4.src ==
192.168.0.0/24
> && udp && udp.src == 67 && udp.dst == 68), action=(reg0[1] = 1; next;)
>   table=4 (ls_out_acl         ), priority=2001 , match=(outport ==
> "275a5a25-794f-47b9-9b04-8a8da053c143" && ip), action=(drop;)
>   table=4 (ls_out_acl         ), priority=2001 , match=(outport ==
> "2c713237-ffc7-4ff1-9e4c-95c1337545e6" && ip), action=(drop;)
>   table=4 (ls_out_acl         ), priority=1    , match=(ip),
> action=(reg0[1] = 1; next;)
>   table=4 (ls_out_acl         ), priority=0    , match=(1), action=
(next;)
>
>
> The last commit in my ovs code:
>
> commit 7efb1e09bb06270248d29c787978593b57101d4f
> Author: Pravin B Shelar <pshelar at ovn.org>
> Date:   Sun Jul 17 19:24:07 2016 -0700
>
>     datapath: Add support for kernel 4.5
>
>     Signed-off-by: Pravin B Shelar <pshelar at ovn.org>
>     Acked-by: Jesse Gross <jesse at kernel.org>
>
>
> Here are some detailed information from openstack:
>
> 1. The vm:
>
> nova show test01
> +--------------------------------------
> +----------------------------------------------------------------+
> | Property                             |
> Value                                                          |
> +--------------------------------------
> +----------------------------------------------------------------+
> | OS-DCF:diskConfig                    |
> MANUAL                                                         |
> | OS-EXT-AZ:availability_zone          |
> nova                                                           |
> | OS-EXT-SRV-ATTR:host                 | LB-dev-chenli
|
> | OS-EXT-SRV-ATTR:hostname             |
> test01                                                         |
> | OS-EXT-SRV-ATTR:hypervisor_hostname  | LB-dev-chenli
|
> | OS-EXT-SRV-ATTR:instance_name        |
> instance-00000001                                              |
> | OS-EXT-SRV-ATTR:kernel_id            |
> 261ca209-430e-4b8f-ac39-0e397df30a46                           |
> | OS-EXT-SRV-ATTR:launch_index         |
> 0                                                              |
> | OS-EXT-SRV-ATTR:ramdisk_id           |
> 04b6a65d-3cff-4eaf-b30b-582caa2379d7                           |
> | OS-EXT-SRV-ATTR:reservation_id       |
> r-wehkr5gi                                                     |
> | OS-EXT-SRV-ATTR:root_device_name     |
> /dev/vda                                                       |
> | OS-EXT-SRV-ATTR:user_data            |
> -                                                              |
> | OS-EXT-STS:power_state               |
> 1                                                              |
> | OS-EXT-STS:task_state                |
> -                                                              |
> | OS-EXT-STS:vm_state                  |
> active                                                         |
> | OS-SRV-USG:launched_at               |
> 2016-07-20T01:18:48.000000                                     |
> | OS-SRV-USG:terminated_at             |
> -                                                              |
> | accessIPv4
> |                                                                |
> | accessIPv6
> |                                                                |
> | config_drive                         |
> True                                                           |
> | created                              |
> 2016-07-20T01:18:42Z                                           |
> | description                          |
> -                                                              |
> | flavor                               | m1.tiny
> (1)                                                    |
> | hostId                               |
> 36ef28d2b661e38d2d07645d814903a15d62da769828b57029306ec0       |
> | host_status                          |
> UP                                                             |
> | id                                   |
> 27264d62-6a7c-4fe9-be81-c06fca56ec00                           |
> | image                                | cirros-0.3.4-x86_64-uec
> (aa86e8b5-0699-46a0-a624-7af794b21404) |
> | key_name                             |
> -                                                              |
> | lb-mgmt-net network                  |
> 192.168.0.12                                                   |
> | locked                               |
> False                                                          |
> | metadata                             |
> {}                                                             |
> | name                                 |
> test01                                                         |
> | os-extended-volumes:volumes_attached |
> []                                                             |
> | progress                             |
> 0                                                              |
> | security_groups                      |
> lb-mgmt-sec-grp                                                |
> | status                               |
> ACTIVE                                                         |
> | tags                                 |
> []                                                             |
> | tenant_id                            |
> 73aebe8aa8ab41f58d5e375a03e279bf                               |
> | updated                              |
> 2016-07-20T01:18:48Z                                           |
> | user_id                              |
> 53f8c8e491e94d2fa9210f3a8e6a85e4                               |
> +--------------------------------------
> +----------------------------------------------------------------+
>
> 2. the security group:
>
> neutron security-group-show  lb-mgmt-sec-grp
> +----------------------
> +--------------------------------------------------------------------+
> | Field                |
> Value                                                              |
> +----------------------
> +--------------------------------------------------------------------+
> | description
> |                                                                    |
> | id                   |
> fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee                               |
> | name                 |
> lb-mgmt-sec-grp                                                    |
> | security_group_rules |
> {                                                                  |
> |                      |      "remote_group_id":
> null,                                      |
> |                      |      "direction":
> "ingress",                                       |
> |                      |      "protocol":
> "icmp",                                           |
> |                      |      "description":
> "",                                            |
> |                      |      "ethertype":
> "IPv4",                                          |
> |                      |      "remote_ip_prefix":
> null,                                     |
> |                      |      "port_range_max":
> null,                                       |
> |                      |      "security_group_id":
> "fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee",  |
> |                      |      "port_range_min":
> null,                                       |
> |                      |      "tenant_id":
> "73aebe8aa8ab41f58d5e375a03e279bf",              |
> |                      |      "id":
> "140677a5-5308-48b2-a5a2-bb5e17994ed5"                  |
> |                      |
> }                                                                  |
> |                      |
> {                                                                  |
> |                      |      "remote_group_id":
> null,                                      |
> |                      |      "direction":
> "ingress",                                       |
> |                      |      "protocol":
> "tcp",                                            |
> |                      |      "description":
> "",                                            |
> |                      |      "ethertype":
> "IPv4",                                          |
> |                      |      "remote_ip_prefix":
> null,                                     |
> |                      |      "port_range_max":
> 22,                                         |
> |                      |      "security_group_id":
> "fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee",  |
> |                      |      "port_range_min":
> 22,                                         |
> |                      |      "tenant_id":
> "73aebe8aa8ab41f58d5e375a03e279bf",              |
> |                      |      "id":
> "39fccc0c-f832-497a-b03d-fa0e40e3f407"                  |
> |                      |
> }                                                                  |
> |                      |
> {                                                                  |
> |                      |      "remote_group_id":
> null,                                      |
> |                      |      "direction":
> "egress",                                        |
> |                      |      "protocol":
> null,                                             |
> |                      |      "description":
> "",                                            |
> |                      |      "ethertype":
> "IPv6",                                          |
> |                      |      "remote_ip_prefix":
> null,                                     |
> |                      |      "port_range_max":
> null,                                       |
> |                      |      "security_group_id":
> "fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee",  |
> |                      |      "port_range_min":
> null,                                       |
> |                      |      "tenant_id":
> "73aebe8aa8ab41f58d5e375a03e279bf",              |
> |                      |      "id":
> "54d134c0-f4bc-4f3d-bf49-0e1d0ac9ef1c"                  |
> |                      |
> }                                                                  |
> |                      |
> {                                                                  |
> |                      |      "remote_group_id":
> null,                                      |
> |                      |      "direction":
> "ingress",                                       |
> |                      |      "protocol":
> "tcp",                                            |
> |                      |      "description":
> "",                                            |
> |                      |      "ethertype":
> "IPv4",                                          |
> |                      |      "remote_ip_prefix":
> null,                                     |
> |                      |      "port_range_max":
> 9443,                                       |
> |                      |      "security_group_id":
> "fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee",  |
> |                      |      "port_range_min":
> 9443,                                       |
> |                      |      "tenant_id":
> "73aebe8aa8ab41f58d5e375a03e279bf",              |
> |                      |      "id":
> "b3e00b04-d398-450b-b1cf-b92fd3dc37a1"                  |
> |                      |
> }                                                                  |
> |                      |
> {                                                                  |
> |                      |      "remote_group_id":
> null,                                      |
> |                      |      "direction":
> "egress",                                        |
> |                      |      "protocol":
> null,                                             |
> |                      |      "description":
> "",                                            |
> |                      |      "ethertype":
> "IPv4",                                          |
> |                      |      "remote_ip_prefix":
> null,                                     |
> |                      |      "port_range_max":
> null,                                       |
> |                      |      "security_group_id":
> "fbc0c663-f32d-4ddf-9bef-eabac9dfd8ee",  |
> |                      |      "port_range_min":
> null,                                       |
> |                      |      "tenant_id":
> "73aebe8aa8ab41f58d5e375a03e279bf",              |
> |                      |      "id":
> "c528b1cf-b065-4498-986c-13adac4c2a0a"                  |
> |                      |
> }                                                                  |
> | tenant_id            |
> 73aebe8aa8ab41f58d5e375a03e279bf                                   |
> +----------------------
> +--------------------------------------------------------------------+
> _______________________________________________
> dev mailing list
> dev at openvswitch.org
>
http://openvswitch.org/mailman/listinfo/dev

For issues between ovn and openstack, the openstack-dev mailing
list with a tag of [networking-ovn] is another mailing list that
you should try.  I've added the same in the CC of this message to
help.

You've shared a lot of information about the ovv/ovs side of the
situation, but more information on the openstack side would also
help.  Since you say "all-in-one" is this running with devstack?
Is this the master branch or a stable branch of neutron?  What
is the latest commit in the networking-ovn repository?  What
settings are there in the neutron.ini and networking-ovn.ini files?

I've personally seen tip of the tree neutron, networking-ovn,
and ovn work together in multi-node setups, so my initial thought
is to check the code versions or configurations you are using...

Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160719/a3bfe511/attachment-0001.html>


More information about the OpenStack-dev mailing list