[Openstack] OVS/DHCP issues in liberty on ubuntu
Kevin Benton
kevin at benton.pub
Mon Mar 7 17:48:06 UTC 2016
This is because the dhcp agent is not using a veth pair to attach to the
OVS bridge. It's just using a tap interface which is put directly into the
namespace.
If you want it to use a veth pair, you can set 'ovs_use_veth=True' in your
DHCP agent config and you should then see the 'ns' interfaces you were
seeing before. However, if you aren't actually seeing an issue with the
dataplane, I would avoid doing that because we are trying to move away from
unnecessary VETHs due to the extra interfaces they impose and the reduced
performance of VETH pairs.
On Mon, Mar 7, 2016 at 8:13 AM, Sinh Lam <sinh.lam at verizondigitalmedia.com>
wrote:
> Hi Everyone,
>
> I don't even know where to start. Linuxbridge works fine, but i'm going
> to start investing some time into ironic and last I looked through the docs
> it prefers (if not requires) the use of OVS instead of linuxbridge. with
> that said, I am running into a peculiar problem where the tap device for
> the dhcp agent seems to be missing.
>
> when i do an 'ovs-vsctl show' it shows a tap port added to the br-int
> bridge. if i do a 'ovs-ofctl show br-int' it shows that both states as
> "LINK DOWN". in the logs, it shows the tap device as missing. an 'ip
> link' also shows it as missing. i'm use to seeing the dhcp network name
> space use the interface name of "ns-XXXXXX" but in this case, i do not but
> i see the interface name of the tap device. This is better explained with
> a couple of pastes. Included is a copy of my ml2 ini, dhcp-agent and the
> outputs of OVS and relevant parts of syslog.
>
> ml2_conf.ini:
>
> [ml2]
> type_drivers = flat,vlan
> tenant_network_types = vlan,flat
> mechanism_drivers = openvswitch
> # extension_drivers = port_security
>
> [ml2_type_flat]
> flat_networks = provider
>
> [ml2_type_vlan]
> network_vlan_ranges = provider
>
> [securitygroup]
> firewall_driver =
> neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
> enable_security_group = True
> enable_ipset = True
>
> [agent]
> l2_population = True
> log_agent_heartbeats = True
> prevent_arp_spoofing = True
>
> [ovs]
> bridge_mappings = provider:br-eth1
> # use_veth_interconnection = True
>
> dhcp_agent.ini:
>
> [DEFAULT]
> debug = True
> verbose = True
> interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
> dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
> enable_isolated_metadata = True
> dnsmasq_config_file = /etc/neutron/dnsmasq-neutron.conf
> use_namespaces = True
> dhcp_delete_namespaces = True
>
> [AGENT]
> log_agent_heartbeats = True
>
> ovs-vsctl show:
>
> 339c6df7-f22f-41c2-962c-59e8e3da2e15
> Bridge "br-eth1"
> Port "phy-br-eth1"
> Interface "phy-br-eth1"
> type: patch
> options: {peer="int-br-eth1"}
> Port "eth1"
> Interface "eth1"
> Port "br-eth1"
> Interface "br-eth1"
> type: internal
> Bridge br-int
> fail_mode: secure
> Port "int-br-eth1"
> Interface "int-br-eth1"
> type: patch
> options: {peer="phy-br-eth1"}
> Port "tapbbe36037-e7"
> tag: 2
> Interface "tapbbe36037-e7"
> type: internal
> Port br-int
> Interface br-int
> type: internal
> ovs_version: "2.4.0"
>
> ovs-ofctl show:
>
> root at epc265:~# ovs-ofctl show br-int
> OFPT_FEATURES_REPLY (xid=0x2): dpid:00002a2dafef4340
> n_tables:254, n_buffers:256
> capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
> actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src
> mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
> 1(int-br-eth1): addr:aa:06:f4:fa:fa:0c
> config: 0
> state: 0
> speed: 0 Mbps now, 0 Mbps max
> 4(tapbbe36037-e7): addr:00:00:00:00:00:00
> config: PORT_DOWN
> state: LINK_DOWN
> speed: 0 Mbps now, 0 Mbps max
> LOCAL(br-int): addr:2a:2d:af:ef:43:40
> config: 0
> state: 0
> speed: 0 Mbps now, 0 Mbps max
> OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
>
> root at epc265:~# ovs-ofctl show br-eth1
> OFPT_FEATURES_REPLY (xid=0x2): dpid:0000ecf4bbde6cda
> n_tables:254, n_buffers:256
> capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
> actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src
> mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
> 1(eth1): addr:ec:f4:bb:de:6c:da
> config: 0
> state: 0
> current: 10GB-FD
> advertised: 10GB-FD FIBER
> supported: 10GB-FD FIBER
> speed: 10000 Mbps now, 10000 Mbps max
> 2(phy-br-eth1): addr:36:98:ab:e4:e0:6f
> config: 0
> state: 0
> speed: 0 Mbps now, 0 Mbps max
> LOCAL(br-eth1): addr:ec:f4:bb:de:6c:da
> config: 0
> state: 0
> speed: 0 Mbps now, 0 Mbps max
> OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
>
> snippets of logs:
>
> syslog:
>
> Mar 7 15:59:33 epc265 dnsmasq[36861]: started, version 2.68 cachesize 150
> Mar 7 15:59:33 epc265 dnsmasq[36861]: compile time options: IPv6
> GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth
> Mar 7 15:59:33 epc265 dnsmasq[36861]: warning: no upstream servers
> configured
> Mar 7 15:59:33 epc265 dnsmasq-dhcp[36861]: DHCP, static leases only on
> 10.81.20.0, lease time 1d
> Mar 7 15:59:33 epc265 dnsmasq-dhcp[36861]: DHCP, sockets bound
> exclusively to interface tapbbe36037-e7
> Mar 7 15:59:33 epc265 dnsmasq[36861]: read
> /var/lib/neutron/dhcp/70fb10ff-6644-4bd0-87ed-ae6970279cbd/addn_hosts - 3
> addresses
> Mar 7 15:59:33 epc265 dnsmasq-dhcp[36861]: read
> /var/lib/neutron/dhcp/70fb10ff-6644-4bd0-87ed-ae6970279cbd/host
> Mar 7 15:59:33 epc265 dnsmasq-dhcp[36861]: read
> /var/lib/neutron/dhcp/70fb10ff-6644-4bd0-87ed-ae6970279cbd/opts
> Mar 7 15:59:34 epc265 dnsmasq[36891]: started, version 2.68 cachesize 150
> Mar 7 15:59:34 epc265 dnsmasq[36891]: compile time options: IPv6
> GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth
> Mar 7 15:59:34 epc265 dnsmasq[36891]: warning: no upstream servers
> configured
> Mar 7 15:59:34 epc265 dnsmasq-dhcp[36891]: DHCP, static leases only on
> 10.81.20.0, lease time 1d
> Mar 7 15:59:34 epc265 dnsmasq-dhcp[36891]: DHCP, sockets bound
> exclusively to interface tapbbe36037-e7
> Mar 7 15:59:34 epc265 dnsmasq[36891]: read
> /var/lib/neutron/dhcp/70fb10ff-6644-4bd0-87ed-ae6970279cbd/addn_hosts - 3
> addresses
> Mar 7 15:59:34 epc265 dnsmasq-dhcp[36891]: read
> /var/lib/neutron/dhcp/70fb10ff-6644-4bd0-87ed-ae6970279cbd/host
> Mar 7 15:59:34 epc265 dnsmasq-dhcp[36891]: read
> /var/lib/neutron/dhcp/70fb10ff-6644-4bd0-87ed-ae6970279cbd/opts
> Mar 7 15:59:34 epc265 dnsmasq[36891]: read
> /var/lib/neutron/dhcp/70fb10ff-6644-4bd0-87ed-ae6970279cbd/addn_hosts - 3
> addresses
> Mar 7 15:59:34 epc265 dnsmasq-dhcp[36891]: read
> /var/lib/neutron/dhcp/70fb10ff-6644-4bd0-87ed-ae6970279cbd/host
> Mar 7 15:59:34 epc265 dnsmasq-dhcp[36891]: read
> /var/lib/neutron/dhcp/70fb10ff-6644-4bd0-87ed-ae6970279cbd/opts
> Mar 7 15:59:34 epc265 dnsmasq-dhcp[36891]: DHCPDISCOVER(tapbbe36037-e7)
> fa:16:3e:21:90:ce no address available
> Mar 7 15:59:37 epc265 dnsmasq-dhcp[36891]: DHCPDISCOVER(tapbbe36037-e7)
> fa:16:3e:21:90:ce no address available
> Mar 7 16:00:00 epc265 dnsmasq-dhcp[36891]: DHCPDISCOVER(tapbbe36037-e7)
> fa:16:3e:a5:a6:da
> Mar 7 16:00:00 epc265 dnsmasq-dhcp[36891]: DHCPOFFER(tapbbe36037-e7)
> 10.81.20.12 fa:16:3e:a5:a6:da
> Mar 7 16:00:00 epc265 dnsmasq-dhcp[36891]: DHCPDISCOVER(tapbbe36037-e7)
> fa:16:3e:21:90:ce no address available
>
> ovs-vswitchd.log (please note the tap device specified in this log is
> different than what is shown above, but the message is the same)
>
> 2016-03-02T03:58:24.346Z|00105|bridge|INFO|bridge br-int: added interface
> tap22aa83ec-86 on port 2
> 2016-03-02T03:58:24.733Z|00106|netdev_linux|INFO|ioctl(SIOCGIFHWADDR) on
> tap22aa83ec-86 device failed: No such device
> 2016-03-02T03:58:24.736Z|00107|netdev_linux|WARN|ioctl(SIOCGIFINDEX) on
> tap22aa83ec-86 device failed: No such device
> 2016-03-02T03:58:25.020Z|00108|connmgr|INFO|br-eth1<->unix: 1 flow_mods in
> the last 0 s (1 adds)
> 2016-03-02T03:58:25.056Z|00109|connmgr|INFO|br-int<->unix: 1 flow_mods in
> the last 0 s (1 adds)
> 2016-03-02T03:58:25.126Z|00110|netdev_linux|WARN|tap22aa83ec-86: removing
> policing failed: No such device
> 2016-03-02T03:58:25.194Z|00111|connmgr|INFO|br-int<->unix: 1 flow_mods in
> the last 0 s (1 deletes)
> 2016-03-02T03:58:25.225Z|00112|connmgr|INFO|br-int<->unix: 1 flow_mods in
> the last 0 s (1 deletes)
>
> ip netns:
>
> root at epc265:/var/log/openvswitch# ip netns list
> qdhcp-70fb10ff-6644-4bd0-87ed-ae6970279cbd
> root at epc265:/var/log/openvswitch# ip netns exec $(ip netns) ip a
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group
> default
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> inet 127.0.0.1/8 scope host lo
> valid_lft forever preferred_lft forever
> inet6 ::1/128 scope host
> valid_lft forever preferred_lft forever
> 55: tapbbe36037-e7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
> noqueue state UNKNOWN group default
> link/ether fa:16:3e:44:19:0d brd ff:ff:ff:ff:ff:ff
> inet 10.81.20.11/24 brd 10.81.20.255 scope global tapbbe36037-e7
> valid_lft forever preferred_lft forever
> inet 169.254.169.254/16 brd 169.254.255.255 scope global
> tapbbe36037-e7
> valid_lft forever preferred_lft forever
> inet6 fe80::f816:3eff:fe44:190d/64 scope link
> valid_lft forever preferred_ft forever
>
> root at epc265:/var/log/openvswitch# ip netns exec $(ip netns) ping 4.2.2.1
> PING 4.2.2.1 (4.2.2.1) 56(84) bytes of data.
> 64 bytes from 4.2.2.1: icmp_seq=1 ttl=56 time=1.18 ms
> 64 bytes from 4.2.2.1: icmp_seq=2 ttl=56 time=1.05 ms
> ^C
> --- 4.2.2.1 ping statistics ---
> 2 packets transmitted, 2 received, 0% packet loss, time 1001ms
> rtt min/avg/max/mdev = 1.051/1.118/1.185/0.067 ms
> root at epc265:/var/log/openvswitch# ip netns exec $(ip netns) ping
> 10.81.20.1
> PING 10.81.20.1 (10.81.20.1) 56(84) bytes of data.
> 64 bytes from 10.81.20.1: icmp_seq=1 ttl=64 time=158 ms
> 64 bytes from 10.81.20.1: icmp_seq=2 ttl=64 time=12.2 ms
> 64 bytes from 10.81.20.1: icmp_seq=3 ttl=64 time=12.5 ms
> 64 bytes from 10.81.20.1: icmp_seq=4 ttl=64 time=109 ms
> 64 bytes from 10.81.20.1: icmp_seq=5 ttl=64 time=62.3 ms
> ^C
> --- 10.81.20.1 ping statistics ---
> 5 packets transmitted, 5 received, 0% packet loss, time 4005ms
> rtt min/avg/max/mdev = 12.290/70.945/158.270/56.603 ms
>
> (not too sure why the RTT is oddly high going to the default gateway but
> at the very least it's returning something)
>
> so. . there you have it. . any information will be much appreciated to
> help me resolve this issue will be much appreciated.
>
> Thanks.
>
> Sinh
>
>
> _______________________________________________
> Mailing list:
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack at lists.openstack.org
> Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20160307/81b68741/attachment.html>
More information about the Openstack
mailing list