[Openstack-operators] VXLAN / Tenant Network Issue
James Denton
james.denton at rackspace.com
Thu Sep 8 20:02:17 UTC 2016
Hi Grant,
You should have an IP address configured on the br-vxlan interface of each host.
- Can you confirm connectivity between the hosts using the IP addresses assigned to br-vxlan? Ping from within the neutron_agents container, as well, since this is ultimately where the Neutron namespace that handles DHCP lives.
- If the ping fails, can you confirm bond1.21 is UP? How about br-vxlan? Is the VLAN trunked properly?
Once connectivity is confirmed, try performing packet captures on vxlan-21 and bond1.21 while you boot the instance and see what’s there. Report back and we’ll go from there.
James
From: Grant Morley <grant at absolutedevops.io>
Date: Thursday, September 8, 2016 at 9:55 AM
To: Vahric Muhtaryan <vahric at doruk.net.tr>, OpenStack Operators <openstack-operators at lists.openstack.org>
Cc: "ian.banks at serverchoice.com" <ian.banks at serverchoice.com>
Subject: Re: [Openstack-operators] VXLAN / Tenant Network Issue
Hi there,
thanks for replying, configs below:
The following are from the neutron agents container.
# Ansible managed: /opt/openstack-ansible/playbooks/roles/os_neutron/templates/plugins/ml2/ml2_conf.ini.j2
# ML2 general
[ml2]
type_drivers = flat,vlan,vxlan,local
tenant_network_types = vxlan,vlan,flat
mechanism_drivers = linuxbridge,l2population
extension_drivers = port_security
path_mtu = 0
segment_mtu = 0
# ML2 flat networks
[ml2_type_flat]
flat_networks = flat
# ML2 VLAN networks
[ml2_type_vlan]
network_vlan_ranges = vlan:101:200,vlan:301:400
# ML2 VXLAN networks
[ml2_type_vxlan]
vxlan_group = 239.1.1.1
vni_ranges = 1:1000
# Security groups
[securitygroup]
enable_security_group = True
enable_ipset = True
--------------------------------------------------------------------
# Ansible managed: /opt/openstack-ansible/playbooks/roles/os_neutron/templates/dhcp_agent.ini.j2
# General
[DEFAULT]
verbose = True
debug = False
num_sync_threads = 6
# Drivers
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver
dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
# Default domain for DHCP leases
dhcp_domain = openstacklocal
# Dnsmasq options
dnsmasq_config_file = /etc/neutron/dnsmasq-neutron.conf
dnsmasq_dns_servers =
dnsmasq_lease_max = 16777216
# Metadata
enable_isolated_metadata = True
---------------------------------------------------------------------
# Ansible managed: /opt/openstack-ansible/playbooks/roles/os_neutron/templates/l3_agent.ini.j2
# General
[DEFAULT]
verbose = True
debug = False
# While this option is deprecated in Liberty, if we remove it then it takes
# a default value of 'br-ex', which we do not want. We therefore leave it
# in place for now and can remove it in Mitaka.
external_network_bridge =
gateway_external_network_id =
# Drivers
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver
# Agent mode (legacy only)
agent_mode = legacy
# Conventional failover
allow_automatic_l3agent_failover = True
# HA failover
ha_confs_path = /var/lib/neutron/ha_confs
ha_vrrp_advert_int = 2
ha_vrrp_auth_password = bee916a2589b14dd7f
ha_vrrp_auth_type = PASS
handle_internal_only_routers = False
send_arp_for_ha = 3
# Metadata
enable_metadata_proxy = True
Regards,
On 08/09/16 13:51, Vahric Muhtaryan wrote:
Hello Grant ,
Possible to share ml2_conf.ini , dhcp_agent.ini and l3_agent.ini files ?
Regards
VM
From: Grant Morley <grant at absolutedevops.io<mailto:grant at absolutedevops.io>>
Date: Thursday 8 September 2016 at 15:12
To: OpenStack Operators <openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>>
Cc: <ian.banks at serverchoice.com<mailto:ian.banks at serverchoice.com>>
Subject: [Openstack-operators] VXLAN / Tenant Network Issue
Hi All,
We are working off the OSA deployment for a new cloud system we are building and everything seems to be working apart from the tenant VXLAN network. We have tried various troubleshooting but the initial DHCP request, is not making it out of the linux bridge on the compute node. We have checked all physical networking and switch setup and they appear to be fine.
Below is an output of related networking components that we have configured. (Sorry for the long post but wanted to get as much info on here) Can anyone see what might be causing the issue or where we have gone wrong?
Neutron subnet and router:
neutron) net-list
+--------------------------------------+----------------------------------------------------+----------------------------------------------------+
| id | name | subnets |
+--------------------------------------+----------------------------------------------------+----------------------------------------------------+
| b1da0a4f-2d06-46af-92aa-962c7a7c36f9 | ext-net | 405f439c-51bb-40b6-820a-9048c2ee69fe |
| | | 185.136.232.0/22 |
| a256ccb2-273a-4738-97ab-bd8bfbc2a2cc | HA network tenant 7b5aad6af3ee450ea60e06aaaba2da50 | 6d98faac-2e3b-43c8-bcd6-f9a6f5dcc45e |
| | | 169.254.192.0/18 |
| f88ceab1-a392-4281-8c60-f57d171a8029 | vxlan-172 | 367e88eb-b09f-4ce5-bfff-5d9e0b0e14b0
| 172.16.0.0/24
+--------------------------------------+----------------------------------------------------+----------------------------------------------------+
(neutron) net-show f88ceab1-a392-4281-8c60-f57d171a8029
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| id | f88ceab1-a392-4281-8c60-f57d171a8029 |
| mtu | 0 |
| name | vxlan-172 |
| port_security_enabled | True |
| provider:network_type | vxlan |
| provider:physical_network | |
| provider:segmentation_id | 21 |
| router:external | False |
| shared | False |
| status | ACTIVE |
| subnets | 367e88eb-b09f-4ce5-bfff-5d9e0b0e14b0 |
| tenant_id | 7b5aad6af3ee450ea60e06aaaba2da50 |
+---------------------------+--------------------------------------+
(neutron) router-show f31ed1fb-1b90-46e3-b869-d9374e3d08b1
+-----------------------+------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | True |
| distributed | False |
| external_gateway_info | {"network_id": "b1da0a4f-2d06-46af-92aa-962c7a7c36f9", "enable_snat": true, "external_fixed_ips": [{"subnet_id": |
| | "405f439c-51bb-40b6-820a-9048c2ee69fe", "ip_address": "185.136.232.55"}]} |
| ha | True |
| id | f31ed1fb-1b90-46e3-b869-d9374e3d08b1 |
| name | ext-router |
| routes | |
| status | ACTIVE |
| tenant_id | 7b5aad6af3ee450ea60e06aaaba2da50 |
+-----------------------+------------------------------------------------------------------------------------------------------------------------+
(neutron) router-port-list f31ed1fb-1b90-46e3-b869-d9374e3d08b1
+--------------------------------------+----------------------------------------+-------------------+------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+----------------------------------------+-------------------+------------------------------------------+
| 443d8a0e-833e-4dd2-9320-c2a361e97bf0 | HA port tenant | fa:16:3e:db:48:be | {"subnet_id": "6d98faac-2e3b- |
| | 7b5aad6af3ee450ea60e06aaaba2da50 | | 43c8-bcd6-f9a6f5dcc45e", "ip_address": |
| | | | "169.254.192.2"} |
| 58312691-77d1-408a-adf2-8c74bb87d35d | HA port tenant | fa:16:3e:26:86:3c | {"subnet_id": "6d98faac-2e3b- |
| | 7b5aad6af3ee450ea60e06aaaba2da50 | | 43c8-bcd6-f9a6f5dcc45e", "ip_address": |
| | | | "169.254.192.1"} |
| 8182e8ca-0e3d-444a-ac4f-f424027aa373 | | fa:16:3e:20:1c:08 | {"subnet_id": "405f439c-51bb-40b6-820a- |
| | | | 9048c2ee69fe", "ip_address": |
| | | | "185.136.232.55"} |
| beaa905d-fc68-46ba-9fd3-9f620584a1f7 | | fa:16:3e:5a:8e:c0 | {"subnet_id": "367e88eb-b09f-4ce5-bfff- |
| | | | 5d9e0b0e14b0", "ip_address": |
| | | | "172.16.0.254"} |
+--------------------------------------+----------------------------------------+-------------------+------------------------------------------+
The bridge and interface for the instance:
root at compute-2:~# brctl show
bridge name bridge id STP enabled interfaces
br-mgmt 8000.1418775ed1bc no bond0.11
br-storage 8000.1418775ed1bc no bond0.31
br-vlan 8000.1418775ed1be no bond1
br-vxlan 8000.1418775ed1be no bond1.21
brqf88ceab1-a3 8000.0a81d25d36ce no tapf9871920-e0
vxlan-21
Network agent node namespaces:
root at network-1_neutron_agents_container-f3caf6a1:~# ip netns
qrouter-f31ed1fb-1b90-46e3-b869-d9374e3d08b1
qdhcp-f88ceab1-a392-4281-8c60-f57d171a8029
qdhcp-b1da0a4f-2d06-46af-92aa-962c7a7c36f9
The two qdhcp namespaces are able to ping to each other.
When booting the instance the DHCP request can be seen:
root at compute-2:~# dhcpdump -i tapf9871920-e0
TIME: 2016-09-08 11:49:03.646
IP: 0.0.0.0 (fa:16:3e:32:7e:79) > 255.255.255.255 (ff:ff:ff:ff:ff:ff)
OP: 1 (BOOTPREQUEST)
HTYPE: 1 (Ethernet)
HLEN: 6
HOPS: 0
XID: 7840761a
SECS: 60
FLAGS: 0
CIADDR: 0.0.0.0
YIADDR: 0.0.0.0
SIADDR: 0.0.0.0
GIADDR: 0.0.0.0
CHADDR: fa:16:3e:32:7e:79:00:00:00:00:00:00:00:00:00:00
SNAME: .
FNAME: .
OPTION: 53 ( 1) DHCP message type 1 (DHCPDISCOVER)
OPTION: 61 ( 7) Client-identifier 01:fa:16:3e:32:7e:79
OPTION: 57 ( 2) Maximum DHCP message size 576
OPTION: 55 ( 9) Parameter Request List 1 (Subnet mask)
3 (Routers)
6 (DNS server)
12 (Host name)
15 (Domainname)
26 (Interface MTU)
28 (Broadcast address)
42 (NTP servers)
121 (Classless Static Route)
OPTION: 60 ( 12) Vendor class identifier udhcp 1.20.1
OPTION: 12 ( 6) Host name cirros
---------------------------------------------------------------------------
The DHCP packet is seen on the tap interface for the instance and the bridge brqf88ceab1-a3, but not on any other interface on the compute host. No DHCP packet is observed on the network agent container running the DHCP namespace.
output of the instance booting:
Starting network...
udhcpc (v1.20.1) started
Sending discover...
Sending discover...
Sending discover...
Usage: /sbin/cirros-dhcpc <up|down>
No lease, failing
WARN: /etc/rc3.d/S40-network failed
cirros-ds 'net' up at 181.24
Regards,
--
[cid:part1.44430905.A1A46B65 at absolutedevops.io]
Grant Morley
Cloud Lead
Absolute DevOps Ltd
Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP
www.absolutedevops.io<http://www.absolutedevops.io/> grant at absolutedevops.io<mailto:grant at absolutedevops.i> 0845 874 0580
_______________________________________________ OpenStack-operators mailing list OpenStack-operators at lists.openstack.org<mailto:OpenStack-operators at lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
--
[cid:image001.png at 01D209E1.FCB1D490]
Grant Morley
Cloud Lead
Absolute DevOps Ltd
Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP
www.absolutedevops.io<http://www.absolutedevops.io/> grant at absolutedevops.io<mailto:grant at absolutedevops.i> 0845 874 0580
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20160908/11066422/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 4370 bytes
Desc: image001.png
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20160908/11066422/attachment-0001.png>
More information about the OpenStack-operators
mailing list
