[Openstack-operators] VXLAN / Tenant Network Issue

Ian Banks Ian.Banks at serverchoice.com
Mon Sep 19 20:48:04 UTC 2016


Hi James,

Unfortunately, we are still experiencing the trouble with DHCP packets not reaching the network agent containers which are running the DHCP namespace. tcpdumps to illustrate this are below. Are you able to advise how we can proceed with troubleshooting this issue? We are currently a bit stumped.

Many thanks for any advice you can give us.

Regards,

Ian Banks

Senior Systems Engineer

ServerChoice Ltd

Unit H, J and K
Gateway 1000
Whittle Way
Stevenage
Hertfordshire
SG1 2FP

Please note we have new telephone numbers.

Direct:  01438 532 307
Office: 01438 532 300
Support: 01438 342 310

http://www.serverchoice.com
ian.banks at serverchoice.com



From: Ian Banks
Sent: 13 September 2016 12:29
To: 'James Denton' <james.denton at rackspace.com>; Grant Morley <grant at absolutedevops.io>; Vahric Muhtaryan <vahric at doruk.net.tr>; OpenStack Operators <openstack-operators at lists.openstack.org>
Subject: RE: [Openstack-operators] VXLAN / Tenant Network Issue

Hi,

Apologies for the delay. Grant will not be available until tomorrow, but I am working on the project also.

The br-vxlan connectivity appears to be fine. The br-vxlan subnet is 10.6.4.0/24. Ping tests between a compute host and neutron agents container are as follows:

-------COMPUTE HOST------

root at compute-2:~# ifconfig br-vxlan
br-vxlan  Link encap:Ethernet  HWaddr 14:18:77:5e:d1:be
          inet addr:10.6.4.9  Bcast:10.6.4.255  Mask:255.255.255.0
          inet6 addr: fe80::1618:77ff:fe5e:d1be/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
          RX packets:6681 errors:0 dropped:0 overruns:0 frame:0
          TX packets:610 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2426368 (2.4 MB)  TX bytes:92158 (92.1 KB)

root at compute-2:~# ping 10.6.4.154 -c 3
PING 10.6.4.154 (10.6.4.154) 56(84) bytes of data.
64 bytes from 10.6.4.154: icmp_seq=1 ttl=64 time=0.203 ms
64 bytes from 10.6.4.154: icmp_seq=2 ttl=64 time=0.130 ms
64 bytes from 10.6.4.154: icmp_seq=3 ttl=64 time=0.157 ms

--- 10.6.4.154 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.130/0.163/0.203/0.031 ms


------NEUTRON AGENTS CONTAINER------

root at network-1_neutron_agents_container-f3caf6a1:~# ifconfig eth10
eth10     Link encap:Ethernet  HWaddr 00:16:3e:a7:0f:54
          inet addr:10.6.4.154  Bcast:10.6.4.255  Mask:255.255.255.0
          inet6 addr: fe80::216:3eff:fea7:f54/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
          RX packets:7076 errors:0 dropped:0 overruns:0 frame:0
          TX packets:32 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2591840 (2.5 MB)  TX bytes:2720 (2.7 KB)

root at network-1_neutron_agents_container-f3caf6a1:~# ping 10.6.4.9 -c3
PING 10.6.4.9 (10.6.4.9) 56(84) bytes of data.
64 bytes from 10.6.4.9: icmp_seq=1 ttl=64 time=0.170 ms
64 bytes from 10.6.4.9: icmp_seq=2 ttl=64 time=0.162 ms
64 bytes from 10.6.4.9: icmp_seq=3 ttl=64 time=0.139 ms

--- 10.6.4.9 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.139/0.157/0.170/0.013 ms

I have run some tcpdumps on the compute host and the neutron agents containers and can see DHCP packets on the compute host  bond1.21 and br-vxlan interfaces, but nothing at all on the neutron agents container vxlan-21 interface:

-------COMPUTE HOST------

root at compute-2:~# tcpdump -i br-vxlan
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-vxlan, link-type EN10MB (Ethernet), capture size 65535 bytes
10:54:42.404618 IP 10.6.4.9.56068 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP6 :: > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28
10:54:42.444596 IP 10.6.4.9.34237 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:32:7e:79 (oui Unknown), length 290
10:54:42.572587 IP 10.6.4.9.56068 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP6 :: > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28
10:54:42.972672 IP 10.6.4.9.52750 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP6 :: > ff02::1:ff32:7e79: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe32:7e79, length 24
10:54:43.972694 IP 10.6.4.9.51617 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP6 fe80::f816:3eff:fe32:7e79 > ip6-allrouters: ICMP6, router solicitation, length 16
10:54:47.980665 IP 10.6.4.9.51617 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP6 fe80::f816:3eff:fe32:7e79 > ip6-allrouters: ICMP6, router solicitation, length 16
10:54:51.988714 IP 10.6.4.9.51617 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP6 fe80::f816:3eff:fe32:7e79 > ip6-allrouters: ICMP6, router solicitation, length 16
10:55:42.505243 IP 10.6.4.9.34237 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:32:7e:79 (oui Unknown), length 290
10:56:42.572668 IP 10.6.4.9.34237 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:32:7e:79 (oui Unknown), length 290

root at compute-2:~# tcpdump -i bond1.21
tcpdump: WARNING: bond1.21: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on bond1.21, link-type EN10MB (Ethernet), capture size 65535 bytes
10:59:58.613760 IP 10.6.4.9.56068 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP6 :: > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28
10:59:58.645796 IP 10.6.4.9.34237 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:32:7e:79 (oui Unknown), length 290
10:59:59.405808 IP 10.6.4.9.52750 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP6 :: > ff02::1:ff32:7e79: ICMP6, neighbor solicitation, who has fe80::f816:3eff:fe32:7e79, length 24
11:00:00.405791 IP 10.6.4.9.51617 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP6 fe80::f816:3eff:fe32:7e79 > ip6-allrouters: ICMP6, router solicitation, length 16
11:00:04.417873 IP 10.6.4.9.51617 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP6 fe80::f816:3eff:fe32:7e79 > ip6-allrouters: ICMP6, router solicitation, length 16
11:00:08.225878 IP 10.6.4.9.56455 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP6 fe80::f816:3eff:fe32:7e79 > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28
11:00:08.425723 IP 10.6.4.9.51617 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP6 fe80::f816:3eff:fe32:7e79 > ip6-allrouters: ICMP6, router solicitation, length 16
11:00:58.729817 IP 10.6.4.9.34237 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:32:7e:79 (oui Unknown), length 290
11:01:58.790420 IP 10.6.4.9.34237 > 239.1.1.1.8472: OTV, flags [I] (0x08), overlay 0, instance 21
IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:32:7e:79 (oui Unknown), length 290


------NEUTRON AGENTS CONTAINER------

root at network-2_neutron_agents_container-3f69b7a0:~# tcpdump -i vxlan-21
tcpdump: WARNING: vxlan-21: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vxlan-21, link-type EN10MB (Ethernet), capture size 65535 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel


Regards,

Ian Banks

Senior Systems Engineer

ServerChoice Ltd

Unit H, J and K
Gateway 1000
Whittle Way
Stevenage
Hertfordshire
SG1 2FP

Please note we have new telephone numbers.

Direct:  01438 532 307
Office: 01438 532 300
Support: 01438 342 310

http://www.serverchoice.com
ian.banks at serverchoice.com<mailto:ian.banks at serverchoice.com>


From: James Denton [mailto:james.denton at rackspace.com]
Sent: 08 September 2016 21:02
To: Grant Morley <grant at absolutedevops.io<mailto:grant at absolutedevops.io>>; Vahric Muhtaryan <vahric at doruk.net.tr<mailto:vahric at doruk.net.tr>>; OpenStack Operators <openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>>
Cc: Ian Banks <Ian.Banks at serverchoice.com<mailto:Ian.Banks at serverchoice.com>>
Subject: Re: [Openstack-operators] VXLAN / Tenant Network Issue

Hi Grant,

You should have an IP address configured on the br-vxlan interface of each host.

-          Can you confirm connectivity between the hosts using the IP addresses assigned to br-vxlan? Ping from within the neutron_agents container, as well, since this is ultimately where the Neutron namespace that handles DHCP lives.

-          If the ping fails, can you confirm bond1.21 is UP? How about br-vxlan? Is the VLAN trunked properly?

Once connectivity is confirmed, try performing packet captures on vxlan-21 and bond1.21 while you boot the instance and see what’s there. Report back and we’ll go from there.

James

From: Grant Morley <grant at absolutedevops.io<mailto:grant at absolutedevops.io>>
Date: Thursday, September 8, 2016 at 9:55 AM
To: Vahric Muhtaryan <vahric at doruk.net.tr<mailto:vahric at doruk.net.tr>>, OpenStack Operators <openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>>
Cc: "ian.banks at serverchoice.com<mailto:ian.banks at serverchoice.com>" <ian.banks at serverchoice.com<mailto:ian.banks at serverchoice.com>>
Subject: Re: [Openstack-operators] VXLAN / Tenant Network Issue


Hi there,

thanks for replying, configs below:
The following are from the neutron agents container.
# Ansible managed: /opt/openstack-ansible/playbooks/roles/os_neutron/templates/plugins/ml2/ml2_conf.ini.j2
# ML2 general
[ml2]
type_drivers = flat,vlan,vxlan,local
tenant_network_types = vxlan,vlan,flat
mechanism_drivers = linuxbridge,l2population
extension_drivers = port_security
path_mtu = 0
segment_mtu = 0
# ML2 flat networks
[ml2_type_flat]
flat_networks = flat

# ML2 VLAN networks
[ml2_type_vlan]
network_vlan_ranges = vlan:101:200,vlan:301:400
# ML2 VXLAN networks
[ml2_type_vxlan]
vxlan_group = 239.1.1.1
vni_ranges = 1:1000
# Security groups
[securitygroup]
enable_security_group = True
enable_ipset = True

--------------------------------------------------------------------
# Ansible managed: /opt/openstack-ansible/playbooks/roles/os_neutron/templates/dhcp_agent.ini.j2
# General
[DEFAULT]
verbose = True
debug = False
num_sync_threads = 6
# Drivers
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver
dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
# Default domain for DHCP leases
dhcp_domain = openstacklocal
# Dnsmasq options
dnsmasq_config_file = /etc/neutron/dnsmasq-neutron.conf
dnsmasq_dns_servers =
dnsmasq_lease_max = 16777216
# Metadata
enable_isolated_metadata = True
---------------------------------------------------------------------
# Ansible managed: /opt/openstack-ansible/playbooks/roles/os_neutron/templates/l3_agent.ini.j2

# General
[DEFAULT]
verbose = True
debug = False
# While this option is deprecated in Liberty, if we remove it then it takes
# a default value of 'br-ex', which we do not want. We therefore leave it
# in place for now and can remove it in Mitaka.
external_network_bridge =
gateway_external_network_id =
# Drivers
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver
# Agent mode (legacy only)
agent_mode = legacy
# Conventional failover
allow_automatic_l3agent_failover = True
# HA failover
ha_confs_path = /var/lib/neutron/ha_confs
ha_vrrp_advert_int = 2
ha_vrrp_auth_password = bee916a2589b14dd7f
ha_vrrp_auth_type = PASS
handle_internal_only_routers = False
send_arp_for_ha = 3
# Metadata
enable_metadata_proxy = True

Regards,
On 08/09/16 13:51, Vahric Muhtaryan wrote:
Hello Grant ,

Possible to share ml2_conf.ini , dhcp_agent.ini and l3_agent.ini files ?

Regards
VM

From: Grant Morley <grant at absolutedevops.io<mailto:grant at absolutedevops.io>>
Date: Thursday 8 September 2016 at 15:12
To: OpenStack Operators <openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>>
Cc: <ian.banks at serverchoice.com<mailto:ian.banks at serverchoice.com>>
Subject: [Openstack-operators] VXLAN / Tenant Network Issue


Hi All,

We are working off the OSA deployment for a new cloud system we are building and everything seems to be working apart from the tenant VXLAN network. We have tried various troubleshooting but the initial DHCP request, is not making it out of the linux bridge on the compute node. We have checked all physical networking and switch setup and they appear to be fine.

Below is an output of related networking components that we have configured. (Sorry for the long post but wanted to get as much info on here) Can anyone see what might be causing the issue or where we have gone wrong?
Neutron subnet and router:
neutron) net-list
+--------------------------------------+----------------------------------------------------+----------------------------------------------------+
| id                                   | name                                               | subnets                                            |
+--------------------------------------+----------------------------------------------------+----------------------------------------------------+
| b1da0a4f-2d06-46af-92aa-962c7a7c36f9 | ext-net                                            | 405f439c-51bb-40b6-820a-9048c2ee69fe               |
|                                      |                                                    | 185.136.232.0/22                                   |
| a256ccb2-273a-4738-97ab-bd8bfbc2a2cc | HA network tenant 7b5aad6af3ee450ea60e06aaaba2da50 | 6d98faac-2e3b-43c8-bcd6-f9a6f5dcc45e               |
|                                      |                                                    | 169.254.192.0/18                                   |
| f88ceab1-a392-4281-8c60-f57d171a8029 | vxlan-172                                          | 367e88eb-b09f-4ce5-bfff-5d9e0b0e14b0
                                                                                            | 172.16.0.0/24
+--------------------------------------+----------------------------------------------------+----------------------------------------------------+

(neutron) net-show f88ceab1-a392-4281-8c60-f57d171a8029
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| id                        | f88ceab1-a392-4281-8c60-f57d171a8029 |
| mtu                       | 0                                    |
| name                      | vxlan-172                            |
| port_security_enabled     | True                                 |
| provider:network_type     | vxlan                                |
| provider:physical_network |                                      |
| provider:segmentation_id  | 21                                   |
| router:external           | False                                |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   | 367e88eb-b09f-4ce5-bfff-5d9e0b0e14b0 |
| tenant_id                 | 7b5aad6af3ee450ea60e06aaaba2da50     |
+---------------------------+--------------------------------------+


(neutron) router-show f31ed1fb-1b90-46e3-b869-d9374e3d08b1
+-----------------------+------------------------------------------------------------------------------------------------------------------------+
| Field                 | Value                                                                                                                  |
+-----------------------+------------------------------------------------------------------------------------------------------------------------+
| admin_state_up        | True                                                                                                                   |
| distributed           | False                                                                                                                  |
| external_gateway_info | {"network_id": "b1da0a4f-2d06-46af-92aa-962c7a7c36f9", "enable_snat": true, "external_fixed_ips": [{"subnet_id":       |
|                       | "405f439c-51bb-40b6-820a-9048c2ee69fe", "ip_address": "185.136.232.55"}]}                                              |
| ha                    | True                                                                                                                   |
| id                    | f31ed1fb-1b90-46e3-b869-d9374e3d08b1                                                                                   |
| name                  | ext-router                                                                                                             |
| routes                |                                                                                                                        |
| status                | ACTIVE                                                                                                                 |
| tenant_id             | 7b5aad6af3ee450ea60e06aaaba2da50                                                                                       |
+-----------------------+------------------------------------------------------------------------------------------------------------------------+

(neutron) router-port-list f31ed1fb-1b90-46e3-b869-d9374e3d08b1
+--------------------------------------+----------------------------------------+-------------------+------------------------------------------+
| id                                   | name                                   | mac_address       | fixed_ips                                |
+--------------------------------------+----------------------------------------+-------------------+------------------------------------------+
| 443d8a0e-833e-4dd2-9320-c2a361e97bf0 | HA port tenant                         | fa:16:3e:db:48:be | {"subnet_id": "6d98faac-2e3b-            |
|                                      | 7b5aad6af3ee450ea60e06aaaba2da50       |                   | 43c8-bcd6-f9a6f5dcc45e", "ip_address":   |
|                                      |                                        |                   | "169.254.192.2"}                         |
| 58312691-77d1-408a-adf2-8c74bb87d35d | HA port tenant                         | fa:16:3e:26:86:3c | {"subnet_id": "6d98faac-2e3b-            |
|                                      | 7b5aad6af3ee450ea60e06aaaba2da50       |                   | 43c8-bcd6-f9a6f5dcc45e", "ip_address":   |
|                                      |                                        |                   | "169.254.192.1"}                         |
| 8182e8ca-0e3d-444a-ac4f-f424027aa373 |                                        | fa:16:3e:20:1c:08 | {"subnet_id": "405f439c-51bb-40b6-820a-  |
|                                      |                                        |                   | 9048c2ee69fe", "ip_address":             |
|                                      |                                        |                   | "185.136.232.55"}                        |
| beaa905d-fc68-46ba-9fd3-9f620584a1f7 |                                        | fa:16:3e:5a:8e:c0 | {"subnet_id": "367e88eb-b09f-4ce5-bfff-  |
|                                      |                                        |                   | 5d9e0b0e14b0", "ip_address":             |
|                                      |                                        |                   | "172.16.0.254"}                          |
+--------------------------------------+----------------------------------------+-------------------+------------------------------------------+

The bridge and interface for the instance:

root at compute-2:~# brctl show
bridge name      bridge id                              STP enabled       interfaces
br-mgmt                              8000.1418775ed1bc         no                           bond0.11
br-storage                           8000.1418775ed1bc         no                           bond0.31
br-vlan                  8000.1418775ed1be        no                           bond1
br-vxlan                               8000.1418775ed1be        no                           bond1.21
brqf88ceab1-a3                8000.0a81d25d36ce         no                           tapf9871920-e0
                                                                                                                vxlan-21
Network agent node namespaces:
root at network-1_neutron_agents_container-f3caf6a1:~# ip netns
qrouter-f31ed1fb-1b90-46e3-b869-d9374e3d08b1
qdhcp-f88ceab1-a392-4281-8c60-f57d171a8029
qdhcp-b1da0a4f-2d06-46af-92aa-962c7a7c36f9

The two qdhcp namespaces are able to ping to each other.
When booting the instance the DHCP request can be seen:
root at compute-2:~# dhcpdump -i tapf9871920-e0
  TIME: 2016-09-08 11:49:03.646
    IP: 0.0.0.0 (fa:16:3e:32:7e:79) > 255.255.255.255 (ff:ff:ff:ff:ff:ff)
    OP: 1 (BOOTPREQUEST)
HTYPE: 1 (Ethernet)
  HLEN: 6
  HOPS: 0
   XID: 7840761a
  SECS: 60
FLAGS: 0
CIADDR: 0.0.0.0
YIADDR: 0.0.0.0
SIADDR: 0.0.0.0
GIADDR: 0.0.0.0
CHADDR: fa:16:3e:32:7e:79:00:00:00:00:00:00:00:00:00:00
SNAME: .
FNAME: .
OPTION:  53 (  1) DHCP message type         1 (DHCPDISCOVER)
OPTION:  61 (  7) Client-identifier         01:fa:16:3e:32:7e:79
OPTION:  57 (  2) Maximum DHCP message size 576
OPTION:  55 (  9) Parameter Request List      1 (Subnet mask)
                                                                                      3 (Routers)
                                                                                      6 (DNS server)
                                                                                     12 (Host name)
                                                                                     15 (Domainname)
                                                                                     26 (Interface MTU)
                                                                                     28 (Broadcast address)
                                                                                     42 (NTP servers)
                                                                                    121 (Classless Static Route)

OPTION:  60 ( 12) Vendor class identifier   udhcp 1.20.1
OPTION:  12 (  6) Host name                 cirros
---------------------------------------------------------------------------
The DHCP packet is seen on the tap interface for the instance and the bridge brqf88ceab1-a3, but not on any other interface on the compute host. No DHCP packet is observed on the network agent container running the DHCP namespace.
output of the instance booting:
 Starting network...
udhcpc (v1.20.1) started
Sending discover...
Sending discover...
Sending discover...
Usage: /sbin/cirros-dhcpc <up|down>
No lease, failing
WARN: /etc/rc3.d/S40-network failed
cirros-ds 'net' up at 181.24

Regards,
--
[cid:part1.44430905.A1A46B65 at absolutedevops.io]
Grant Morley
Cloud Lead
Absolute DevOps Ltd
Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP
www.absolutedevops.io<http://www.absolutedevops.io/> grant at absolutedevops.io<mailto:grant at absolutedevops.i> 0845 874 0580
_______________________________________________ OpenStack-operators mailing list OpenStack-operators at lists.openstack.org<mailto:OpenStack-operators at lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--
[cid:image001.png at 01D212BF.7FD38DB0]
Grant Morley
Cloud Lead
Absolute DevOps Ltd
Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP
www.absolutedevops.io<http://www.absolutedevops.io/> grant at absolutedevops.io<mailto:grant at absolutedevops.i> 0845 874 0580
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20160919/66cb9ce0/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 4370 bytes
Desc: image001.png
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20160919/66cb9ce0/attachment-0001.png>


More information about the OpenStack-operators mailing list