<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<base href="x-msg://772/">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<div apple-content-edited="true"><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
Hi Jon,</div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<br>
</div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
Thanks for your help! Both the kernel and the iproute packages are updated. RDO does a great job with this. Beside the 2.6.32 + netns kernel provided by RDO I also tested it with a 3.9.8, with the same results. I'd add to your troubleshooting steps a very simple
test to check if netns is enabled in the kernel: checking if the "/proc/self/ns" path exists. </div>
</span><br class="Apple-interchange-newline">
</div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
Back to the original issue, there are no errors on the Quantum side.</div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<br>
</div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<br>
</div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
Thanks,</div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<br>
</div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
Alessandro</div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<br>
</div>
</span></div>
</span><br class="Apple-interchange-newline">
</span><br class="Apple-interchange-newline">
</div>
<br>
<div>
<div>On Jul 7, 2013, at 02:36 , Hathaway.Jon <<a href="mailto:Jon.Hathaway@IGT.com">Jon.Hathaway@IGT.com</a>></div>
<div> wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div lang="EN-GB" link="blue" vlink="purple" style="font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; ">
<div class="WordSection1" style="page: WordSection1; ">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Hi Allessandro<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">I know this is probably something you have probably already tested for the RDO installation, but have you upgraded the CentOS kernel and the iproute package as both are
missing the NetNS support required for Quantum. Ubuntu fixed this issue back in 10.04 but for whatever reason the current production kernel for CentOS still hasn’t.<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">We had to update the kernel and the iproute package. If you check the log files for l3-agent, especially the dhcp logs you may find errors like “command not recognised”
as the shipped iproute cor CentOS 6.4 doesn’t support the netns extensions.<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><a href="https://www.redhat.com/archives/rdo-list/2013-May/msg00015.html" style="color: purple; text-decoration: underline; ">https://www.redhat.com/archives/rdo-list/2013-May/msg00015.html</a><o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">My workaround was:<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; ">If installing on a EPEL6 distribution like Centos 6.4, there is a bug in the Kernel release which has disabled the Network Names (netns) support that is required to create overlapping networks in Quantum and
is required to run the DHCP agent that assigns IP addresses on boot, and also setup the l3-agent that is responsible for forwarding requests from the instances to the API to retrieve any specific instance meta data.<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; ">A quick check on the node configured with Quantum in /var/log/quantum/dhcp-agent.log will show something like:<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 9pt; font-family: Calibri, sans-serif; ">RuntimeError:<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 9pt; font-family: Calibri, sans-serif; ">Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ip', '-o', 'netns', 'list']<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 9pt; font-family: Calibri, sans-serif; ">Exit code: 255<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 9pt; font-family: Calibri, sans-serif; ">Stdout: ''<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 9pt; font-family: Calibri, sans-serif; ">Stderr: 'Object "netns" is unknown, try "ip help".\n'<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; ">If you try and run ‘ip netns’ from the command and it fails, you will need to update the kernel and possibly the iproute2 package:<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 9pt; font-family: Calibri, sans-serif; "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 9pt; font-family: Calibri, sans-serif; ">[root@oscontroller ~]# ip nets<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 9pt; font-family: Calibri, sans-serif; ">Object "nets" is unknown, try "ip help".<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; ">Netns is available in the iproute2 package , but it requires additional support from the kernel. A new kernel has been released for testing only by Redhat version kernel-2.6.32-358.6.2.openstack.el6.x86_64 whilst
the installed version is kernel-</span><span style="font-family: Calibri, sans-serif; "> </span><span style="font-family: Calibri, sans-serif; ">2.6.32-358.el6.x86_64 that comes with Centos 6.4.<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; ">To add the new kernel and iproute2 packages requires updating the kernel and kernel-firmware packages from the Grizzly repository.<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 9pt; font-family: Calibri, sans-serif; ">yum install<span class="Apple-converted-space"> </span></span><span style="font-family: Calibri, sans-serif; "><a href="http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/epel-6/kernel-firmware-2.6.32-358.6.2.openstack.el6.noarch.rpm" style="color: purple; text-decoration: underline; "><span style="font-size: 9pt; ">http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/epel-6/kernel-firmware-2.6.32-358.6.2.openstack.el6.noarch.rpm</span></a></span><span style="font-size: 9pt; font-family: Calibri, sans-serif; "><o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 9pt; font-family: Calibri, sans-serif; "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 9pt; font-family: Calibri, sans-serif; ">yum install<span class="Apple-converted-space"> </span></span><span style="font-family: Calibri, sans-serif; "><a href="http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/epel-6/kernel-2.6.32-358.6.2.openstack.el6.x86_64.rpm" style="color: purple; text-decoration: underline; "><span style="font-size: 9pt; ">http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/epel-6/kernel-2.6.32-358.6.2.openstack.el6.x86_64.rpm</span></a></span><span style="font-size: 9pt; font-family: Calibri, sans-serif; "><o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 9pt; font-family: Calibri, sans-serif; "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 9pt; font-family: Calibri, sans-serif; ">yum install<span class="Apple-converted-space"> </span><a href="http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/epel-6/iproute-2.6.32-23.el6_4.netns.1.x86_64.rpm" style="color: purple; text-decoration: underline; ">http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/epel-6/iproute-2.6.32-23.el6_4.netns.1.x86_64.rpm</a><o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; ">Check in /etc/grub.conf that the new kernel is being reference and then restart the node running Quantum.<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; ">After reboot, try and run the ‘ip netns’ it should run without an error.<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; ">If you have previously added an instance before upgrading the packages, you will need to remove the networks, routers and ports and re-add them before continuing. However – it is likely that you will end up with
stake ports on the Quantum Server. As shown below:<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; ">[root@oscontroller quantum]# ovs-vsctl show<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; ">e4b86f82-2d16-49b1-9077-93abf2b32400<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> Bridge br-ex<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> Port "qg-3d8f69e7-5d"<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> Interface "qg-3d8f69e7-5d"<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> type: internal<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> Port br-ex<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> Interface br-ex<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> type: internal<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> Bridge br-int<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> Port "qr-c7145535-d1"<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> tag: 1<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> Interface "qr-c7145535-d1"<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> type: internal<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> Port "tapc4fb5d73-3e"<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> tag: 1<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> Interface "tapc4fb5d73-3e"<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> type: internal<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> Port br-int<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> Interface br-int<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> type: internal<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> Port "tape76c5e3c-1b"<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> tag: 2<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> Interface "tape76c5e3c-1b"<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> type: internal<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> ovs_version: "1.10.0"<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 8pt; font-family: Calibri, sans-serif; "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; ">These ports/interfaces will need to be deleted before the networking will work successfully.</span><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Just a thought.<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span></div>
<div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Jon<o:p></o:p></span></div>
</div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "> </span></div>
<div>
<div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(181, 196, 223); padding: 3pt 0cm 0cm; ">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<b><span lang="EN-US" style="font-size: 10pt; font-family: Calibri, sans-serif; ">From:</span></b><span lang="EN-US" style="font-size: 10pt; font-family: Calibri, sans-serif; "><span class="Apple-converted-space"> </span>Openstack [mailto:openstack-<a href="mailto:bounces+jon.hathaway=igt.com@lists.launchpad.net" style="color: purple; text-decoration: underline; ">bounces+jon.hathaway=igt.com@lists.launchpad.net</a>]<span class="Apple-converted-space"> </span><b>On
Behalf Of<span class="Apple-converted-space"> </span></b>Alessandro Pilotti<br>
<b>Sent:</b><span class="Apple-converted-space"> </span>06 July 2013 16:23<br>
<b>To:</b><span class="Apple-converted-space"> </span>OpenStack<br>
<b>Subject:</b><span class="Apple-converted-space"> </span>Re: [Openstack] [HyperV][Quantum] Quantum dhcp agent not working for Hyper-V<o:p></o:p></span></div>
</div>
</div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; "> </span></div>
<div>
<div>
<div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 13.5pt; font-family: Calibri, sans-serif; ">Hi Bruno,<o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 13.5pt; font-family: Calibri, sans-serif; "> </span></div>
</div>
<div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 13.5pt; font-family: Calibri, sans-serif; ">I just hit the same (or a very similar) issue doing a multinode deployment with RDO on CentOS 6.4 (OVS 1.10) while we had no problem until now using Ubuntu 12.04 (OVS 1.4).<o:p></o:p></span></div>
</div>
</div>
<div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 13.5pt; font-family: Calibri, sans-serif; ">Can you please provide some more details about the Linux OS you are using and your multinode configuration?<o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 13.5pt; font-family: Calibri, sans-serif; "> </span></div>
</div>
<div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 13.5pt; font-family: Calibri, sans-serif; ">I tested it with flat and VLAN networks, so far it doesn't look like an Hyper-V related issue.<o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 13.5pt; font-family: Calibri, sans-serif; "> </span></div>
</div>
<div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 13.5pt; font-family: Calibri, sans-serif; "> </span></div>
</div>
<div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 13.5pt; font-family: Calibri, sans-serif; ">Thanks,<o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 13.5pt; font-family: Calibri, sans-serif; "> </span></div>
</div>
<div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-size: 13.5pt; font-family: Calibri, sans-serif; ">Alessandro<o:p></o:p></span></div>
</div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; "> </span></div>
</div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; "> </span></div>
<div>
<div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; ">On Jun 7, 2013, at 23:51 , Bruno Oliveira ~lychinus <<a href="mailto:brunnop.oliveira@gmail.com" style="color: purple; text-decoration: underline; ">brunnop.oliveira@gmail.com</a>> wrote:<o:p></o:p></span></div>
</div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; "><br>
<br>
<o:p></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; ">"(...)Do you have your vSwitch properly configured on your hyper-v host?(...)"<br>
<br>
<br>
<o:p></o:p></span></div>
<blockquote style="margin-top: 5pt; margin-bottom: 5pt; ">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; ">I can't say for sure, Peter, but I think so...<o:p></o:p></span></div>
</blockquote>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; "><br>
In troubleshooting we did (and are still doing) I can tell that<br>
regardless of the network model that we're using (FLAT or VLAN<br>
Network),<br>
the instance that is provisioned on Hyper-V (for some reason) can't<br>
reach the quantum-l3-agent "by default"<br>
(I said "default" because, we just managed to do it after a hard, long<br>
and boring troubleshoting,<br>
yet, we're not sure if that's how it should be done, indeed)<br>
<br>
Since it's not something quick to explain, I'll present the scenario:<br>
(I'm not sure if it might be a candidate for a fix in quantum-l3-agent,<br>
so quantum-devs might be interested too)<br>
<br>
<br>
Here's how our network interfaces turns out, in our network controller:<br>
<br>
==============================<br>
External bridge network<br>
==============================<br>
<br>
Bridge "br-eth1"<br>
Port "br-eth1"<br>
Interface "br-eth1"<br>
type: internal<br>
Port "eth1.11"<br>
Interface "eth1.11"<br>
Port "phy-br-eth1"<br>
Interface "phy-br-eth1"<br>
<br>
==============================<br>
Internal network<br>
==============================<br>
<br>
Bridge br-int<br>
Port "int-br-eth1"<br>
Interface "int-br-eth1"<br>
Port br-int<br>
Interface br-int<br>
type: internal<br>
Port "tapb610a695-46"<br>
tag: 1<br>
Interface "tapb610a695-46"<br>
type: internal<br>
Port "qr-ef10bef4-fa"<br>
tag: 1<br>
Interface "qr-ef10bef4-fa"<br>
type: internal<br>
<br>
==============================<br>
<br>
There's another iface named "br-ex" that we're using for floating_ips,<br>
but it has nothing to do with what we're doing right now, so I'm skipping it...<br>
<br>
<br>
************ So, for the hands-on ****************<br>
<br>
I know it may be a little bit hard to understand, but I'll do my best<br>
trying to explain:<br>
<br>
1) the running instance in Hyper-V, which is linked to Hyper-V vSwitch<br>
is actually<br>
communicating to bridge: "br-eth1" (that is in the network controller).<br>
<br>
NOTE: That's where the DHCP REQUEST (from the instance) lands<br>
<br>
<br>
2) The interface MAC Address, of that running instance on Hyper-V, is:<br>
fa:16:3e:95:95:e4. (we're gonna use it on later steps)<br>
Since DHCP is not fully working yet, we had to manually set an IP for<br>
that instance: "10.5.5.3"<br>
<br>
<br>
3) From that instance interface, the dhcp_broadcast should be forward -><br>
FROM interface "eth1.12" TO "phy-br-eth1"<br>
And FROM interface "phy-br-eth1" TO the bridge "br-int" *** THIS<br>
IS WHERE THE PACKETS ARE DROPPED ***.<br>
<br>
Check it out for the "actions:drop"<br>
---------------------------------------------------------------------------------------------<br>
root@osnetwork:~# ovs-dpctl dump-flows br-int |grep 10.5.5.3<br>
<br>
in_port(4),eth(src=fa:16:3e:f0:ac:8e,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=10.5.5.3,tip=10.5.5.1,op=1,sha=fa:16:3e:f0:ac:8e,tha=00:00:00:00:00:00),<br>
packets:20, bytes:1120, used:0.412s, actions:drop<br>
---------------------------------------------------------------------------------------------<br>
<br>
4) Finally, when the packet reaches the bridge "br-int", the<br>
DHCP_REQUEST should be forward to the<br>
dhcp_interface, that is: tapb610a695-46 *** WHICH IS NOT<br>
HAPPENING EITHER ***<br>
<br>
<br>
5) How to fix :: bridge br-eth1<br>
<br>
-------------------------------------------<br>
5.1. Getting to know the ifaces of 'br-eth1'<br>
-------------------------------------------<br>
root@osnetwork:~# ovs-ofctl show br-eth1<br>
<br>
OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:0000e0db554e164b<br>
n_tables:255, n_buffers:256 features: capabilities:0xc7, actions:0xfff<br>
<br>
1(eth1.11): addr:e0:db:55:4e:16:4b<br>
config: 0<br>
state: 0<br>
current: 10GB-FD AUTO_NEG<br>
advertised: 1GB-FD 10GB-FD FIBER AUTO_NEG<br>
supported: 1GB-FD 10GB-FD FIBER AUTO_NEG<br>
<br>
3(phy-br-eth1): addr:26:9b:97:93:b9:70<br>
config: 0<br>
state: 0<br>
current: 10GB-FD COPPER<br>
<br>
LOCAL(br-eth1): addr:e0:db:55:4e:16:4b<br>
config: 0<br>
state: 0<br>
<br>
OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0<br>
<br>
<br>
-------------------------------------------<br>
5.2. Adding flow rules to enable passing (instead of dropping)<br>
-------------------------------------------<br>
<br>
# the source mac_address (dl_src) is the from the interface of the<br>
# running instance on Hyper-V. This fix the DROP (only)<br>
<br>
root@osnetwork:~# ovs-ofctl add-flow br-eth1<br>
priority=10,in_port=3,dl_src=fa:16:3e:95:95:e4,actions=normal<br>
<br>
<br>
<br>
6) How to fix :: bridge br-int<br>
<br>
-------------------------------------------<br>
6.1. Getting to know the ifaces of 'br-int'<br>
-------------------------------------------<br>
<br>
root@osnetwork:~# ovs-ofctl show br-int<br>
<br>
OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:000092976d64274d<br>
<br>
n_tables:255, n_buffers:256 features: capabilities:0xc7, actions:0xfff<br>
<br>
1(tapb610a695-46): addr:19:01:00:00:00:00<br>
config: PORT_DOWN<br>
state: LINK_DOWN<br>
<br>
4(int-br-eth1): addr:5a:56:e1:53:e9:90<br>
config: 0<br>
state: 0<br>
current: 10GB-FD COPPER<br>
<br>
5(qr-ef10bef4-fa): addr:19:01:00:00:00:00<br>
config: PORT_DOWN<br>
state: LINK_DOWN<br>
<br>
LOCAL(br-int): addr:92:97:6d:64:27:4d<br>
config: 0<br>
state: 0<br>
<br>
-------------------------------------------<br>
6.2. Adding flow rules to enable FORWARD<br>
FROM: interface int-br-eth1 (4)<br>
TO: interface tapb610a695-46 (1) -> dhcp_interface<br>
<br>
and the REVERSE_FORWARD: from (1) to (4)<br>
-------------------------------------------<br>
root@osnetwork:~# ovs-ofctl add-flow br-int<br>
priority=12,in_port=4,dl_src=fa:16:3e:f0:ac:8e,action=1,normal<br>
root@osnetwork:~# ovs-ofctl add-flow br-int<br>
priority=12,in_port=1,dl_dst=fa:16:3e:f0:ac:8e,action=4,normal<br>
<br>
<br>
==================<br>
Conclusion<br>
==================<br>
<br>
That way, and only *that way*, the Hyper-V instance is able to<br>
exchange ARP with the dhcp (network controller).<br>
<br>
Even though, it is functional, we're not sure if that's how it HAS to<br>
be done. May I have your thoughts on it?<br>
<br>
Should we really have to create those rules/actions in openVSwitch to<br>
make the instance (of hyper-v) to reach out<br>
DHCP ? It seems either bug or something is wierd in my configurations...<br>
<br>
May I have your opinions on it?<br>
<br>
<br>
We'd greatly appreciate your feedback. Thank you very much.<br>
<br>
_______________________________________________<br>
Mailing list:<span class="Apple-converted-space"> </span><a href="https://launchpad.net/~openstack" style="color: purple; text-decoration: underline; ">https://launchpad.net/~openstack</a><br>
Post to :<span class="Apple-converted-space"> </span><a href="mailto:openstack@lists.launchpad.net" style="color: purple; text-decoration: underline; ">openstack@lists.launchpad.net</a><br>
Unsubscribe :<span class="Apple-converted-space"> </span><a href="https://launchpad.net/~openstack" style="color: purple; text-decoration: underline; ">https://launchpad.net/~openstack</a><br>
More help :<span class="Apple-converted-space"> </span><a href="https://help.launchpad.net/ListHelp" style="color: purple; text-decoration: underline; ">https://help.launchpad.net/ListHelp</a><o:p></o:p></span></div>
</div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif; ">
<span style="font-family: Calibri, sans-serif; "> </span></div>
</div>
</div>
</blockquote>
</div>
<br>
</body>
</html>