<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><base href="x-msg://772/"><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.apple-style-span
{mso-style-name:apple-style-span;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-GB link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks Alessandro – I’ll add it our test processes!<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:EN-US'>Jon Hathaway<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:EN-US'>Distinguished Architect<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:EN-US'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:EN-US'>International Game Technology (UK)<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:EN-US'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:EN-US'>Office (US): +1 </span><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:EN-US'>702 669 8185, (x698185)</span><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:EN-US'><o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:EN-US'>Mobile (UK): +44 7799 711 981<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:EN-US'><a href="mailto:jon.hathaway@igt.com">jon.hathaway@igt.com</a><o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Openstack [mailto:openstack-bounces+jon.hathaway=igt.com@lists.launchpad.net] <b>On Behalf Of </b>Alessandro Pilotti<br><b>Sent:</b> 06 July 2013 17:28<br><b>To:</b> OpenStack<br><b>Subject:</b> Re: [Openstack] [HyperV][Quantum] Quantum dhcp agent not working for Hyper-V<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><div><div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black'>Hi Jon,<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black'>Thanks for your help! Both the kernel and the iproute packages are updated. RDO does a great job with this. Beside the 2.6.32 + netns kernel provided by RDO I also tested it with a 3.9.8, with the same results. I'd add to your troubleshooting steps a very simple test to check if netns is enabled in the kernel: checking if the "/proc/self/ns" path exists. <o:p></o:p></span></p></div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black'>Back to the original issue, there are no errors on the Quantum side.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black'>Thanks,<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black'>Alessandro<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black'><o:p> </o:p></span></p></div></div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black'><br><br></span><o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>On Jul 7, 2013, at 02:36 , Hathaway.Jon <<a href="mailto:Jon.Hathaway@IGT.com">Jon.Hathaway@IGT.com</a>><o:p></o:p></p></div><div><p class=MsoNormal> wrote:<o:p></o:p></p></div><p class=MsoNormal><br><br><o:p></o:p></p><div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Allessandro</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I know this is probably something you have probably already tested for the RDO installation, but have you upgraded the CentOS kernel and the iproute package as both are missing the NetNS support required for Quantum. Ubuntu fixed this issue back in 10.04 but for whatever reason the current production kernel for CentOS still hasn’t.</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>We had to update the kernel and the iproute package. If you check the log files for l3-agent, especially the dhcp logs you may find errors like “command not recognised” as the shipped iproute cor CentOS 6.4 doesn’t support the netns extensions.</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><a href="https://www.redhat.com/archives/rdo-list/2013-May/msg00015.html"><span style='color:purple'>https://www.redhat.com/archives/rdo-list/2013-May/msg00015.html</span></a></span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>My workaround was:</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>If installing on a EPEL6 distribution like Centos 6.4, there is a bug in the Kernel release which has disabled the Network Names (netns) support that is required to create overlapping networks in Quantum and is required to run the DHCP agent that assigns IP addresses on boot, and also setup the l3-agent that is responsible for forwarding requests from the instances to the API to retrieve any specific instance meta data.</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>A quick check on the node configured with Quantum in /var/log/quantum/dhcp-agent.log will show something like:</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Calibri","sans-serif"'>RuntimeError:</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Calibri","sans-serif"'>Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ip', '-o', 'netns', 'list']</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Calibri","sans-serif"'>Exit code: 255</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Calibri","sans-serif"'>Stdout: ''</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Calibri","sans-serif"'>Stderr: 'Object "netns" is unknown, try "ip help".\n'</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>If you try and run ‘ip netns’ from the command and it fails, you will need to update the kernel and possibly the iproute2 package:</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Calibri","sans-serif"'>[root@oscontroller ~]# ip nets</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Calibri","sans-serif"'>Object "nets" is unknown, try "ip help".</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>Netns is available in the iproute2 package , but it requires additional support from the kernel. A new kernel has been released for testing only by Redhat version kernel-2.6.32-358.6.2.openstack.el6.x86_64 whilst the installed version is kernel- 2.6.32-358.el6.x86_64 that comes with Centos 6.4.</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>To add the new kernel and iproute2 packages requires updating the kernel and kernel-firmware packages from the Grizzly repository.</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Calibri","sans-serif"'>yum install<span class=apple-converted-space> </span></span><span style='font-family:"Calibri","sans-serif"'><a href="http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/epel-6/kernel-firmware-2.6.32-358.6.2.openstack.el6.noarch.rpm"><span style='font-size:9.0pt;color:purple'>http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/epel-6/kernel-firmware-2.6.32-358.6.2.openstack.el6.noarch.rpm</span></a></span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Calibri","sans-serif"'>yum install<span class=apple-converted-space> </span></span><span style='font-family:"Calibri","sans-serif"'><a href="http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/epel-6/kernel-2.6.32-358.6.2.openstack.el6.x86_64.rpm"><span style='font-size:9.0pt;color:purple'>http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/epel-6/kernel-2.6.32-358.6.2.openstack.el6.x86_64.rpm</span></a></span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Calibri","sans-serif"'>yum install<span class=apple-converted-space> </span><a href="http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/epel-6/iproute-2.6.32-23.el6_4.netns.1.x86_64.rpm"><span style='color:purple'>http://repos.fedorapeople.org/repos/openstack/openstack-grizzly/epel-6/iproute-2.6.32-23.el6_4.netns.1.x86_64.rpm</span></a></span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>Check in /etc/grub.conf that the new kernel is being reference and then restart the node running Quantum.</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>After reboot, try and run the ‘ip netns’ it should run without an error.</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>If you have previously added an instance before upgrading the packages, you will need to remove the networks, routers and ports and re-add them before continuing. However – it is likely that you will end up with stake ports on the Quantum Server. As shown below:</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'>[root@oscontroller quantum]# ovs-vsctl show</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'>e4b86f82-2d16-49b1-9077-93abf2b32400</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> Bridge br-ex</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> Port "qg-3d8f69e7-5d"</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> Interface "qg-3d8f69e7-5d"</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> type: internal</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> Port br-ex</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> Interface br-ex</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> type: internal</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> Bridge br-int</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> Port "qr-c7145535-d1"</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> tag: 1</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> Interface "qr-c7145535-d1"</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> type: internal</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> Port "tapc4fb5d73-3e"</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> tag: 1</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> Interface "tapc4fb5d73-3e"</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> type: internal</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> Port br-int</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> Interface br-int</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> type: internal</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> Port "tape76c5e3c-1b"</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> tag: 2</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> Interface "tape76c5e3c-1b"</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> type: internal</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> ovs_version: "1.10.0"</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:8.0pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>These ports/interfaces will need to be deleted before the networking will work successfully.</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Just a thought.</span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Jon</span><o:p></o:p></p></div></div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p></div><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'><div><p class=MsoNormal><b><span lang=EN-US style='font-size:10.0pt;font-family:"Calibri","sans-serif"'>From:</span></b><span class=apple-converted-space><span lang=EN-US style='font-size:10.0pt;font-family:"Calibri","sans-serif"'> </span></span><span lang=EN-US style='font-size:10.0pt;font-family:"Calibri","sans-serif"'>Openstack [mailto:openstack-<a href="mailto:bounces+jon.hathaway=igt.com@lists.launchpad.net"><span style='color:purple'>bounces+jon.hathaway=igt.com@lists.launchpad.net</span></a>]<span class=apple-converted-space> </span><b>On Behalf Of<span class=apple-converted-space> </span></b>Alessandro Pilotti<br><b>Sent:</b><span class=apple-converted-space> </span>06 July 2013 16:23<br><b>To:</b><span class=apple-converted-space> </span>OpenStack<br><b>Subject:</b><span class=apple-converted-space> </span>Re: [Openstack] [HyperV][Quantum] Quantum dhcp agent not working for Hyper-V</span><o:p></o:p></p></div></div></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><div><div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Calibri","sans-serif"'>Hi Bruno,</span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Calibri","sans-serif"'>I just hit the same (or a very similar) issue doing a multinode deployment with RDO on CentOS 6.4 (OVS 1.10) while we had no problem until now using Ubuntu 12.04 (OVS 1.4).</span><o:p></o:p></p></div></div></div><div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Calibri","sans-serif"'>Can you please provide some more details about the Linux OS you are using and your multinode configuration?</span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Calibri","sans-serif"'>I tested it with flat and VLAN networks, so far it doesn't look like an Hyper-V related issue.</span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Calibri","sans-serif"'>Thanks,</span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div></div><div><div><p class=MsoNormal><span style='font-size:13.5pt;font-family:"Calibri","sans-serif"'>Alessandro</span><o:p></o:p></p></div></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div><div><div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>On Jun 7, 2013, at 23:51 , Bruno Oliveira ~lychinus <<a href="mailto:brunnop.oliveira@gmail.com"><span style='color:purple'>brunnop.oliveira@gmail.com</span></a>> wrote:</span><o:p></o:p></p></div></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'><br><br><br></span><o:p></o:p></p></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>"(...)Do you have your vSwitch properly configured on your hyper-v host?(...)"<br><br><br><br></span><o:p></o:p></p></div><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'>I can't say for sure, Peter, but I think so...</span><o:p></o:p></p></div></blockquote><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'><br>In troubleshooting we did (and are still doing) I can tell that<br>regardless of the network model that we're using (FLAT or VLAN<br>Network),<br>the instance that is provisioned on Hyper-V (for some reason) can't<br>reach the quantum-l3-agent "by default"<br>(I said "default" because, we just managed to do it after a hard, long<br>and boring troubleshoting,<br>yet, we're not sure if that's how it should be done, indeed)<br><br>Since it's not something quick to explain, I'll present the scenario:<br>(I'm not sure if it might be a candidate for a fix in quantum-l3-agent,<br>so quantum-devs might be interested too)<br><br><br>Here's how our network interfaces turns out, in our network controller:<br><br>==============================<br>External bridge network<br>==============================<br><br>Bridge "br-eth1"<br> Port "br-eth1"<br> Interface "br-eth1"<br> type: internal<br> Port "eth1.11"<br> Interface "eth1.11"<br> Port "phy-br-eth1"<br> Interface "phy-br-eth1"<br><br>==============================<br>Internal network<br>==============================<br><br> Bridge br-int<br> Port "int-br-eth1"<br> Interface "int-br-eth1"<br> Port br-int<br> Interface br-int<br> type: internal<br> Port "tapb610a695-46"<br> tag: 1<br> Interface "tapb610a695-46"<br> type: internal<br> Port "qr-ef10bef4-fa"<br> tag: 1<br> Interface "qr-ef10bef4-fa"<br> type: internal<br><br>==============================<br><br>There's another iface named "br-ex" that we're using for floating_ips,<br>but it has nothing to do with what we're doing right now, so I'm skipping it...<br><br><br>************ So, for the hands-on ****************<br><br>I know it may be a little bit hard to understand, but I'll do my best<br>trying to explain:<br><br>1) the running instance in Hyper-V, which is linked to Hyper-V vSwitch<br>is actually<br>communicating to bridge: "br-eth1" (that is in the network controller).<br><br>NOTE: That's where the DHCP REQUEST (from the instance) lands<br><br><br>2) The interface MAC Address, of that running instance on Hyper-V, is:<br>fa:16:3e:95:95:e4. (we're gonna use it on later steps)<br>Since DHCP is not fully working yet, we had to manually set an IP for<br>that instance: "10.5.5.3"<br><br><br>3) From that instance interface, the dhcp_broadcast should be forward -><br> FROM interface "eth1.12" TO "phy-br-eth1"<br> And FROM interface "phy-br-eth1" TO the bridge "br-int" *** THIS<br>IS WHERE THE PACKETS ARE DROPPED ***.<br><br>Check it out for the "actions:drop"<br>---------------------------------------------------------------------------------------------<br>root@osnetwork:~# ovs-dpctl dump-flows br-int |grep 10.5.5.3<br><br>in_port(4),eth(src=fa:16:3e:f0:ac:8e,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=10.5.5.3,tip=10.5.5.1,op=1,sha=fa:16:3e:f0:ac:8e,tha=00:00:00:00:00:00),<br>packets:20, bytes:1120, used:0.412s, actions:drop<br>---------------------------------------------------------------------------------------------<br><br>4) Finally, when the packet reaches the bridge "br-int", the<br>DHCP_REQUEST should be forward to the<br> dhcp_interface, that is: tapb610a695-46 *** WHICH IS NOT<br>HAPPENING EITHER ***<br><br><br>5) How to fix :: bridge br-eth1<br><br>-------------------------------------------<br>5.1. Getting to know the ifaces of 'br-eth1'<br>-------------------------------------------<br>root@osnetwork:~# ovs-ofctl show br-eth1<br><br>OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:0000e0db554e164b<br>n_tables:255, n_buffers:256 features: capabilities:0xc7, actions:0xfff<br><br>1(eth1.11): addr:e0:db:55:4e:16:4b<br> config: 0<br> state: 0<br> current: 10GB-FD AUTO_NEG<br> advertised: 1GB-FD 10GB-FD FIBER AUTO_NEG<br> supported: 1GB-FD 10GB-FD FIBER AUTO_NEG<br><br>3(phy-br-eth1): addr:26:9b:97:93:b9:70<br> config: 0<br> state: 0<br> current: 10GB-FD COPPER<br><br>LOCAL(br-eth1): addr:e0:db:55:4e:16:4b<br> config: 0<br> state: 0<br><br>OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0<br><br><br>-------------------------------------------<br>5.2. Adding flow rules to enable passing (instead of dropping)<br>-------------------------------------------<br><br># the source mac_address (dl_src) is the from the interface of the<br># running instance on Hyper-V. This fix the DROP (only)<br><br>root@osnetwork:~# ovs-ofctl add-flow br-eth1<br>priority=10,in_port=3,dl_src=fa:16:3e:95:95:e4,actions=normal<br><br><br><br>6) How to fix :: bridge br-int<br><br>-------------------------------------------<br>6.1. Getting to know the ifaces of 'br-int'<br>-------------------------------------------<br><br>root@osnetwork:~# ovs-ofctl show br-int<br><br>OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:000092976d64274d<br><br>n_tables:255, n_buffers:256 features: capabilities:0xc7, actions:0xfff<br><br>1(tapb610a695-46): addr:19:01:00:00:00:00<br> config: PORT_DOWN<br> state: LINK_DOWN<br><br>4(int-br-eth1): addr:5a:56:e1:53:e9:90<br> config: 0<br> state: 0<br> current: 10GB-FD COPPER<br><br>5(qr-ef10bef4-fa): addr:19:01:00:00:00:00<br> config: PORT_DOWN<br> state: LINK_DOWN<br><br>LOCAL(br-int): addr:92:97:6d:64:27:4d<br> config: 0<br> state: 0<br><br>-------------------------------------------<br>6.2. Adding flow rules to enable FORWARD<br> FROM: interface int-br-eth1 (4)<br> TO: interface tapb610a695-46 (1) -> dhcp_interface<br><br> and the REVERSE_FORWARD: from (1) to (4)<br>-------------------------------------------<br>root@osnetwork:~# ovs-ofctl add-flow br-int<br>priority=12,in_port=4,dl_src=fa:16:3e:f0:ac:8e,action=1,normal<br>root@osnetwork:~# ovs-ofctl add-flow br-int<br>priority=12,in_port=1,dl_dst=fa:16:3e:f0:ac:8e,action=4,normal<br><br><br>==================<br>Conclusion<br>==================<br><br>That way, and only *that way*, the Hyper-V instance is able to<br>exchange ARP with the dhcp (network controller).<br><br>Even though, it is functional, we're not sure if that's how it HAS to<br>be done. May I have your thoughts on it?<br><br>Should we really have to create those rules/actions in openVSwitch to<br>make the instance (of hyper-v) to reach out<br>DHCP ? It seems either bug or something is wierd in my configurations...<br><br>May I have your opinions on it?<br><br><br>We'd greatly appreciate your feedback. Thank you very much.<br><br>_______________________________________________<br>Mailing list:<span class=apple-converted-space> </span><a href="https://launchpad.net/~openstack"><span style='color:purple'>https://launchpad.net/~openstack</span></a><br>Post to :<span class=apple-converted-space> </span><a href="mailto:openstack@lists.launchpad.net"><span style='color:purple'>openstack@lists.launchpad.net</span></a><br>Unsubscribe :<span class=apple-converted-space> </span><a href="https://launchpad.net/~openstack"><span style='color:purple'>https://launchpad.net/~openstack</span></a><br>More help :<span class=apple-converted-space> </span><a href="https://help.launchpad.net/ListHelp"><span style='color:purple'>https://help.launchpad.net/ListHelp</span></a></span><o:p></o:p></p></div></div><div><p class=MsoNormal><span style='font-family:"Calibri","sans-serif"'> </span><o:p></o:p></p></div></div></div><p class=MsoNormal><o:p> </o:p></p></div></body></html>