<div><div>I launched a new instance:</div><div><br></div><div><div><br></div></div><div>I ssh'd to 172.29.1.4 and my instance was running but nova list shows no floating IP assigned.</div><div><br></div><div><div><div>
nova list</div><div>+--------------------------------------+--------------+--------+----------------------------------+</div><div>| ID | Name | Status | Networks |</div>
<div>+--------------------------------------+--------------+--------+----------------------------------+</div><div>| ff796bfa-0582-4ccb-8af0-b112b0c15326 | test1 | ACTIVE | development=10.2.0.5 |</div><div>
+--------------------------------------+--------------+--------+----------------------------------+</div></div></div><div><br></div><div>The database says this is assigned</div><div><br></div><div>mysql> select * from floating_ips where address="172.29.1.4"\G;</div>
<div>*************************** 1. row ***************************</div><div> created_at: 2012-03-26 08:32:20</div><div> updated_at: 2012-03-28 13:01:31</div><div> deleted_at: NULL</div><div> deleted: 0</div><div>
id: 4</div><div> address: 172.29.1.4</div><div> fixed_ip_id: 1030</div><div> project_id: 01b1e8df305b49998e6ecbac02cb9f70</div><div> host: openstack1</div><div>auto_assigned: 1</div><div> pool: nova</div>
<div> interface: eth5</div><div>1 row in set (0.00 sec)</div></div><div><br></div><div>Cheers,</div><div><br></div><div>Kev</div><br><div class="gmail_quote">On 28 March 2012 14:00, Day, Phil <span dir="ltr"><<a href="mailto:philip.day@hp.com">philip.day@hp.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-GB" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hum – very odd. Have you looked in the DB tables to see if its assigned properly in there ? If its not getting assigned in the DB then there’s a risk that it could be used again by the next instance.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:uksysadmin@gmail.com" target="_blank">uksysadmin@gmail.com</a> [mailto:<a href="mailto:uksysadmin@gmail.com" target="_blank">uksysadmin@gmail.com</a>] <b>On Behalf Of </b>Kevin Jackson<br>
<b>Sent:</b> 28 March 2012 13:58<br><b>To:</b> Day, Phil<br><b>Cc:</b> <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br><b>Subject:</b> Re: [Openstack] Auto Assigned Floating IPs take a long time to associate<u></u><u></u></span></p>
<div><div class="h5"><p class="MsoNormal"><u></u> <u></u></p><p class="MsoNormal">Cheers Phil,<u></u><u></u></p><div><p class="MsoNormal">It's not that I can't get to the instances - I'd assume it was an ARP issue at that point - its the fact that Nova hasn't notified me that it has assigned an IP address: see my additional note where I guessed an IP address it would've assigned and I could access it... but management tools were still blissfully unaware.<u></u><u></u></p>
</div><div><p class="MsoNormal">I used to have a working Diablo set up with pretty much this same set up (this is under Essex RC1 / Ubuntu 12.04) and never had an issue.<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p>
</div><div><p class="MsoNormal">This just looks like it has assigned the IP to the instance, but hasn't notified the management underbelly of MySQL so any client tools interrogating the instance doesn't see that nova has done this work already.<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal" style="margin-bottom:12.0pt">Kev<u></u><u></u></p><div><p class="MsoNormal">On 28 March 2012 13:04, Day, Phil <<a href="mailto:philip.day@hp.com" target="_blank">philip.day@hp.com</a>> wrote:<u></u><u></u></p>
<div><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Are you sure that its Nova that is taking the time to associate the IP, and not an ARP issue in your network ?</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I’ve seen this behaviour when quickly reusing floating IP addresses – Nova does the assignment and sends out an unsolicited ARP response (assuming you have the “send_arp_for_ha” flag set) – this is in network/linux_net.bind_floating_ip(). However sometimes an unsolicited APR response can get dropped in the network, ad so if the switch doesn’t see this message and it already has a previous ARP mapping in its cache then it will continue to try and send traffic to the previous user of that address until the cache times out. </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Note that some network failover systems send multiple requests to get more certainty around this (for example I’ve seen a VPN solution use 6 messages).</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">There are a couple of things you could try:</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p style="margin-left:54.0pt"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">-</span><span style="font-size:7.0pt;color:#1f497d"> </span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Add a flag to be able to increase the number of arp_responses sent</span><u></u><u></u></p>
<p style="margin-left:54.0pt"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">-</span><span style="font-size:7.0pt;color:#1f497d"> </span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Change the allocation of floating_ips so that instead of picking the first free one in the DB you pick the one which has been unused for the longest time (reduces the risk of reusing an address before the switch times out the entry in its cache).</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Phil</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> openstack-bounces+philip.day=<a href="mailto:hp.com@lists.launchpad.net" target="_blank">hp.com@lists.launchpad.net</a> [mailto:<a href="mailto:openstack-bounces%2Bphilip.day" target="_blank">openstack-bounces+philip.day</a>=<a href="mailto:hp.com@lists.launchpad.net" target="_blank">hp.com@lists.launchpad.net</a>] <b>On Behalf Of </b>Kevin Jackson<br>
<b>Sent:</b> 28 March 2012 12:49<br><b>To:</b> <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br><b>Subject:</b> [Openstack] Auto Assigned Floating IPs take a long time to associate</span><u></u><u></u></p>
<div><div><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal">Hi all,<u></u><u></u></p><div><p class="MsoNormal">I've got the following set in my nova.conf:<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p>
</div><div><p class="MsoNormal">--auto_assign_floating_ip<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">and I fire up an instance.<u></u><u></u></p></div><div><p class="MsoNormal">
Everything works, a private IP is assigned... the instance is running... but it can take an inordinate amount of time (anywhere upwards of 2 mins, sometimes a lot longer) to associate a floating IP automatically.<u></u><u></u></p>
</div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">Anybody else experienced this? Any clues on what I can do to troubleshoot this? What is the condition when a Floating IP is assigned? I've seen it assign Floating IPs very quickly when it is still Booting, not Active, say.<u></u><u></u></p>
</div><div><p class="MsoNormal">Is it dependent on anything within the Instance itself?<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">Cheers,<u></u><u></u></p></div><div>
<p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">Kev<u></u><u></u></p></div><div><p class="MsoNormal">-- <br>Kevin Jackson<br>@itarchitectkev<u></u><u></u></p></div></div></div></div></div></div><p class="MsoNormal">
<br><br clear="all"><u></u><u></u></p><div><p class="MsoNormal"><u></u> <u></u></p></div><p class="MsoNormal">-- <br>Kevin Jackson<br>@itarchitectkev<u></u><u></u></p></div></div></div></div></div></blockquote></div><br><br clear="all">
<div><br></div>-- <br>Kevin Jackson<br>@itarchitectkev<br>