<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body bgcolor=white lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Grant,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I didn't see any DNS errors either. The solution was to explicitly configure the dns servers in the subnet. Are you doing this already? Or are you relying on the dnsmasq processes created for the router to respond to DNS queries (and forward them respectively)?<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><br>Eric<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext'> Grant Morley [mailto:grant@civo.com] <br><b>Sent:</b> Wednesday, December 04, 2019 5:40 PM<br><b>To:</b> Eric K. Miller; openstack-operators@lists.openstack.org<br><b>Cc:</b> Ian Banks<br><b>Subject:</b> Re: DHCP timeout when creating instances for specific tenants<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p>Hi Eric,<o:p></o:p></p><p>Thanks for getting back to me. I am fairly sure it is a DHCP error. The instances are getting an IP when they eventually boot, it is just taking a long time for them to bring up networking. The strange thing is, it only seems to be new tenants. All existing tenants are absolutely fine. <o:p></o:p></p><p>I can check DNS as well just to be on the safe side, however I wasn't seeing any errors in the Nova or Neutron logs when the instance(s) were being created.<o:p></o:p></p><p>Regards,<o:p></o:p></p><div><p class=MsoNormal>On 04/12/2019 22:47, Eric K. Miller wrote:<o:p></o:p></p></div><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Grant,</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Are you sure this is a DHCP timeout and not a DNS resolution issue? I ask because we have seen a strange DNS issue occur that can cause something similar.</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Are the VMs being assigned an IP after they finally boot?</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Eric K. Miller</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Genesis Hosting Solutions, LLC</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Try our Genesis Public Cloud - powered by OpenStack!eut</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><a href="https://genesishosting.com/">https://genesishosting.com/</a></span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Grant Morley [<a href="mailto:grant@civo.com">mailto:grant@civo.com</a>] <br><b>Sent:</b> Wednesday, December 04, 2019 11:00 AM<br><b>To:</b> <a href="mailto:openstack-operators@lists.openstack.org">openstack-operators@lists.openstack.org</a><br><b>Cc:</b> Ian Banks<br><b>Subject:</b> DHCP timeout when creating instances for specific tenants</span><o:p></o:p></p></div></div><p class=MsoNormal> <o:p></o:p></p><p>Hi all,<o:p></o:p></p><p>I wonder if anyone can help shed any light on an odd issue we are seeing with only a couple of specific tenants. Basically if they launch an instance they are taking about 5 minutes to launch rather than our usual 30 second or so launch.<o:p></o:p></p><p>We are seeing the following on the instance logs:<o:p></o:p></p><p><a href="https://pastebin.com/hDstsd8G">https://pastebin.com/hDstsd8G</a><o:p></o:p></p><p>Weirdly it only seems to be happening for 1 or 2 new tenants. I have tested this on our personal account and a few other customers have tested and their instances launch really quickly as expected. <o:p></o:p></p><p>Is there anything specific during the tenant creation that can cause this issue? Or are there any logs in nova / neutron I should be looking out for that might shed some light? <o:p></o:p></p><p>I haven't seen anything that is obvious. Any help would be much appreciated as we are a little stumped at the moment.<o:p></o:p></p><p>Many thanks,<o:p></o:p></p><div><p class=MsoNormal>-- <o:p></o:p></p><div><p class=MsoNormal> <o:p></o:p></p></div><div><div><p class=MsoNormal><span style='font-size:8.5pt;font-family:"Verdana","sans-serif";color:#A8D8FF;border:solid windowtext 1.0pt;padding:0in'><img border=0 width=100 height=100 id="_x0000_i1025" src="cid:~WRD374.jpg" alt="Image removed by sender."></span><o:p></o:p></p><div style='margin-left:67.5pt;margin-bottom:3.75pt'><p class=MsoNormal style='mso-line-height-alt:11.25pt'><span style='font-size:13.5pt;font-family:"Verdana","sans-serif";color:#239DFF'>Grant Morley</span><span style='font-size:10.0pt;font-family:"Helvetica","sans-serif";color:#6DBEFF'> </span><o:p></o:p></p></div><div style='margin-left:67.5pt;margin-bottom:7.5pt'><p class=MsoNormal style='mso-line-height-alt:11.25pt'><span style='font-size:11.5pt;font-family:"Verdana","sans-serif";color:#888888'>Cloud Lead, Civo Ltd</span><o:p></o:p></p></div><div style='margin-left:67.5pt'><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Helvetica","sans-serif"'><a href="https://www.civo.com/"><span style='font-family:"Verdana","sans-serif";text-decoration:none'>www.civo.com</span></a></span><span style='font-size:10.0pt;font-family:"Helvetica","sans-serif";color:#888888'> | </span><span style='font-size:10.0pt;font-family:"Helvetica","sans-serif"'><a href="https://www.civo.com/signup"><span style='font-family:"Verdana","sans-serif";text-decoration:none'>Signup for an account!</span></a></span><o:p></o:p></p></div></div></div></div></div></blockquote></div></div></body></html>