DHCP timeout when creating instances for specific tenants

Cory Hawkless Cory at hawkless.id.au
Wed Dec 4 23:49:35 UTC 2019


Are they failing to contact the metadata service and hanging during the boot process while they try and receive metadata?
From the VM can you hit http://169.254.169.254 – That’s the default IP of the metadata server, it should respond with a basic page showing some date based subdirectories
If it doesn’t respond you can start following the metadata service path instead of DHCP
Given that the machines come up with an IP eventually leads me to think the DHCP service is actually working ok.

From: Grant Morley [mailto:grant at civo.com]
Sent: Thursday, 5 December 2019 10:10 AM
To: Eric K. Miller <emiller at genesishosting.com>; openstack-operators at lists.openstack.org
Cc: Ian Banks <ian at civo.com>
Subject: Re: DHCP timeout when creating instances for specific tenants


Hi Eric,

Thanks for getting back to me. I am fairly sure it is a DHCP error. The instances are getting an IP when they eventually boot, it is just taking a long time for them to bring up networking. The strange thing is, it only seems to be new tenants. All existing tenants are absolutely fine.

I can check DNS as well just to be on the safe side, however I wasn't seeing any errors in the Nova or Neutron logs when the instance(s) were being created.

Regards,
On 04/12/2019 22:47, Eric K. Miller wrote:
Hi Grant,

Are you sure this is a DHCP timeout and not a DNS resolution issue?  I ask because we have seen a strange DNS issue occur that can cause something similar.

Are the VMs being assigned an IP after they finally boot?

Eric K. Miller
Genesis Hosting Solutions, LLC
Try our Genesis Public Cloud - powered by OpenStack!eut
https://genesishosting.com/



From: Grant Morley [mailto:grant at civo.com]
Sent: Wednesday, December 04, 2019 11:00 AM
To: openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>
Cc: Ian Banks
Subject: DHCP timeout when creating instances for specific tenants


Hi all,

I wonder if anyone can help shed any light on an odd issue we are seeing with only a couple of specific tenants. Basically if they launch an instance they are taking about 5 minutes to launch rather than our usual 30 second or so launch.

We are seeing the following on the instance logs:

https://pastebin.com/hDstsd8G

Weirdly it only seems to be happening for 1 or 2 new tenants. I have tested this on our personal account and a few other customers have tested and their instances launch really quickly as expected.

Is there anything specific during the tenant creation that can cause this issue? Or are there any logs in nova / neutron I should be looking out for that might shed some light?

I haven't seen anything that is obvious. Any help would be much appreciated as we are a little stumped at the moment.

Many thanks,
--

[https://www.civo.com/images/email-logo.jpg]
Grant Morley
Cloud Lead, Civo Ltd
www.civo.com<https://www.civo.com/> | Signup for an account!<https://www.civo.com/signup>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20191204/fd4c3904/attachment.html>


More information about the openstack-discuss mailing list