DHCP timeout when creating instances for specific tenants
Grant Morley
grant at civo.com
Wed Dec 4 23:53:17 UTC 2019
HI Eric,
We are indeed already setting the DNS servers explicitly in the subnet
so I don't think that is the issue ( from what I can tell ) I was
wondering if it could be an issue with Neutron not responding in time
for the DHCP request from the instance, however I haven't yet found any
evidence of this.
The only other thought I had was that it could be an issue with RabbitMQ
somehow and potentially increasing the "rpc timeout" on neutron to see
if that helps as I have seen some errors stating that RabbitMQ didn't
respond to a message request in time. However I think it could be a red
herring as I would assume if RabbitMQ was to blame, existing tenants and
instances would also be suffering.
Grant
On 04/12/2019 23:46, Eric K. Miller wrote:
>
> Hi Grant,
>
> I didn't see any DNS errors either. The solution was to explicitly
> configure the dns servers in the subnet. Are you doing this already?
> Or are you relying on the dnsmasq processes created for the router to
> respond to DNS queries (and forward them respectively)?
>
>
> Eric
>
> *From:*Grant Morley [mailto:grant at civo.com]
> *Sent:* Wednesday, December 04, 2019 5:40 PM
> *To:* Eric K. Miller; openstack-operators at lists.openstack.org
> *Cc:* Ian Banks
> *Subject:* Re: DHCP timeout when creating instances for specific tenants
>
> Hi Eric,
>
> Thanks for getting back to me. I am fairly sure it is a DHCP error.
> The instances are getting an IP when they eventually boot, it is just
> taking a long time for them to bring up networking. The strange thing
> is, it only seems to be new tenants. All existing tenants are
> absolutely fine.
>
> I can check DNS as well just to be on the safe side, however I wasn't
> seeing any errors in the Nova or Neutron logs when the instance(s)
> were being created.
>
> Regards,
>
> On 04/12/2019 22:47, Eric K. Miller wrote:
>
> Hi Grant,
>
> Are you sure this is a DHCP timeout and not a DNS resolution
> issue? I ask because we have seen a strange DNS issue occur that
> can cause something similar.
>
> Are the VMs being assigned an IP after they finally boot?
>
> Eric K. Miller
>
> Genesis Hosting Solutions, LLC
>
> Try our Genesis Public Cloud - powered by OpenStack!eut
>
> https://genesishosting.com/
>
> *From:*Grant Morley [mailto:grant at civo.com]
> *Sent:* Wednesday, December 04, 2019 11:00 AM
> *To:* openstack-operators at lists.openstack.org
> <mailto:openstack-operators at lists.openstack.org>
> *Cc:* Ian Banks
> *Subject:* DHCP timeout when creating instances for specific tenants
>
> Hi all,
>
> I wonder if anyone can help shed any light on an odd issue we are
> seeing with only a couple of specific tenants. Basically if they
> launch an instance they are taking about 5 minutes to launch
> rather than our usual 30 second or so launch.
>
> We are seeing the following on the instance logs:
>
> https://pastebin.com/hDstsd8G
>
> Weirdly it only seems to be happening for 1 or 2 new tenants. I
> have tested this on our personal account and a few other customers
> have tested and their instances launch really quickly as expected.
>
> Is there anything specific during the tenant creation that can
> cause this issue? Or are there any logs in nova / neutron I should
> be looking out for that might shed some light?
>
> I haven't seen anything that is obvious. Any help would be much
> appreciated as we are a little stumped at the moment.
>
> Many thanks,
>
> --
>
> Image removed by sender.
>
> Grant Morley
>
> Cloud Lead, Civo Ltd
>
> www.civo.com <https://www.civo.com/>| Signup for an account!
> <https://www.civo.com/signup>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20191204/1b532b34/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ~WRD374.jpg
Type: image/jpeg
Size: 823 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20191204/1b532b34/attachment-0001.jpg>
More information about the openstack-discuss
mailing list