[Openstack-operators] [openstack-dev] [openstack-operators][neutron[dhcp][dnsmask]: duplicate entries in addn_hosts causing no IP allocation
Neil Jerram
Neil.Jerram at metaswitch.com
Wed Jul 1 08:23:38 UTC 2015
Well, the bug discussion seems to point specifically to this dnsmasq fix:
http://thekelleys.org.uk/gitweb/?p=dnsmasq.git;a=commit;h=9380ba70d67db6b69f817d8e318de5ba1e990b12
Neil
On 01/07/15 07:34, Daniel Comnea wrote:
> Hi,
>
> sorry for no feedback, i've been doing more and more test and after
> enabled the dnsmasq log i found the error which i'm not longer sure if
> is related to having duplicated entries
>
> dnsmasq-dhcp[21231]: 0 DHCPRELEASE(tap8ecf66b6-72) 192.168.111.24
> fa:16:3e:72:04:82 unknown lease
>
> Looking around it seems i'm hitting this bug [1] but not clear from the
> description what was the problem on dnsmasp 2.59 (which comes wiht Fuel 5.1)
>
> Any ideas?
>
> Cheers,
> Dani
>
> [1] https://bugs.launchpad.net/neutron/+bug/1271344
>
> On Wed, Jun 10, 2015 at 7:13 AM, Daniel Comnea <comnea.dani at gmail.com
> <mailto:comnea.dani at gmail.com>> wrote:
>
> Thanks a bunch Kevin!
>
> I'll try this patch and report back.
>
> Dani
>
>
> On Tue, Jun 9, 2015 at 2:50 AM, Kevin Benton <blak111 at gmail.com
> <mailto:blak111 at gmail.com>> wrote:
>
> Hi Daniel,
>
> I'm concerned that we are encountered out-of-order port events
> on the DHCP agent side so the delete message is processed before
> the create message. Would you be willing to apply a small patch
> to your dhcp agent to see if it fixes the issue?
>
> If it does fix the issue, you should see occasional warnings in
> the DHCP agent log that show "Received message for port that was
> already deleted". If it doesn't fix the issue, we may be losing
> the delete event entirely. If that's the case, it would be great
> if you can enable debuging on the agent and upload a log of a
> run when it happens.
>
> Cheers,
> Kevin Benton
>
> Here is the patch:
>
> diff --git a/neutron/agent/dhcp_agent.py
> b/neutron/agent/dhcp_agent.py
> index 71c9709..9b9b637 100644
> --- a/neutron/agent/dhcp_agent.py
> +++ b/neutron/agent/dhcp_agent.py
> @@ -71,6 +71,7 @@ class DhcpAgent(manager.Manager):
> self.needs_resync = False
> self.conf = cfg.CONF
> self.cache = NetworkCache()
> + self.deleted_ports = set()
> self.root_helper = config.get_root_helper(self.conf)
> self.dhcp_driver_cls =
> importutils.import_class(self.conf.dhcp_driver)
> ctx = context.get_admin_context_without_session()
> @@ -151,6 +152,7 @@ class DhcpAgent(manager.Manager):
> LOG.info(_('Synchronizing state'))
> pool = eventlet.GreenPool(cfg.CONF.num_sync_threads)
> known_network_ids = set(self.cache.get_network_ids())
> + self.deleted_ports = set()
>
> try:
> active_networks =
> self.plugin_rpc.get_active_networks_info()
> @@ -302,6 +304,10 @@ class DhcpAgent(manager.Manager):
> @utils.synchronized('dhcp-agent')
> def port_update_end(self, context, payload):
> """Handle the port.update.end notification event."""
> + if payload['port']['id'] in self.deleted_ports:
> + LOG.warning(_("Received message for port that was "
> + "already deleted: %s"),
> payload['port']['id'])
> + return
> updated_port = dhcp.DictModel(payload['port'])
> network =
> self.cache.get_network_by_id(updated_port.network_id)
> if network:
> @@ -315,6 +321,7 @@ class DhcpAgent(manager.Manager):
> def port_delete_end(self, context, payload):
> """Handle the port.delete.end notification event."""
> port = self.cache.get_port_by_id(payload['port_id'])
> + self.deleted_ports.add(payload['port_id'])
> if port:
> network =
> self.cache.get_network_by_id(port.network_id)
> self.cache.remove_port(port)
>
>
>
>
>
>
>
>
> On Mon, Jun 8, 2015 at 8:26 AM, Daniel Comnea
> <comnea.dani at gmail.com <mailto:comnea.dani at gmail.com>> wrote:
>
> Any help, ideas please?
>
> Thx,
> Dani
>
> On Mon, Jun 8, 2015 at 9:25 AM, Daniel Comnea
> <comnea.dani at gmail.com <mailto:comnea.dani at gmail.com>> wrote:
>
> + Operators
>
> Much thanks in advance,
> Dani
>
>
>
>
> On Sun, Jun 7, 2015 at 6:31 PM, Daniel Comnea
> <comnea.dani at gmail.com <mailto:comnea.dani at gmail.com>>
> wrote:
>
> Hi all,
>
> I'm running IceHouse (build using Fuel 5.1.1) on
> Ubuntu where dnsmask version 2.59-4.
> I have a very basic network layout where i have a
> private net which has 2 subnets
>
> 2fb7de9d-d6df-481f-acca-2f7860cffa60 |
> private-net |
> e79c3477-d3e5-471c-a728-8d881cf31bee
> 192.168.110.0/24 <http://192.168.110.0/24> |
> |
> |
> |
> f48c3223-8507-455c-9c13-8b727ea5f441
> 192.168.111.0/24 <http://192.168.111.0/24> |
>
> and i'm creating VMs via HEAT.
> What is happening is that sometimes i get duplicated
> entries in [1] and because of that the VM which was
> spun up doesn't get an ip.
> The Dnsmask processes are running okay [2] and i
> can't see anything special/ wrong in it.
>
> Any idea why this is happening? Or are you aware of
> any bugs around this area? Do you see a problems
> with having 2 subnets mapped to 1 private-net?
>
>
>
> Thanks,
> Dani
>
> [1]
> /var/lib/neutron/dhcp/2fb7de9d-d6df-481f-acca-2f7860cffa60/addn_hosts
>
> [2]
>
> nobody 5664 1 0 Jun02 ? 00:00:08
> dnsmasq --no-hosts --no-resolv --strict-order
> --bind-interfaces --interface=tapc9164734-0c
> --except-interface=lo
> --pid-file=/var/lib/neutron/dhcp/2fb7de9d-d6df-481f-acca-2f7860cffa60/pid
> --dhcp-hostsfile=/var/lib/neutron/dhcp/2fb7de9d-d6df-481f-acca-2f7860cffa60/host
> --addn-hosts=/var/lib/neutron/dhcp/2fb7de9d-d6df-481f-acca-2f7860cffa60/addn_hosts
> --dhcp-optsfile=/var/lib/neutron/dhcp/2fb7de9d-d6df-481f-acca-2f7860cffa60/opts
> --leasefile-ro --dhcp-authoritative
> --dhcp-range=set:tag0,192.168.110.0,static,86400s
> --dhcp-range=set:tag1,192.168.111.0,static,86400s
> --dhcp-lease-max=512 --conf-file= --server=10.0.0.31
> --server=10.0.0.32 --domain=openstacklocal
>
>
>
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> <mailto:OpenStack-operators at lists.openstack.org>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
>
>
> --
> Kevin Benton
>
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
More information about the OpenStack-operators
mailing list