Open Stack

Hi Daniel,

I'm concerned that we are encountered out-of-order port events on the DHCP
agent side so the delete message is processed before the create message.
Would you be willing to apply a small patch to your dhcp agent to see if it
fixes the issue?

If it does fix the issue, you should see occasional warnings in the DHCP
agent log that show "Received message for port that was already deleted".
If it doesn't fix the issue, we may be losing the delete event entirely. If
that's the case, it would be great if you can enable debuging on the agent
and upload a log of a run when it happens.

Cheers,
Kevin Benton

Here is the patch:

diff --git a/neutron/agent/dhcp_agent.py b/neutron/agent/dhcp_agent.py
index 71c9709..9b9b637 100644
--- a/neutron/agent/dhcp_agent.py
+++ b/neutron/agent/dhcp_agent.py
@@ -71,6 +71,7 @@ class DhcpAgent(manager.Manager):
         self.needs_resync = False
         self.conf = cfg.CONF
         self.cache = NetworkCache()
+        self.deleted_ports = set()
         self.root_helper = config.get_root_helper(self.conf)
         self.dhcp_driver_cls =
importutils.import_class(self.conf.dhcp_driver)
         ctx = context.get_admin_context_without_session()
@@ -151,6 +152,7 @@ class DhcpAgent(manager.Manager):
         LOG.info(_('Synchronizing state'))
         pool = eventlet.GreenPool(cfg.CONF.num_sync_threads)
         known_network_ids = set(self.cache.get_network_ids())
+        self.deleted_ports = set()

         try:
             active_networks = self.plugin_rpc.get_active_networks_info()
@@ -302,6 +304,10 @@ class DhcpAgent(manager.Manager):
     @utils.synchronized('dhcp-agent')
     def port_update_end(self, context, payload):
         """Handle the port.update.end notification event."""
+        if payload['port']['id'] in self.deleted_ports:
+            LOG.warning(_("Received message for port that was "
+                          "already deleted: %s"), payload['port']['id'])
+            return
         updated_port = dhcp.DictModel(payload['port'])
         network = self.cache.get_network_by_id(updated_port.network_id)
         if network:
@@ -315,6 +321,7 @@ class DhcpAgent(manager.Manager):
     def port_delete_end(self, context, payload):
         """Handle the port.delete.end notification event."""
         port = self.cache.get_port_by_id(payload['port_id'])
+        self.deleted_ports.add(payload['port_id'])
         if port:
             network = self.cache.get_network_by_id(port.network_id)
             self.cache.remove_port(port)








On Mon, Jun 8, 2015 at 8:26 AM, Daniel Comnea <comnea.dani at gmail.com> wrote:

> Any help, ideas please?
>
> Thx,
> Dani
>
> On Mon, Jun 8, 2015 at 9:25 AM, Daniel Comnea <comnea.dani at gmail.com>
> wrote:
>
>> + Operators
>>
>> Much thanks in advance,
>> Dani
>>
>>
>>
>>
>> On Sun, Jun 7, 2015 at 6:31 PM, Daniel Comnea <comnea.dani at gmail.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> I'm running IceHouse (build using Fuel 5.1.1) on Ubuntu where dnsmask
>>> version 2.59-4.
>>> I have a very basic network layout where i have a private net which has
>>> 2 subnets
>>>
>>>  2fb7de9d-d6df-481f-acca-2f7860cffa60 | private-net
>>>                  | e79c3477-d3e5-471c-a728-8d881cf31bee 192.168.110.0/24
>>> |
>>> |
>>> |                                                         |
>>> f48c3223-8507-455c-9c13-8b727ea5f441 192.168.111.0/24 |
>>>
>>> and i'm creating VMs via HEAT.
>>> What is happening is that sometimes i get duplicated entries in [1] and
>>> because of that the VM which was spun up doesn't get an ip.
>>> The Dnsmask processes are running okay [2] and i can't see anything
>>> special/ wrong in it.
>>>
>>> Any idea why this is happening? Or are you aware of any bugs around this
>>> area? Do you see a problems with having 2 subnets mapped to 1 private-net?
>>>
>>>
>>>
>>> Thanks,
>>> Dani
>>>
>>> [1]
>>> /var/lib/neutron/dhcp/2fb7de9d-d6df-481f-acca-2f7860cffa60/addn_hosts
>>> [2]
>>>
>>> nobody    5664     1  0 Jun02 ?        00:00:08 dnsmasq --no-hosts
>>> --no-resolv --strict-order --bind-interfaces --interface=tapc9164734-0c
>>> --except-interface=lo
>>> --pid-file=/var/lib/neutron/dhcp/2fb7de9d-d6df-481f-acca-2f7860cffa60/pid
>>> --dhcp-hostsfile=/var/lib/neutron/dhcp/2fb7de9d-d6df-481f-acca-2f7860cffa60/host
>>> --addn-hosts=/var/lib/neutron/dhcp/2fb7de9d-d6df-481f-acca-2f7860cffa60/addn_hosts
>>> --dhcp-optsfile=/var/lib/neutron/dhcp/2fb7de9d-d6df-481f-acca-2f7860cffa60/opts
>>> --leasefile-ro --dhcp-authoritative
>>> --dhcp-range=set:tag0,192.168.110.0,static,86400s
>>> --dhcp-range=set:tag1,192.168.111.0,static,86400s --dhcp-lease-max=512
>>> --conf-file= --server=10.0.0.31 --server=10.0.0.32 --domain=openstacklocal
>>>
>>>
>>
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>


-- 
Kevin Benton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20150608/64e13a87/attachment-0001.html>