<div dir="ltr">Hi Daniel,<div><br></div><div>I'm concerned that we are encountered out-of-order port events on the DHCP agent side so the delete message is processed before the create message. Would you be willing to apply a small patch to your dhcp agent to see if it fixes the issue?</div><div><br></div><div>If it does fix the issue, you should see occasional warnings in the DHCP agent log that show "Received message for port that was already deleted". If it doesn't fix the issue, we may be losing the delete event entirely. If that's the case, it would be great if you can enable debuging on the agent and upload a log of a run when it happens.</div><div><br></div><div>Cheers,</div><div>Kevin Benton</div><div><br></div><div>Here is the patch:</div><div><br></div><div><div>diff --git a/neutron/agent/dhcp_agent.py b/neutron/agent/dhcp_agent.py</div><div>index 71c9709..9b9b637 100644</div><div>--- a/neutron/agent/dhcp_agent.py</div><div>+++ b/neutron/agent/dhcp_agent.py</div><div>@@ -71,6 +71,7 @@ class DhcpAgent(manager.Manager):</div><div> self.needs_resync = False</div><div> self.conf = cfg.CONF</div><div> self.cache = NetworkCache()</div><div>+ self.deleted_ports = set()</div><div> self.root_helper = config.get_root_helper(self.conf)</div><div> self.dhcp_driver_cls = importutils.import_class(self.conf.dhcp_driver)</div><div> ctx = context.get_admin_context_without_session()</div><div>@@ -151,6 +152,7 @@ class DhcpAgent(manager.Manager):</div><div> LOG.info(_('Synchronizing state'))</div><div> pool = eventlet.GreenPool(cfg.CONF.num_sync_threads)</div><div> known_network_ids = set(self.cache.get_network_ids())</div><div>+ self.deleted_ports = set()</div><div><br></div><div> try:</div><div> active_networks = self.plugin_rpc.get_active_networks_info()</div><div>@@ -302,6 +304,10 @@ class DhcpAgent(manager.Manager):</div><div> @utils.synchronized('dhcp-agent')</div><div> def port_update_end(self, context, payload):</div><div> """Handle the port.update.end notification event."""</div><div>+ if payload['port']['id'] in self.deleted_ports:</div><div>+ LOG.warning(_("Received message for port that was "</div><div>+ "already deleted: %s"), payload['port']['id'])</div><div>+ return</div><div> updated_port = dhcp.DictModel(payload['port'])</div><div> network = self.cache.get_network_by_id(updated_port.network_id)</div><div> if network:</div><div>@@ -315,6 +321,7 @@ class DhcpAgent(manager.Manager):</div><div> def port_delete_end(self, context, payload):</div><div> """Handle the port.delete.end notification event."""</div><div> port = self.cache.get_port_by_id(payload['port_id'])</div><div>+ self.deleted_ports.add(payload['port_id'])</div><div> if port:</div><div> network = self.cache.get_network_by_id(port.network_id)</div><div> self.cache.remove_port(port)</div></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jun 8, 2015 at 8:26 AM, Daniel Comnea <span dir="ltr"><<a href="mailto:comnea.dani@gmail.com" target="_blank">comnea.dani@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div>Any help, ideas please?<br><br></div>Thx,<br></div>Dani<br></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jun 8, 2015 at 9:25 AM, Daniel Comnea <span dir="ltr"><<a href="mailto:comnea.dani@gmail.com" target="_blank">comnea.dani@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div>+ Operators <br><br></div>Much thanks in advance,<br></div>Dani<div><div><br><br><br><div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Jun 7, 2015 at 6:31 PM, Daniel Comnea <span dir="ltr"><<a href="mailto:comnea.dani@gmail.com" target="_blank">comnea.dani@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div><div>Hi all,<br><br></div>I'm running IceHouse (build using Fuel 5.1.1) on Ubuntu where dnsmask version 2.59-4.<br></div>I have a very basic network layout where i have a private net which has 2 subnets<br><br> 2fb7de9d-d6df-481f-acca-2f7860cffa60 | private-net | e79c3477-d3e5-471c-a728-8d881cf31bee <a href="http://192.168.110.0/24" target="_blank">192.168.110.0/24</a> |<br>| | | f48c3223-8507-455c-9c13-8b727ea5f441 <a href="http://192.168.111.0/24" target="_blank">192.168.111.0/24</a> |<br><br></div>and i'm creating VMs via HEAT.<br></div>What is happening is that sometimes i get duplicated entries in [1] and because of that the VM which was spun up doesn't get an ip.<br></div>The Dnsmask processes are running okay [2] and i can't see anything special/ wrong in it.<br><br></div>Any idea why this is happening? Or are you aware of any bugs around this area? Do you see a problems with having 2 subnets mapped to 1 private-net?<br><br><br><br></div>Thanks,<br></div>Dani<br><div><div><div><div><div><br>[1] /var/lib/neutron/dhcp/2fb7de9d-d6df-481f-acca-2f7860cffa60/addn_hosts <br>[2] <br><br>nobody 5664 1 0 Jun02 ? 00:00:08 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tapc9164734-0c --except-interface=lo --pid-file=/var/lib/neutron/dhcp/2fb7de9d-d6df-481f-acca-2f7860cffa60/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/2fb7de9d-d6df-481f-acca-2f7860cffa60/host --addn-hosts=/var/lib/neutron/dhcp/2fb7de9d-d6df-481f-acca-2f7860cffa60/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/2fb7de9d-d6df-481f-acca-2f7860cffa60/opts --leasefile-ro --dhcp-authoritative --dhcp-range=set:tag0,192.168.110.0,static,86400s --dhcp-range=set:tag1,192.168.111.0,static,86400s --dhcp-lease-max=512 --conf-file= --server=10.0.0.31 --server=10.0.0.32 --domain=openstacklocal<br><br></div></div></div></div></div></div>
</blockquote></div><br></div></div></div></div></div></div></div>
</blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div><div>Kevin Benton</div></div>
</div></div>