<html><head><base href="x-msg://521/"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hey Phil. I think you have a case of old-coditis<div><br></div><div>This was modified to properly query in multihost mode before the essex release:</div><div><br></div><div><div>996 def fixed_ip_disassociate_all_by_timeout(context, host, time):</div><div> 997 session = get_session()</div><div> 998 # NOTE(vish): only update fixed ips that "belong" to this</div><div> 999 # host; i.e. the network host or the instance</div><div>1000 # host matches. Two queries necessary because</div><div>1001 # join with update doesn't work.</div><div>1002 host_filter = or_(and_(models.Instance.host == host,</div><div>1003 models.Network.multi_host == True),</div><div>1004 models.Network.host == host)</div><div>1005 result = session.query(models.FixedIp.id).\</div><div>1006 filter(models.FixedIp.deleted == False).\</div><div>1007 filter(models.FixedIp.allocated == False).\</div><div>1008 filter(models.FixedIp.updated_at < time).\</div><div>1009 join((models.Network,</div><div>1010 models.Network.id == models.FixedIp.network_id)).\</div><div>1011 join((models.Instance,</div><div>1012 models.Instance.id == models.FixedIp.instance_id)).\</div><div>1013 filter(host_filter).\</div><div>1014 all()</div><div>1015 fixed_ip_ids = [fip[0] for fip in result]</div><div>1016 if not fixed_ip_ids:</div><div>1017 return 0</div><div>1018 result = model_query(context, models.FixedIp, session=session).\</div><div>1019 filter(models.FixedIp.id.in_(fixed_ip_ids)).\</div><div>1020 update({'instance_id': None,</div><div>1021 'leased': False,</div><div>1022 'updated_at': utils.utcnow()},</div><div>1023 synchronize_session='fetch')</div><div>1024 return result</div></div><div><br></div><div><br></div><div><div><div>On Apr 27, 2012, at 11:30 AM, Day, Phil wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div lang="EN-GB" link="blue" vlink="purple"><div class="WordSection1" style="page: WordSection1; "><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">Hi Folks,<o:p></o:p></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><o:p> </o:p></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">In multi-host mode the “host” field of a network never seems to get set (as only IPs are allocated, not networks)<o:p></o:p></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><o:p> </o:p></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; ">However the periodic revovery task in NetworkManager uses the host field to filter what addresses it should consider cleaning up (to catch the case where the message from dnsmasq is either never sent or not delivered for some reason)<o:p></o:p></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><o:p> </o:p></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> if self.timeout_fixed_ips:<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> now = utils.utcnow()<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> timeout = FLAGS.fixed_ip_disassociate_timeout<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> time = now - datetime.timedelta(seconds=timeout)<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> num = self.db.fixed_ip_disassociate_all_by_timeout(context,<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> self.host,<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> time)<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> if num:<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> LOG.debug(_('Dissassociated %s stale fixed ip(s)'), num)<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); ">Where “db.fixed_ip_disassociate_all_by_timeout” is:<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); ">def fixed_ip_disassociate_all_by_timeout(_context, host, time):<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> session = get_session()<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> inner_q = session.query(models.Network.id).\<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> filter_by(host=host).\<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> subquery()<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> result = session.query(models.FixedIp).\<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> filter(models.FixedIp.network_id.in_(inner_q)).\<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> filter(models.FixedIp.updated_at < time).\<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> filter(models.FixedIp.instance_id != None).\<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> filter_by(allocated=False).\<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> update({'instance_id': None,<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> 'leased': False,<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> 'updated_at': utils.utcnow()},<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> synchronize_session='fetch')<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "> return result<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); ">So what this seems to do to me is:<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 36pt; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: -18pt; "><span style="color: rgb(31, 73, 125); "><span>-<span style="font: normal normal normal 7pt/normal 'Times New Roman'; "> <span class="Apple-converted-space"> </span></span></span></span><span style="color: rgb(31, 73, 125); ">Find all of the fixed_ips which are:<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 72pt; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: -18pt; "><span style="font-family: 'Courier New'; color: rgb(31, 73, 125); "><span>o<span style="font: normal normal normal 7pt/normal 'Times New Roman'; "> <span class="Apple-converted-space"> </span></span></span></span><span style="color: rgb(31, 73, 125); ">on networks assigned to this host<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 72pt; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: -18pt; "><span style="font-family: 'Courier New'; color: rgb(31, 73, 125); "><span>o<span style="font: normal normal normal 7pt/normal 'Times New Roman'; "> <span class="Apple-converted-space"> </span></span></span></span><span style="color: rgb(31, 73, 125); ">Were last updated more that “Timeout” seconds ago<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 72pt; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: -18pt; "><span style="font-family: 'Courier New'; color: rgb(31, 73, 125); "><span>o<span style="font: normal normal normal 7pt/normal 'Times New Roman'; "> <span class="Apple-converted-space"> </span></span></span></span><span style="color: rgb(31, 73, 125); ">Are associated to an instance<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 72pt; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: -18pt; "><span style="font-family: 'Courier New'; color: rgb(31, 73, 125); "><span>o<span style="font: normal normal normal 7pt/normal 'Times New Roman'; "> <span class="Apple-converted-space"> </span></span></span></span><span style="color: rgb(31, 73, 125); ">Are not allocated<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><o:p> </o:p></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); ">Because in multi-host mode the network host field is always Null, this query does nothing apart from give the DB a good work out every 10 seconds – so there could be a slow leakage of IP addresses.<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); ">Has anyone else spotted this – and if so do you have a good strategy for dealing with it ?<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); ">It seems that running this on every network_manager every 10 seconds is excessive – so what still running on all netwok_managers but using a long random sleep between runs in mult-host mode ?<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); ">Thoughts ?<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); ">Cheers,<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span style="color: rgb(31, 73, 125); ">Phil</span><o:p></o:p></div></div></div></blockquote></div><br></div></body></html>