<div dir="ltr">This sounds like the "select; if result: update; else: insert" race condition and subsequent deadlock issue. We talked at length about it in the db meeting yesterday. I've tagged this bug as "db" related.<br>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Feb 7, 2013 at 7:43 AM, rezroo <span dir="ltr"><<a href="mailto:reza@dslextreme.com" target="_blank">reza@dslextreme.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>Ok - thanks. I reported a bug:<br>
<br>
<a href="https://bugs.launchpad.net/nova/+bug/1118457" target="_blank">https://bugs.launchpad.net/nova/+bug/1118457</a><br>
<br>
Rez<div><div class="h5"><br>
<br>
On 02/06/2013 10:58 PM, Vishvananda Ishaya wrote:<br>
</div></div></div><div><div class="h5">
<blockquote type="cite">
<p dir="ltr">Going from memory of the code I believe this is
possible. Iirc the lock is a select for update so assuming
innodb it will prevent two workers from assigning the same
network to multiple tenants but won't prevent two networks being
assigned to the same tenant. This is probably an untested
consequence of clan plus multi host. A decent workaround would
be to assign a network to a tenant immediately after creation
instead of lazily.</p>
<div class="gmail_quote">On Feb 5, 2013 12:56 PM, "rezroo" <<a href="mailto:reza@dslextreme.com" target="_blank">reza@dslextreme.com</a>>
wrote:<br type="attribution">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> I've run into this
issue and I'd like to get some opinions before logging a
defect. I'm using folsom, deployed using VlanManager. It
seems to me that the assignment of a network to a tenant is
lazy evaluated until the first instance for the project is
launched. This can be confirmed by creating a new tenant
and then doing 'nova-manage network list'. If one now
launches multiple tenants by setting the number of instances
to 4 or 5 then there is a race condition and it is possible
that two or more vlan/networks are assigned to the tenant.
I'm running into this issue.<br>
<br>
I posted to Openstack-operators already and I got one reply
back claiming that this is a configuration issue, without
any insight as to what is misconfigured. Do you know what
kind of misconfiguration leads to this? Or what
concurrency/locking mechanism prevents this from happening?
Presumably a select is done in the networks table to see if
a vlan is already assigned to a tenant. If you have
sufficiently fast environment it is possible that two
simultaneous queries are done, both with no results, and two
networks are assigned and committed to the database. This
is what happened for me. What misconfiguration should I
look for, or what concurrency/locking mechanism is there to
prevent this?
<div><br>
Or I can log a defect.<br>
<br>
Thanks,<br>
Rez<br>
<br>
-------- Original Message --------
<table border="0" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<th align="RIGHT" nowrap valign="BASELINE">Subject:
</th>
<td>Re: [Openstack-operators] Simultaneous launch of
multiple instances from horizon leads to multiple
LANs assigned to new tenant</td>
</tr>
<tr>
<th align="RIGHT" nowrap valign="BASELINE">Date:
</th>
<td>Tue, 05 Feb 2013 05:41:37 -0800</td>
</tr>
<tr>
<th align="RIGHT" nowrap valign="BASELINE">From:
</th>
<td>rezroo <a href="mailto:reza@dslextreme.com" target="_blank"><reza@dslextreme.com></a></td>
</tr>
<tr>
<th align="RIGHT" nowrap valign="BASELINE">To:
</th>
<td>Vivek Singh Raghuwanshi <a href="mailto:vivekraghuwanshi@gmail.com" target="_blank"><vivekraghuwanshi@gmail.com></a></td>
</tr>
</tbody>
</table>
<br>
<br>
<div>Still using nova-network due to its better
integration with Horizon.<br>
Do you know what kind of misconfiguration leads to this?
Or what concurrency/locking mechanism prevents this from
happening? So presumably a select is done in the
networks table to see if a vlan is already assigned to a
tenant. If you have sufficiently fast environment it is
possible that two simultaneous queries are done, both
with no results, and two networks are assigned and
committed to the database. This is what happened for
me. What misconfiguration should I look for, or what
concurrency/locking mechanism is there to prevent this?<br>
Thanks,<br>
Rez<br>
<br>
On 02/04/2013 11:03 PM, Vivek Singh Raghuwanshi wrote:<br>
</div>
<blockquote type="cite">This only happens due to
misconfiguration, but are you still with nova-network.<br>
its advisable to switch your cloud with Quantum<br>
<br>
<div class="gmail_quote">On Tue, Feb 5, 2013 at 5:05 AM,
rezroo <span dir="ltr"><<a href="mailto:reza@dslextreme.com" target="_blank">reza@dslextreme.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi
- I'm wondering if others have seen this defect and
if it has already been reported or not.<br>
I'm using folsom, deployed using VlanManager.<br>
When a new tenant is created no vlan/network is
assigned to it until the first instance for the
project is launched. This can be confirmed by
creating a new tenant and then doing 'nova-manage
network list'.<br>
If now one launches multiple tenants by setting the
number of instances to 4 or 5 then there is a race
condition and it is possible that two or more
vlan/networks are assigned to the tenant.<br>
Should I log this as a defect?<br>
Thanks,<br>
Rez<br>
<br>
_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><br>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
ViVek Raghuwanshi<br>
Mobile -<a href="tel:%2B91-09595950504" value="+919595950504" target="_blank">+91-09595950504</a><br>
Skype - vivek_raghuwanshi<br>
IRC vivekraghuwanshi<br>
<a href="http://vivekraghuwanshi.wordpress.com/" target="_blank">http://vivekraghuwanshi.wordpress.com/</a><br>
<a href="http://in.linkedin.com/in/vivekraghuwanshi" target="_blank">http://in.linkedin.com/in/vivekraghuwanshi</a><br>
</blockquote>
<br>
<br>
</div>
<br>
</div>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</blockquote>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
OpenStack-dev mailing list
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
</div></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>