Open Stack

This sounds like the "select; if result: update; else: insert" race
condition and subsequent deadlock issue. We talked at length about it in
the db meeting yesterday. I've tagged this bug as "db" related.


On Thu, Feb 7, 2013 at 7:43 AM, rezroo <reza at dslextreme.com> wrote:

>  Ok - thanks.  I reported a bug:
>
> https://bugs.launchpad.net/nova/+bug/1118457
>
> Rez
>
>
> On 02/06/2013 10:58 PM, Vishvananda Ishaya wrote:
>
> Going from memory of the code I believe this is possible. Iirc the lock is
> a select for update so assuming innodb it will prevent two workers from
> assigning the same network to multiple tenants but won't prevent two
> networks being assigned to the same tenant. This is probably an untested
> consequence of clan plus multi host. A decent workaround would be to assign
> a network to a tenant immediately after creation instead of lazily.
> On Feb 5, 2013 12:56 PM, "rezroo" <reza at dslextreme.com> wrote:
>
>>  I've run into this issue and I'd like to get some opinions before
>> logging a defect. I'm using folsom, deployed using VlanManager.  It seems
>> to me that the assignment of a network to a tenant is lazy evaluated until
>> the first instance for the project is launched.  This can be confirmed by
>> creating a new tenant and then doing 'nova-manage network list'. If one now
>> launches multiple tenants by setting the number of instances to 4 or 5 then
>> there is a race condition and it is possible that two or more vlan/networks
>> are assigned to the tenant. I'm running into this issue.
>>
>> I posted to Openstack-operators already and I got one reply back claiming
>> that this is a configuration issue, without any insight as to what is
>> misconfigured. Do you know what kind of misconfiguration leads to this? Or
>> what concurrency/locking mechanism prevents this from happening?
>> Presumably a select is done in the networks table to see if a vlan is
>> already assigned to a tenant.  If you have sufficiently fast environment it
>> is possible that two simultaneous queries are done, both with no results,
>> and two networks are assigned and committed to the database.  This is what
>> happened for me.  What misconfiguration should I look for, or what
>> concurrency/locking mechanism is there to prevent this?
>>
>> Or I can log a defect.
>>
>> Thanks,
>> Rez
>>
>> -------- Original Message --------  Subject: Re: [Openstack-operators]
>> Simultaneous launch of multiple instances from horizon leads to multiple
>> LANs assigned to new tenant  Date: Tue, 05 Feb 2013 05:41:37 -0800  From:
>> rezroo <reza at dslextreme.com> <reza at dslextreme.com>  To: Vivek Singh
>> Raghuwanshi <vivekraghuwanshi at gmail.com> <vivekraghuwanshi at gmail.com>
>>
>> Still using nova-network due to its better integration with Horizon.
>> Do you know what kind of misconfiguration leads to this? Or what
>> concurrency/locking mechanism prevents this from happening?  So presumably
>> a select is done in the networks table to see if a vlan is already assigned
>> to a tenant.  If you have sufficiently fast environment it is possible that
>> two simultaneous queries are done, both with no results, and two networks
>> are assigned and committed to the database.  This is what happened for me.
>> What misconfiguration should I look for, or what concurrency/locking
>> mechanism is there to prevent this?
>> Thanks,
>> Rez
>>
>> On 02/04/2013 11:03 PM, Vivek Singh Raghuwanshi wrote:
>>
>> This only happens due to misconfiguration, but are you still with
>> nova-network.
>> its advisable to switch your  cloud with Quantum
>>
>> On Tue, Feb 5, 2013 at 5:05 AM, rezroo <reza at dslextreme.com> wrote:
>>
>>> Hi - I'm wondering if others have seen this defect and if it has already
>>> been reported or not.
>>> I'm using folsom, deployed using VlanManager.
>>> When a new tenant is created no vlan/network is assigned to it until the
>>> first instance for the project is launched.  This can be confirmed by
>>> creating a new tenant and then doing 'nova-manage network list'.
>>> If now one launches multiple tenants by setting the number of instances
>>> to 4 or 5 then there is a race condition and it is possible that two or
>>> more vlan/networks are assigned to the tenant.
>>> Should I log this as a defect?
>>> Thanks,
>>> Rez
>>>
>>> _______________________________________________
>>> OpenStack-operators mailing list
>>> OpenStack-operators at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>>
>>
>>
>>
>> --
>> ViVek Raghuwanshi
>> Mobile -+91-09595950504
>> Skype - vivek_raghuwanshi
>> IRC vivekraghuwanshi
>> http://vivekraghuwanshi.wordpress.com/
>> http://in.linkedin.com/in/vivekraghuwanshi
>>
>>
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> _______________________________________________
> OpenStack-dev mailing listOpenStack-dev at lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130208/da3d8ff4/attachment.html>