Open Stack

>What is left to be done in opt.2 is to handle situation when object is
trying to update the state and the revision is not the latest.


Once the revision counter is added, this will generate an error (StaleData
I think) that will be caught by the retry decorator so the process can be
started over by fetching the latest state.

On Fri, May 13, 2016 at 1:22 AM, Korzeniewski, Artur <
artur.korzeniewski at intel.com> wrote:

> Hi Ilya,
> Thanks for investigating the concurrency issue. It is indeed a problem
> from multithreaded neutron-server point of view.
>
> Regarding the opt.2, there is ongoing work to add the revision number[1]
> to main neutron resources (port, network, subnet...) This is connected to
> spec [2] and Launchpad bug [3]: push all object changes as AMQP
> notifications. I think [1] is implementing part of your idea about version
> number. If version number will be added to standard attributes table, it
> will automagically appear as a new field in object.
>
> What is left to be done in opt.2 is to handle situation when object is
> trying to update the state and the revision is not the latest. Then, the
> object should before update() fetch the current state and try to merge the
> requested changes to the current state of the object, and then apply it in
> the DB. Also it would send the newest object state via AMQP notification,
> obsoleting the previous configuration.
>
> We can also check how nova is handling the concurrency issue in their OVO
> usage model.
>
> Regards,
> Artur
> IRC: korzen
>
> [1] https://review.openstack.org/#/c/303966/11
> [2] https://review.openstack.org/#/c/225995
> [3] https://bugs.launchpad.net/neutron/+bug/1516195
>
> -----Original Message-----
> From: Ilya Chukhnakov [mailto:ichukhnakov at mirantis.com]
> Sent: Thursday, May 12, 2016 8:26 PM
> To: OpenStack Development Mailing List <openstack-dev at lists.openstack.org>
> Subject: [openstack-dev] [neutron][ovo] NeutronDbObject concurrency issues
>
> Hi everyone.
>
> I’ve recently found that straightforward use of NeutronDbObject is prone
> to concurrency-related problems.
>
> I’ve submitted a patch set [3] with some tests to show that without
> special treatment using NeutronDbObject could lead to unexpected results.
>
> Further patch sets will provide acquire_object/acquire_objects
> contextmanager methods to the NeutronDbObject class. These methods are to
> be used in place of get_object/get_objects whenever the user intends to
> make changes to the object.
> These methods would start an autonested_transaction.
>
> There are (at least) two potential options for the implementation:
>
> 1. Based on the DB locks (e.g. SELECT FOR UPDATE/SqlAlchemy
> with_for_update).
>
>    pros:
>      - the object is guaranteed to not be changed while within the context
>
>    cons:
>      - prone to deadlocks ([1] and potentially when locking multiple
> objects)
>
> 2. Lock-free CAS based on object version counter. Can use SqlAlchemy
> version
>    counter [2] or add our own. If conflicting changes are detected upon
> exiting
>    the context (i.e. version counter held differs from the one in the DB),
> will
>    raise OSLO RetryRequest exception.
>
>    pros:
>      - does not require locking
>
>    cons:
>      - require an additional field in the models
>
> While opt.2 only prevents the conflicting changes, but does not guarantee
> that the object does not change while within the context, opt.1 may seem
> preferential. But even with opt.1 the user should not expect that the
> changes made to the object while within the context will get to the
> database as the autonested_transaction could fail on flush/commit.
>
> So I’d like to hear others’ opinion on the problem and which of the two
> implementation options would be preferred? Or maybe someone has a better
> idea.
>
> [1]
> https://wiki.openstack.org/wiki/OpenStack_and_SQLAlchemy#MySQLdb_.2B_eventlet_.3D_sad
> [2] http://docs.sqlalchemy.org/en/rel_0_9/orm/versioning.html
>
> [3] https://review.openstack.org/#/c/315705/
>
> --
> Thanks,
> Ilya
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160513/14868e22/attachment-0001.html>