Open Stack

> First of all I propose to wrap HTTP handlers by begin/commit/rollback

I don't know what you are talking about, but we do wrap handlers in
transaction for a long time. Here's the code
https://github.com/stackforge/fuel-web/blob/2de3806128f398d192d7e31f4ca3af571afeb0b2/nailgun/nailgun/api/v1/handlers/base.py#L53-L84

The issue is that we sometimes perform `.commit()` inside the code
(e.g. `task.execute()`) and therefore it's hard to predict which data
are committed and which are not.

In order to avoid this, we have to declare strict scopes for different
layers. Yes, we definitely should base on idea that handlers should
open transaction on the begin and close it on the end. But that won't
solve all problems, because sometimes we should commit data before
handler's end. For instance, commit some task before sending message
to Astute. Such cases complicate the things.. and it would be cool if
could avoid them by re-factoring our architecture. Perhaps, we could
send tasks to Astute when the handler is done? What do you think?

Thanks,
igor

On Wed, May 6, 2015 at 12:15 PM, Lukasz Oles <loles at mirantis.com> wrote:
> On Wed, May 6, 2015 at 10:51 AM, Alexander Kislitsky
> <akislitsky at mirantis.com> wrote:
>> Hi!
>>
>> The refactoring of transactions management in Nailgun is critically required
>> for scaling.
>>
>> First of all I propose to wrap HTTP handlers by begin/commit/rollback
>> decorator.
>> After that we should introduce transactions wrapping decorator into Task
>> execute/message calls.
>> And the last one is the wrapping of receiver calls.
>>
>> As result we should have begin/commit/rollback calls only in transactions
>> decorator.
>
> Big +1 for this. I always wondered why we don't have it.
>
>>
>
>> Also I propose to separate working with DB objects into separate lair and
>> use only high level Nailgun objects in the code and tests. This work was
>> started long time ago, but not finished yet.
>>
>> On Thu, Apr 30, 2015 at 12:36 PM, Roman Prykhodchenko <me at romcheg.me> wrote:
>>>
>>> Hi folks!
>>>
>>> Recently I faced a pretty sad fact that in Nailgun there’s no common
>>> approach to manage transactions. There are commits and flushes in random
>>> places of the code and it used to work somehow just because it was all
>>> synchronous.
>>>
>>> However, after just a few of the subcomponents have been moved to
>>> different processes, it all started producing races and deadlocks which are
>>> really hard to resolve because there is absolutely no way to predict how a
>>> specific transaction is managed but by analyzing the source code. That is
>>> rather an ineffective and error-prone approach that has to be fixed before
>>> it became uncontrollable.
>>>
>>> Let’s arrange a discussions to design a document which will describe where
>>> and how transactions are managed and refactor Nailgun according to it in
>>> 7.0. Otherwise results may be sad.
>>>
>>>
>>> - romcheg
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Łukasz Oleś
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev