Open Stack

Excerpts from Anant Patil's message of 2014-12-02 10:37:31 -0800:
> Yes, that's the synchronization block for which we use the stack lock.
> Currently, a thread spin waits to acquire the lock to enter this critical
> section.
> 
> I don't really know how to do application level transaction. Is there an
> external library for that? AFAIK, we cannot switch from DB transaction to
> application level to do some computation to resolve next set of resources,
> submit to async queue, and again continue with the DB transaction. The
> transnational method looks attractive and clean to me but I am limited by
> my knowledge.Or do you mean to have DB transaction object made available
> via DB APIs and use them in the application? Please share your thoughts.
> 

Every DB request has a context attached to the currently running
greenthread which has the DB session object. So yes, you do begin the
transaction in one db API call, and try to commit it in another after
having attempted any application logic.  The whole thing should always be
in a try/except retry loop to handle deadlocks and conflicts introduced
by multi-master sync replication like Galera.

For non-transactional backends, they would have to use a spin lock like
you're using now.

> To avoid locking the stack, we were thinking of designating a single engine
> with responsibility of processing all notifications for a stack. All the
> workers will notify on the stack topic, which only one engine listens to
> and then the notifications end up in a queue (local to the engine, per
> stack), from where they are taken up one-by-one to continue the stack
> operation. The convergence jobs are produced by this engine for the stack
> it is responsible for, and they might end-up in any of the engines. But the
> notifications for a stack are directed to one engine to avoid contention
> for lock. The convergence load is leveled and distributed, and stack lock
> is not needed.
> 

No don't do that. You already have that engine, it is the database engine
and it is intended to be used for synchronization and will achieve a
higher degree of concurrency in bigger stacks because it will only block
two workers when they try to inspect or change the same rows.