Open Stack

On 5/11/15 2:02 PM, Attila Fazekas wrote:
>
> Not just with local database connections,
> the 10G network itself also fast. Is is possible you spend more time even on
> the kernel side tcp/ip stack (and the context switch..) (Not in physical I/O wait)
> than in the actual work on the DB side. (Check netperf TCP_RR)
>
> The scary part of a blocking I/O call is when you have two
> python thread (or green thread) and one of them is holding a DB lock the other
> is waiting for the same lock in a native blocking I/O syscall.
that's a database deadlock and whether you use eventlet, threads, 
asycnio or even just two transactions in a single-threaded script, that 
can happen regardless.  if your two eventlet "non blocking" greenlets 
are waiting forever for a deadlock,  you're just as deadlocked as if you 
have OS threads.


> If you do a read(2) in native code, the python itself might not be able to preempt it
> Your transaction might be finished with `DB Lock wait timeout`,
> with 30 sec of doing nothing, instead of scheduling to the another python thread,
> which would be able to release the lock.


Here's the "you're losing me" part because Python threads are OS 
threads, so Python isn't directly involved trying to "preempt" anything, 
unless you're referring to the effect of the GIL locking up the 
program.   However, it's pretty easy to make two threads in Python hit a 
database and do a deadlock against each other, and the rest of the 
program's threads continue to run just fine; in a DB deadlock situation 
you are blocked on IO and IO releases the GIL.

If you can illustrate a test script that demonstrates the actual failing 
of OS threads that does not occur greenlets here, that would make it 
immediately apparent what it is you're getting at here.



>
>> [1] http://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/
>>
>>
>>
>>
>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev