Open Stack

The other factor I haven't seen mentioned here is that our usual
eventlet+mysqldb setup can deadlock rather easily(*), resulting in the
entire python process sitting idle for 30s until the mysqldb deadlock timer
goes off and raises an exception.  At this point (in nova at least), the
request is usually retried and almost certainly succeeds.  In neutron (as
another example), the requests weren't so easy to retry and code examples
where this was noticed had to be explicitly restructured to defer work that
might yield the greenlet.
ie: it's a big cost on coding, and deployments, as well as just being plain
incorrect code.

(*) eg: send simultaneous requests that modify the same resource.
Eventually you'll have two greenlets within the same api server executing
conflicting database operations.  Entire python process freezes.   (Note
this is not a regular db transaction block, neither greenlet will ever exit
their transactions without the deadlock timer).

Bug here: https://bugs.launchpad.net/oslo.db/+bug/1350149
Test that reproduces the issue here:
https://review.openstack.org/#/c/104436/

I'm dismayed that the conversation always skews towards discussing
performance when the current choice isn't even correct yet, and that as a
community we seem to be unwilling/unable to get behind what should be quite
an obvious technical issue with straightforward solutions.

If we absolutely can't switch to another mysql driver, another option that
was suggested recently (and passes the above test) is using
eventlet.monkey_patch(MySQLdb=True).  I haven't done the investigation to
find out why that isn't the default, or what the downsides are.  This
obviously doesn't help us with other factors, like python3-ness either.

 - Gus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150130/fc2945a6/attachment.html>