[openstack-dev] [Neutron] Issue with pymysql
mbayer at redhat.com
Fri Jun 12 15:39:54 UTC 2015
On 6/12/15 6:42 AM, Sean Dague wrote:
> On 06/12/2015 06:31 AM, Joe Gordon wrote:
>> On Fri, Jun 12, 2015 at 7:13 PM, Sean Dague <sean at dague.net
>> <mailto:sean at dague.net>> wrote:
>> On 06/12/2015 01:17 AM, Salvatore Orlando wrote:
>> > It is however interesting that both "lock wait timeouts" and "missing
>> > savepoint" errors occur in operations pertaining the same table -
>> > securitygroups in this case.
>> > I wonder if the switch to pymysl has not actually uncovered some other
>> > bug in Neutron.
>> > I have no opposition to a revert, but since this will affect most
>> > projects, it's probably worth finding some time to investigate what is
>> > triggering this failure when sqlalchemy is backed by pymysql before
>> > doing that.
>> Right, we knew that the db driver would move some bugs around because
>> we're no longer blocking python processes on db access (so there used to
>> be a pseudo synchronization point before you ever got to the database).
>> My feeling is this should be looked into before it is straight reverted
>> (are jobs failing beyond Rally?). There are a number of benefits with
>> A quick look at logstash.openstack.org <http://logstash.openstack.org>
>> shows some of the stacktraces are happening in other neutron jobs as well.
>> the new driver, and we can't get to python3 with the old one.
>> Agreed, pymysql is not to blame it looks like we have hit some neutron
>> issues. So lets try to fix neutron. Just because neutron reverts the
>> default sql connector doesn't mean operators won't end up trying pymysql.
>> Rally failing is also an indicator that just such and implicit lock was
>> behavior that was depended on before, because it will be sending a bunch
>> of similar operations all at once as a kind of stress test. It would
>> tend to expose issues like this first.
>> Glad to see us catch these issues early.
> So, looking at little deeper this is also exposing across the board as
> well. I filed this bug - https://bugs.launchpad.net/neutron/+bug/1464612.
> I'm going to trigger and push the revert, as it's blocked fixes for
> stable/kilo being able to merge code (so it's cascading).
Please keep me very deeply in the loop on this because it will save
everyone a lot of time if I can decipher the interaction with SQLAlchemy
internals for folks.
> We can decide if we bring this back for all non-neutron jobs once things
> are working again (it lives behind a config var in devstack, so easy to
> set it per job).
More information about the OpenStack-dev