Open Stack

Why pymysql over mysql-python?

Endre Karlson
21. Aug. 2014 09:05 skrev "Angus Lees" <gus at inodes.org> følgende:

> On Wed, 20 Aug 2014 05:03:51 PM Clark Boylan wrote:
> > On Mon, Aug 18, 2014, at 01:59 AM, Ihar Hrachyshka wrote:
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA512
> > >
> > > On 17/08/14 02:09, Angus Lees wrote:
> > > > On 16 Aug 2014 06:09, "Doug Hellmann" <doug at doughellmann.com
> > > >
> > > > <mailto:doug at doughellmann.com>> wrote:
> > > >> On Aug 15, 2014, at 9:29 AM, Ihar Hrachyshka
> > > >> <ihrachys at redhat.com
> > > >
> > > > <mailto:ihrachys at redhat.com>> wrote:
> > > >>> Signed PGP part Some updates on the matter:
> > > >>>
> > > >>> - oslo-spec was approved with narrowed scope which is now
> > > >>> 'enabled mysqlconnector as an alternative in gate' instead of
> > > >>> 'switch the default db driver to mysqlconnector'. We'll revisit
> > > >>> the switch part the next cycle once we have the new driver
> > > >>> running in gate and real benchmarking is heavy-lifted.
> > > >>>
> > > >>> - there are several patches that are needed to make devstack
> > > >>> and tempest passing deployment and testing. Those are collected
> > > >>> under the hood of: https://review.openstack.org/#/c/114207/ Not
> > > >>> much of them.
> > > >>>
> > > >>> - we'll need a new oslo.db release to bump versions (this is
> > > >>> needed to set raise_on_warnings=False for the new driver, which
> > > >>> was incorrectly set to True in sqlalchemy till very recently).
> > > >>> This is expected to be released this month (as per Roman
> > > >>> Podoliaka).
> > > >>
> > > >> This release is currently blocked on landing some changes in
> > > >> projects
> > > >
> > > > using the library so they don?t break when the new version starts
> > > > using different exception classes. We?re tracking that work in
> > > > https://etherpad.openstack.org/p/sqla_exceptions_caught
> > > >
> > > >> It looks like we?re down to 2 patches, one for cinder
> > > >
> > > > (https://review.openstack.org/#/c/111760/) and one for glance
> > > > (https://review.openstack.org/#/c/109655). Roman, can you verify
> > > > that those are the only two projects that need changes for the
> > > > exception issue?
> > > >
> > > >>> - once the corresponding patch for sqlalchemy-migrate is
> > > >>> merged, we'll also need a new version released for this.
> > > >
> > > > So we're going for a new version of sqlalchemy?  (We have a
> > > > separate workaround for raise_on_warnings that doesn't require the
> > > > new sqlalchemy release if this brings too many other issues)
> > >
> > > Wrong. We're going for a new version of *sqlalchemy-migrate*. Which is
> > > the code that we inherited from Mike and currently track in stackforge.
> > >
> > > >>> - on PyPI side, no news for now. The last time I've heard from
> > > >>> Geert (the maintainer of MySQL Connector for Python), he was
> > > >>> working on this. I suspect there are some legal considerations
> > > >>> running inside Oracle. I'll update once I know more about
> > > >>> that.
> > > >>
> > > >> If we don?t have the new package on PyPI, how do we plan to
> > > >> include it
> > > >
> > > > in the gate? Are there options to allow an exception, or to make
> > > > the mirroring software download it anyway?
> > > >
> > > > We can test via devstack without waiting for pypi, since devstack
> > > > will install via rpms/debs.
> > >
> > > I expect that it will be settled. I have no indication that the issue
> > > is unsolvable, it will just take a bit more time than we're accustomed
> > > to. :)
> > >
> > > At the moment, we install MySQLdb from distro packages for devstack.
> > > Same applies to new driver. It will be still great to see the package
> > > published on PyPI so that we can track its version requirements
> > > instead of relying on distros to package it properly. But I don't see
> > > it as a blocker.
> > >
> > > Also, we will probably be able to run with other drivers supported by
> > > SQLAlchemy once all the work is done.
> >
> > So I got bored last night and decided to take a stab at making PyMySQL
> > work since I was a proponent of it earlier. Thankfully it did just
> > mostly work like I thought it would.
> > https://review.openstack.org/#/c/115495/ is the WIP devstack change to
> > test this out.
>
> Thanks!
>
> > Postgres tests fail because it was applying the pymysql driver to the
> > postgres connection string. We can clean this up later in devstack
> > and/or devstack-gate depending on how we need to apply this stuff.
> > Bashate failed because I had to "monkeypatch" in a fix for a ceilometer
> > issue loading sqlalchemy drivers. The tempest neutron full job fails on
> > one test occasionally. Not sure yet if that is normal neutron full
> > failure mode or if a new thing from PyMySQL. The regular tempest job
> > passes just fine.
> >
> > There are also some DB related errors in the logs that will need to be
> > cleaned up but overall it just works. So I would like to repropose that
> > we stop focusing all of this effort on the hard thing and use the easy
> > thing if it meets our needs. We can continue to make alternatives work,
> > but that is a different problem that we can solve at a different pace. I
> > am not sure how to test the neutron thing that Gus was running into
> > though so we should also check that really quickly.
>
> TL;DR: pymysql passes my test case.
> I'm perfectly happy to move towards using mysql+pymysql in gate tests.
> (The
> various changes I've been submitting are to support _any_ non-default
> driver).
>
> If anyone cares, my test case is in
> https://review.openstack.org/#/c/104436/
> Unfortunately it requires a small amount of hacking up the relevant
> utils.py
> functions to test with an arbitrary sqlalchemy connect string.  I had some
> clumsy attempts at making that easier, but they were rejected in favour of
> a
> far better solution that still hasn't landed yet :/
>
> --
>  - Gus
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140821/bb511007/attachment.html>