Open Stack

Hi,

TL;DR we must replace mysql-python with mysqclient to get Python 3 support, it's safe and *very* easy

I restart the discussion because the previous one gone far from my point which is getting Python 3 support. IMO the fastest way to get Python 3 support is to replace mysql-python with mysqclient.

Update since the previous discussion began:

* Debian and Red Hat told me that they will probably base their package on mysqlclient
* I ran OpenStack tests on mysqlclient: all tests pass
* Nova, Neutron and Heat specs for Python 3 are almost accepted (or already accepted for Heat)

--

Good news, the portage of OpenStack to Python 3 is going well: Nova and Neutron specs to add Python 3 are almost accepted, the Heat spec was already merged.

   https://review.openstack.org/176868
   https://review.openstack.org/172962
   https://review.openstack.org/175340

The two hot dependencies are suds and mysql-python: none support Python 3.


My patch to replace suds with suds-jurko has been merged into oslo.vmware. I'm not aware of any blocking point for cinder and nova patches (to replace suds with suds-jurko or drop suds dependency):

   https://review.openstack.org/#/c/180130/
   https://review.openstack.org/#/c/181554/

suds project is no more maintained since 2011. suds-jurko is a fork which adds Python 3 compatibility (and problably other cool things like bugfixes).


Now, MySQL: OpenStack currently uses MySQL-python as the MySQL driver. MySQL-python is no more maintained since january 2014 (last commit, last release), there are 32 open issues and 25 pending pull requests. I heard that the maintainer basically gave up. Different people (including me) reported that they sent emails to the maintainer last 6 months, but he didn't reply. Check yourself the status of the project:

   https://github.com/farcepest/MySQLdb1/

Good news (again, I only have good news today!), as suds-jurko, mysqlclient is a fork of mysql-python, it is actively maintainted and supports Python 3:

   https://github.com/PyMySQL/mysqlclient-python

Debian and Red Hat maintainers told me that they will probably base their package on mysqlclient without changing the name of the package (python-mysqldb and MySQL-python). I propose to do the same in OpenStack to get Python 3 support. Patches for requirements and nova:

   https://review.openstack.org/#/c/179745/
   https://review.openstack.org/#/c/180128/

Note: I sent an email to the mysql-python maintainer to ask him to allow mysqlclient maintainer to takeover mysql-python, it would be the most natural and simplest solution, but he doesn't reply :-(

--

Mike Bayer wants better performances, he proposes to run DB queries in thread pools or use PyMySQL to be able to monkey-patch the driver. IMO it's a different topic, it may impact the architecture of applications. Such change is more complex than simply replacing *one line* in requirements.txt or test-requirements.txt, and we need Python 3 support right now ;-)

Mike Bayer also fears bugs or compatibility issues. I ran OpenStack tests with mysqlclient: well, all tests pass:

   https://review.openstack.org/#/c/180128/

But I agree, as any other project, mysqlclient can have bugs. IMO it's safer to rely on a library actively developed to quickly get fixes and new releases. Think about security vulnerabilities: what happens if a vulnerability is found in MySQL-python? Remind that the project didn't get an commit since january 2014...

Sorry, I don't understand why it's much more difficult to replace MySQL-Python with mysqlclient, than replacing suds with suds-jurko, the case is exactly the same.

Victor