Open Stack

On Mon, Aug 18, 2014, at 01:59 AM, Ihar Hrachyshka wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
> 
> On 17/08/14 02:09, Angus Lees wrote:
> > 
> > On 16 Aug 2014 06:09, "Doug Hellmann" <doug at doughellmann.com 
> > <mailto:doug at doughellmann.com>> wrote:
> >> 
> >> 
> >> On Aug 15, 2014, at 9:29 AM, Ihar Hrachyshka
> >> <ihrachys at redhat.com
> > <mailto:ihrachys at redhat.com>> wrote:
> >> 
> >>> Signed PGP part Some updates on the matter:
> >>> 
> >>> - oslo-spec was approved with narrowed scope which is now
> >>> 'enabled mysqlconnector as an alternative in gate' instead of
> >>> 'switch the default db driver to mysqlconnector'. We'll revisit
> >>> the switch part the next cycle once we have the new driver
> >>> running in gate and real benchmarking is heavy-lifted.
> >>> 
> >>> - there are several patches that are needed to make devstack
> >>> and tempest passing deployment and testing. Those are collected
> >>> under the hood of: https://review.openstack.org/#/c/114207/ Not
> >>> much of them.
> >>> 
> >>> - we'll need a new oslo.db release to bump versions (this is
> >>> needed to set raise_on_warnings=False for the new driver, which
> >>> was incorrectly set to True in sqlalchemy till very recently).
> >>> This is expected to be released this month (as per Roman
> >>> Podoliaka).
> >> 
> >> This release is currently blocked on landing some changes in
> >> projects
> > using the library so they don?t break when the new version starts
> > using different exception classes. We?re tracking that work in 
> > https://etherpad.openstack.org/p/sqla_exceptions_caught
> >> 
> >> It looks like we?re down to 2 patches, one for cinder
> > (https://review.openstack.org/#/c/111760/) and one for glance 
> > (https://review.openstack.org/#/c/109655). Roman, can you verify
> > that those are the only two projects that need changes for the
> > exception issue?
> >> 
> >>> 
> >>> - once the corresponding patch for sqlalchemy-migrate is
> >>> merged, we'll also need a new version released for this.
> > 
> > So we're going for a new version of sqlalchemy?  (We have a
> > separate workaround for raise_on_warnings that doesn't require the
> > new sqlalchemy release if this brings too many other issues)
> 
> Wrong. We're going for a new version of *sqlalchemy-migrate*. Which is
> the code that we inherited from Mike and currently track in stackforge.
> 
> > 
> >>> - on PyPI side, no news for now. The last time I've heard from
> >>> Geert (the maintainer of MySQL Connector for Python), he was
> >>> working on this. I suspect there are some legal considerations
> >>> running inside Oracle. I'll update once I know more about
> >>> that.
> >> 
> >> If we don?t have the new package on PyPI, how do we plan to
> >> include it
> > in the gate? Are there options to allow an exception, or to make
> > the mirroring software download it anyway?
> > 
> > We can test via devstack without waiting for pypi, since devstack
> > will install via rpms/debs.
> 
> I expect that it will be settled. I have no indication that the issue
> is unsolvable, it will just take a bit more time than we're accustomed
> to. :)
> 
> At the moment, we install MySQLdb from distro packages for devstack.
> Same applies to new driver. It will be still great to see the package
> published on PyPI so that we can track its version requirements
> instead of relying on distros to package it properly. But I don't see
> it as a blocker.
> 
> Also, we will probably be able to run with other drivers supported by
> SQLAlchemy once all the work is done.
> 
So I got bored last night and decided to take a stab at making PyMySQL
work since I was a proponent of it earlier. Thankfully it did just
mostly work like I thought it would.
https://review.openstack.org/#/c/115495/ is the WIP devstack change to
test this out.

Postgres tests fail because it was applying the pymysql driver to the
postgres connection string. We can clean this up later in devstack
and/or devstack-gate depending on how we need to apply this stuff.
Bashate failed because I had to "monkeypatch" in a fix for a ceilometer
issue loading sqlalchemy drivers. The tempest neutron full job fails on
one test occasionally. Not sure yet if that is normal neutron full
failure mode or if a new thing from PyMySQL. The regular tempest job
passes just fine.

There are also some DB related errors in the logs that will need to be
cleaned up but overall it just works. So I would like to repropose that
we stop focusing all of this effort on the hard thing and use the easy
thing if it meets our needs. We can continue to make alternatives work,
but that is a different problem that we can solve at a different pace. I
am not sure how to test the neutron thing that Gus was running into
though so we should also check that really quickly.

Also, the tests themselves don't seem to run any faster or slower than
when using the default mysql driver. Hard to complain about that :)

Clark
> > 
> > - Gus
> > 
> > 
> > 
> > _______________________________________________ OpenStack-dev
> > mailing list OpenStack-dev at lists.openstack.org 
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
> 
> iQEcBAEBCgAGBQJT8cCDAAoJEC5aWaUY1u57YgQH/0T6QxHfYv0NUWh95AOS4U24
> YQ70/A5RD41rn+b/+RU27B97WCY7kJDSn//tfW+THFTZFWZDLy/g2AuXKkbTT3DU
> 4DTEIvkk4pTtmUhRGLDp4hVWnZ/wKMq1Xtu+jtuXEvz0dcSxgtzwKE3auJ1fD/SL
> gZPhUQwPpdQASQo8DSh1iziftlpTzvmhsvAAexvDRFdpZs87b7VTH2AFLYRgW47P
> 07eow5WL9KprR+Yxfg680A9GoghtB0ffGLvQmdnfOaln+MRx51ywTcq3RKeUU4RH
> fgJjddZOPsKhHHPHEwak8+qd2iZ/AvUh0OvkZ3QqX9Dj3ZcpYnJYMApHQNvnebw=
> =n5u7
> -----END PGP SIGNATURE-----
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev