Open Stack

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 21/08/14 02:03, Clark Boylan wrote:
> On Mon, Aug 18, 2014, at 01:59 AM, Ihar Hrachyshka wrote: On
> 17/08/14 02:09, Angus Lees wrote:
>>>> 
>>>> On 16 Aug 2014 06:09, "Doug Hellmann" <doug at doughellmann.com
>>>>  <mailto:doug at doughellmann.com>> wrote:
>>>>> 
>>>>> 
>>>>> On Aug 15, 2014, at 9:29 AM, Ihar Hrachyshka 
>>>>> <ihrachys at redhat.com
>>>> <mailto:ihrachys at redhat.com>> wrote:
>>>>> 
>>>>>> Signed PGP part Some updates on the matter:
>>>>>> 
>>>>>> - oslo-spec was approved with narrowed scope which is
>>>>>> now 'enabled mysqlconnector as an alternative in gate'
>>>>>> instead of 'switch the default db driver to
>>>>>> mysqlconnector'. We'll revisit the switch part the next
>>>>>> cycle once we have the new driver running in gate and
>>>>>> real benchmarking is heavy-lifted.
>>>>>> 
>>>>>> - there are several patches that are needed to make
>>>>>> devstack and tempest passing deployment and testing.
>>>>>> Those are collected under the hood of:
>>>>>> https://review.openstack.org/#/c/114207/ Not much of
>>>>>> them.
>>>>>> 
>>>>>> - we'll need a new oslo.db release to bump versions (this
>>>>>> is needed to set raise_on_warnings=False for the new
>>>>>> driver, which was incorrectly set to True in sqlalchemy
>>>>>> till very recently). This is expected to be released this
>>>>>> month (as per Roman Podoliaka).
>>>>> 
>>>>> This release is currently blocked on landing some changes
>>>>> in projects
>>>> using the library so they don?t break when the new version
>>>> starts using different exception classes. We?re tracking that
>>>> work in 
>>>> https://etherpad.openstack.org/p/sqla_exceptions_caught
>>>>> 
>>>>> It looks like we?re down to 2 patches, one for cinder
>>>> (https://review.openstack.org/#/c/111760/) and one for glance
>>>>  (https://review.openstack.org/#/c/109655). Roman, can you
>>>> verify that those are the only two projects that need changes
>>>> for the exception issue?
>>>>> 
>>>>>> 
>>>>>> - once the corresponding patch for sqlalchemy-migrate is 
>>>>>> merged, we'll also need a new version released for this.
>>>> 
>>>> So we're going for a new version of sqlalchemy?  (We have a 
>>>> separate workaround for raise_on_warnings that doesn't
>>>> require the new sqlalchemy release if this brings too many
>>>> other issues)
> 
> Wrong. We're going for a new version of *sqlalchemy-migrate*. Which
> is the code that we inherited from Mike and currently track in
> stackforge.
> 
>>>> 
>>>>>> - on PyPI side, no news for now. The last time I've heard
>>>>>> from Geert (the maintainer of MySQL Connector for
>>>>>> Python), he was working on this. I suspect there are some
>>>>>> legal considerations running inside Oracle. I'll update
>>>>>> once I know more about that.
>>>>> 
>>>>> If we don?t have the new package on PyPI, how do we plan
>>>>> to include it
>>>> in the gate? Are there options to allow an exception, or to
>>>> make the mirroring software download it anyway?
>>>> 
>>>> We can test via devstack without waiting for pypi, since
>>>> devstack will install via rpms/debs.
> 
> I expect that it will be settled. I have no indication that the
> issue is unsolvable, it will just take a bit more time than we're
> accustomed to. :)
> 
> At the moment, we install MySQLdb from distro packages for
> devstack. Same applies to new driver. It will be still great to see
> the package published on PyPI so that we can track its version
> requirements instead of relying on distros to package it properly.
> But I don't see it as a blocker.
> 
> Also, we will probably be able to run with other drivers supported
> by SQLAlchemy once all the work is done.
> 
>> So I got bored last night and decided to take a stab at making
>> PyMySQL work since I was a proponent of it earlier. Thankfully it
>> did just mostly work like I thought it would. 
>> https://review.openstack.org/#/c/115495/ is the WIP devstack
>> change to test this out.

Great!

> 
>> Postgres tests fail because it was applying the pymysql driver to
>> the postgres connection string. We can clean this up later in
>> devstack and/or devstack-gate depending on how we need to apply
>> this stuff. Bashate failed because I had to "monkeypatch" in a
>> fix for a ceilometer issue loading sqlalchemy drivers. The
>> tempest neutron full job fails on one test occasionally. Not sure
>> yet if that is normal neutron full failure mode or if a new thing
>> from PyMySQL. The regular tempest job passes just fine.
> 
>> There are also some DB related errors in the logs that will need
>> to be cleaned up but overall it just works. So I would like to
>> repropose that we stop focusing all of this effort on the hard
>> thing and use the easy thing if it meets our needs. We can
>> continue to make alternatives work, but that is a different
>> problem that we can solve at a different pace. I am not sure how
>> to test the neutron thing that Gus was running into though so we
>> should also check that really quickly.

In our patches througout the projects, we're actually not focusing on
any specific driver, even though the original spec is focused on MySQL
Connector. I still think we should achieve MySQL Connector working in
gate in the very near future. The current progress can be tracked at:

https://review.openstack.org/#/c/114207/

> 
>> Also, the tests themselves don't seem to run any faster or slower
>> than when using the default mysql driver. Hard to complain about
>> that :)

Remember, the switch is more about reliability than speed; also,
tempest tests can be insufficiently parallelized to show the difference.

> 
>> Clark
>>>> 
>>>> - Gus
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> OpenStack-dev mailing list OpenStack-dev at lists.openstack.org
>>>>  
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>
>>
>>>> 
_______________________________________________
>> OpenStack-dev mailing list OpenStack-dev at lists.openstack.org 
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>> 
> _______________________________________________ OpenStack-dev
> mailing list OpenStack-dev at lists.openstack.org 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)

iQEcBAEBCgAGBQJT9cqrAAoJEC5aWaUY1u57nisH/3guBDZX+p3lmot+rxB/C79B
xIG7Kv/Z7n74n9JjS8fhr8GuDig11u/dMoW9pndVQJAmHR01S+3163PSnNRK1T4O
MTn53iQXy6KApl1X/CYKWl6lQS2En36AjA1PKRwSl8nH0S5/KcJR5bsO3BrGjODs
A+x4CPphY3HjZqHnVvDtkDPFc1AUWJLSPgw5dfOp84InTcWmyGFy5fDKqqSAc4Rc
GBs9GYnlvV3wcPJT+OsQYsZ71OMXGR2AxwmNxZ7W7q9Jm29IUXR+/WS7V+ZQ32iE
9Vy8EkLqwvr1JH15Wryi3EJ4ODcCZ+8I8nzt7eWUOh96J67VhU7/gIF41gkRf2E=
=LNUQ
-----END PGP SIGNATURE-----