Open Stack

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hi all,

tl;dr we should enforce utf8 on server and client sides of db
connection, and this requires changes to docs and oslo.db. The latter
may require raising effective libpq version dep to 9.1+.

Recently I was working on making sure we always run using utf8 on both
database server and client side, and that we do it with all the proper
connection options set (f.e. mysqldb requires use_unicode=0 to avoid
performance drop - a thing that is not set anywhere in gate as of now,
and not documented).

For server side, it's a matter of how we create databases, and in
devstack, it's usually utf8 (though, for unclear reason, latin1 is
still enforced for nova for initial creation due to Folsom (sic!) bug;
I've reverted to utf8 with [1]).

For client side, it's not that clear. Though devstack enforces utf8 in
SQL connection strings, we don't ask to do so in official docs [2].
There is a bug [3] in LP that seems to be relevant.

I think the proper way is to enforce utf8 everywhere. The first step
would be to make sure we suggest adding charset=utf8 to SQL connection
strings, and configure SQL servers to use utf8. That is something to
be covered by docs team.

But we can do better. We should also enforce utf8 on client side, so
that there is no way to run with a different encoding, and so that we
may get rid of additional options in sql connection strings. I've sent
a patch for oslo.db [4] to do just that.

Though as you can see, the patch fails on py26 postgresql unit tests.
This is because I set charset_encoding for PostgreSQL via libpq
connection option that is available since PostgreSQL 9.1 only. The
failure is because we run py26 test suite on CentOS 6.5 that still
uses PostgreSQL 8.4.*, meaning the option is not available for us.

Now, there are several other ways available for us to enforce encoding
without using the option. First, we may set encoding thru
PGCLIENTENCODING environment variable. Second, we may execute "SET
CLIENT_ENCODING TO utf8" after connecting to psql server.

Now, the question is whether it's worth reimplementing the feature
using some of the available hacks that will work with PSQL < 9.1.
We're about to drop Python 2.6 support in Kilo, and that probably
means that CentOS 6.5 will be dropped from gate too. Do we have any
other distribution of interest that uses both python 2.7 and psql <
9.1, and is going to run Kilo pieces?

On Red Hat side, we're not interested in running Juno on EL6 that uses
old psql. As for EL7, it has a psql version that supports the
connection option. On Ubuntu side, the only LTS release that is still
supported and uses psql < 9.1 is Lucid, but it's based on py26, so it
should not be an issue. Any other consumers that could be affected by
effectively raising minimum version bar for psql to 9.1 in Kilo?

Though we've (almost) decided to drop py26 support in Kilo, we didn't
have similar discussions for other components that we use. So I'd like
to hear what's our take on that. Do we really drop *py26* support, or
maybe what we really do is we drop *CentOS 6.5* support?..

On a more general note, do we track those kind of external
dependencies anywhere? There are lots of them: neutron depends on
specific versions of openvswitch; oslo.db depends on specific versions
of underlying backend libraries, etc.

Comments?

[1]: https://review.openstack.org/126267
[2]:
http://docs.openstack.org/icehouse/install-guide/install/yum/content/nova-controller.html
[3]: https://bugs.launchpad.net/glance/+bug/1279000
[4]: https://review.openstack.org/111236

/Ihar
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)

iQEcBAEBCgAGBQJUMotwAAoJEC5aWaUY1u57IQQIAN2yHfxbj0DYDMHE40OGbEwU
8fis/KOAxSyUEhHbbECS8ODGjHNK9SeuddUplzL5sKVbYIMZgt3wKXGT/rT5ex0C
iMmc7/mb6JGGTUYvR1sF2EXxioJ3kWJSt5oQqBhVT14y2g8rwqN6tQaTusfDVZWN
yZ7KCmQDc+3HMyY3EmYCp6V1ndubLbx8Oh56/jtdSGdNtc7V4IwYIF3F4WQaKBcB
54tZkAO0wvlbbRrXqEiyf99/dVLDiMquM3LNKEbYg4zjDUEX74fGJ3mscQtWdDLv
0TcqOvQ2pwLvpksf30kvBBHGcNCF9x8Ov8H4vg5wRTD4bkpjEDwmXzQXTk7xfHM=
=aYRM
-----END PGP SIGNATURE-----