Open Stack

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512



On 06/10/14 19:10, Mike Bayer wrote:
> 
> On Oct 6, 2014, at 9:56 AM, Ihar Hrachyshka <ihrachys at redhat.com>
> wrote:
> 
>>> 
>>> But we can do better. We should also enforce utf8 on client
>>> side, so that there is no way to run with a different encoding,
>>> and so that we may get rid of additional options in sql
>>> connection strings. I've sent a patch for oslo.db [4] to do
>>> just that.
> 
> i would recommend that we definitely do *not* set explicit client
> encodings on all columns, and go with the most minimal approach for
> whatever the target database is.    For example, with Postgresql
> 8.4, utf-8 is not an issue so long as client_encoding is set within
> postgresql.conf.    I’ve had this kind of discussion many times in
> the past with folks who are trying to “protect” some subset of
> users that don’t have this setting in their conf file, either
> because they installed from an OSX package or some other weird
> reason, and generally I’m not buying it.    There’s no need to
> build tremendous verbosity and fine-grained specificity throughout
> a system in order to appease very narrow edge cases like this.
> It’s not just about potential performance problems, it’s also just
> a schema and code management issue, in that it is complexity that
> IMHO is just not needed.

- From server side, indeed, proper configuration described in
documentation may be enough. Though atm we don't document PostgreSQL
settings anywhere.

[As for MySQL, both server and client encodings are indeed advised to
be set to utf8.]

That said, I wonder how we're going to manage cases when those
*global* settings for the whole server should be really limited to
specific databases. Isn't it better to enforce utf8 on service side,
since we already know that we always run against utf8 database?

> 
> For this reason I’m pretty ambivalent overall about any kind of
> utf-8 hardcoding within the application connection logic.  IMHO
> this should be handled at the configurational layer as much as
> possible.  Though as long as it’s just an application time setting,
> and not something hardwired throughout the schema (implying we now
> have to *rely* upon UTF-8 encoding explicitly for all columns
> everywhere…and then what happens if we are on some target database
> that doesn’t quite do things the same way, e.g. DB2, oracle,
> others?), it’s not *too* big of a deal for me if it solves some
> problems right now.

Please let me clarify... Do you say that setting client encoding on
oslo.db side is actually ok? I haven't suggested to enforce utf8 per
column/table, though I guess we're already there.

> 
> short answer, there should be no need to drop PG8.X support for
> this reason.   If the client encoding setting is something we want
> hardcoded in the app layer and it fails for those versions (which
> I’m not familiar with? what is the thing that is not actually
> supported prior to libpq 9.1 ?  psycopg2’s set_client_encoding,
> really?   this doesn’t sound familiar two me), we can detect that
> and forego the setting - sqlalchemy dialect has server_version_info
> for this reason.

Forgoing, again, means some users running with utf8 client encoding,
others without it. I strive towards consistency here, that's why I try
to find a way to set the setting for all clients we support (and the
question is *which* of those clients we really support).

The thing that is not supported in psql < 9.1 is a connection option
added to libpq as of:
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=02e14562a806a96f38120c96421d39dfa7394192

As I said before, we have other ways to set client encoding for psql
clients that use libpq < 9.1, and the thing I try to determine is
whether we consider supporting those versions of libpq.

/Ihar
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)

iQEcBAEBCgAGBQJUM9yfAAoJEC5aWaUY1u571T0IALBgfRSL0PY1q9PC/vpiee2B
d++dVM+CU/SRhXh7zlY703K/sdRVvsPGlUNkzPjT0TGa81LbXPKtwVqPIQeUpI5i
a+X9HKHksaeidgLyuOPbSic+hFSoQESoYBswLfb0+vhT9JEn2XwzEyJo83c4MDH3
grgi5Crk+xx2uAa3mnEKCkeCOiq6t6V+7s7DKM8qdAEZOvKWlY3JK5W+AF6Smjlj
XJkywLXkDy+t+jMMIo2XqlR7xIVoy4wSTZRgbqrHpBqxqhL3/wzk31JnQgant8A+
2HF8p+Goy0CxoLOYb0LCE4JiHh6aoHXCcUOXW9BmcVBxfRg7el3OcKWJb9g2Q+M=
=78+Z
-----END PGP SIGNATURE-----