[openstack-dev] [tc] revised Postgresql support status patch for governance
mbayer at redhat.com
Mon May 22 01:35:44 UTC 2017
On 05/21/2017 03:51 PM, Monty Taylor wrote:
>>> So I don't see the problem of "consistent utf8 support" having much to
>>> do with whether or not we support Posgtresql - you of course need your
>>> "CREATE DATABASE" to include the utf8 charset like we do on MySQL, but
>>> that's it.
>> That's where we stand which means that we're doing 3 byte UTF8 on MySQL,
>> and 4 byte on PG. That's actually an API facing difference today. It's
>> work to dig out of from the MySQL side, maybe the PG one is just all
>> super cool and done. But it's still a consideration point.
> The biggest concern for me is that we're letting API behavior be
> dictated by database backend and/or database config choices. The API
> should behave like the API behaves.
The API should behave like, "we store utf-8". We should accept that
"utf-8" means "up to four bytes" and make sure we are using utf8mb4 for
all MySQL backends. That the API of MySQL has made this bizarre
decision about what utf-8 is to be would be a bug in MySQL that needs to
be worked around by the calling application. Other databases that want
to work with openstack need to also do utf-8 with four bytes. We can
easily add some tests to oslo.db that round trip an assortment of
unicode glyphs to confirm this (if there's one kind of test I've written
more than anyone should, it's pushing out non-ascii bytes to a database
and testing they come back the same).
>> Sure, it's work. But that's fine. The point of that list was that there
>> is stuff that is work because SQLA is a leaky abstraction. Which is fine
>> if there are people taking that work off the table.
> I would not characterize this as SQLA being a leaky abstraction.
yeessss ! win! :)
> I'd say that at some point we didn't make a decision as to what we
> wanted to do with text input and how it would be stored or not stored
> and how it would be searched and sorted. Case sensitive collations have
> been available to us the entire time, but we never decided whether our
> API was case sensitive or case insensitive. OR - we *DID* decide that
> our API is case insensitive the fact that it isn't on some deployments
> is a bug. I'm putting money on the 'nobody made a decision' answer.
I wasn't there but perhaps early Openstack versions didn't have "textual
search" kinds of features ? maybe they were added by folks who didn't
consider the case sensitivity issue at that time. I'd be strongly in
favor of making use of oslo.db / SQLAlchemy constructs that are
explicitly case sensitive or not. It's true, SQLAlchemy also does not
force you to "make a decision" on this, if it did, this would be in the
"hooray the abstraction did not leak!" category. But SQLA makes lots
of these kinds of decisions to be kind of hands-off about things like
this as developers often don't want there to be a decision made here
(lest it adds even more to the "SQLAlchemy forces me to make so many
decisions!" complaint I have to read on twitter every day).
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
More information about the OpenStack-dev