Open Stack

On 05/21/2017 03:51 PM, Monty Taylor wrote:
>>>
>>> So I don't see the problem of "consistent utf8 support" having much to
>>> do with whether or not we support Posgtresql - you of course need your
>>> "CREATE DATABASE" to include the utf8 charset like we do on MySQL, but
>>> that's it.
>>
>> That's where we stand which means that we're doing 3 byte UTF8 on MySQL,
>> and 4 byte on PG. That's actually an API facing difference today. It's
>> work to dig out of from the MySQL side, maybe the PG one is just all
>> super cool and done. But it's still a consideration point.
> 
> The biggest concern for me is that we're letting API behavior be 
> dictated by database backend and/or database config choices. The API 
> should behave like the API behaves.

The API should behave like, "we store utf-8".  We should accept that 
"utf-8" means "up to four bytes" and make sure we are using utf8mb4 for 
all MySQL backends.      That the API of MySQL has made this bizarre 
decision about what utf-8 is to be would be a bug in MySQL that needs to 
be worked around by the calling application.   Other databases that want 
to work with openstack need to also do utf-8 with four bytes.  We can 
easily add some tests to oslo.db that round trip an assortment of 
unicode glyphs to confirm this (if there's one kind of test I've written 
more than anyone should, it's pushing out non-ascii bytes to a database 
and testing they come back the same).


>>
>> Sure, it's work. But that's fine. The point of that list was that there
>> is stuff that is work because SQLA is a leaky abstraction. Which is fine
>> if there are people taking that work off the table.
> 
> I would not characterize this as SQLA being a leaky abstraction.

yeessss !   win!    :)

> 
> I'd say that at some point we didn't make a decision as to what we 
> wanted to do with text input and how it would be stored or not stored 
> and how it would be searched and sorted. Case sensitive collations have 
> been available to us the entire time, but we never decided whether our
> API was case sensitive or case insensitive. OR - we *DID* decide that 
> our API is case insensitive the fact that it isn't on some deployments 
> is a bug. I'm putting money on the 'nobody made a decision' answer.

I wasn't there but perhaps early Openstack versions didn't have "textual 
search" kinds of features ?   maybe they were added by folks who didn't 
consider the case sensitivity issue at that time.     I'd be strongly in 
favor of making use of oslo.db / SQLAlchemy constructs that are 
explicitly case sensitive or not.  It's true, SQLAlchemy also does not 
force you to "make a decision" on this, if it did, this would be in the 
"hooray the abstraction did not leak!" category.   But SQLA makes lots 
of these kinds of decisions to be kind of hands-off about things like 
this as developers often don't want there to be a decision made here 
(lest it adds even more to the "SQLAlchemy forces me to make so many 
decisions!" complaint I have to read on twitter every day).




> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev