Open Stack

On 09/18/2014 08:49 PM, Clint Byrum wrote:
> Excerpts from Christopher Yeoh's message of 2014-09-18 16:57:12 -0700:
>> On Thu, 18 Sep 2014 12:12:28 -0400
>> Sean Dague <sean at dague.net> wrote:
>>>> When we can return the json-schema to user in the future, can we say
>>>> that means API accepting utf8 or utf8mb4 is discoverable? If it is
>>>> discoverable, then we needn't limit anything in our python code.
>>>
>>> Honestly, we should accept utf8 (no weird mysqlism not quite utf8). We
>>> should make the default scheme for our dbs support that on names (but
>>> only for the name columns). The failure of a backend to do utf8 for
>>> real should return an error to the user. Let's not make this more
>>> complicated than it needs to be.
>>
>> I agree that discoverability for this is not the way to go - I think its
>> too complicated for end users. I don't know enough about mysql to know
>> if utf8mb4 is going to a performance issue but if its not then we
>> should just support utf-8 properly. 
>>
>> We can we can catch the db errors. However whilst converting db
>> errors causing 500s is fairly straightforward when an error occurs that
>> deep in Nova it also means a lot of potential unwinding work in the db
>> and compute layers which is complicated and error prone. So i'd prefer
>> to avoid the situation with input validation in the first place. 
> 
> Just to add a reference into the discussion:
> 
> http://dev.mysql.com/doc/refman/5.5/en/charset-unicode-utf8mb4.html
> 
> It does have the same limitation of making fixed width keys and CHAR()
> columns. It goes from 3 bytes per CHAR position, to 4, so it should not
> be a database wide default, but something we use sparingly.
> 
> Note that the right answer for things that are not utf-8 (like UUID's)
> is not to set a charset of latin1, but use BINARY/VARBINARY. Last
> time I tried I had a difficult time coercing SQLAlchemy to model the
> difference.. but maybe I just didn't look in the right part of the manual.

Agreed, honestly if we could get the UUIDs to be actually BINARY in the
database I suspect it would have a pretty substantial performance
increase based on past projects that did the same transition. The join
time goes way down... and at least in Nova we do a ton of joins.

	-Sean

-- 
Sean Dague
http://dague.net