Open Stack

On 09/11/2013 06:37 AM, Steven Hardy wrote:

> I'm investigating some issues, where data stored to a text column in mysql
> is silently truncated if it's too big.
>
> It appears that the default configuration of mysql, and the sessions
> established via sqlalchemy is to simply warn on truncation rather than
> raise an error.
>
> This seems to me to be almost never what you want, since on retrieval the
> data is corrupt and bad/unexpected stuff is likely.
>
> This AFAICT is a mysql specific issue[1], which can be resolved by setting
> sql_mode to "traditional"[2,3], after which an error is raised on truncation,
> allowing us to catch the error before the data is stored.
>
> My question is, how do other projects, or oslo.db, handle this atm?
>
> It seems we either have to make sure the DB enforces the schema/model, or
> validate every single value before attempting to store, which seems like an
> unreasonable burden given that the schema changes pretty regularly.
>
> Can any mysql, sqlalchemy and oslo.db experts pitch in with opinions on
> this?

Nova has a PostgreSQL devstack gate, which occasionally catches errors 
that MySQL lets through.  For example, 
https://bugs.launchpad.net/nova/+bug/1217167

Unfortunately we have some MySQL-only code, and PostgreSQL obviously 
can't catch such errors there.

I think we should consider turning off auto-truncation for MySQL on our 
CI boxes.

-- 
David Ripton   Red Hat   dripton at redhat.com