Open Stack

On Mon, Jul 24, 2017 at 5:41 PM, Michael Bayer <mbayer at redhat.com> wrote:
>> oslo_db.sqlalchemy.String(255, ndb_type=TINYTEXT) -> VARCHAR(255) for most
>> dbs, TINYTEXT for ndb
>> oslo_db.sqlalchemy.String(4096, ndb_type=TEXT) -> VARCHAR(4096) for most
>> dbs, TEXT for ndb
>> oslo_db.sqlalchemy.String(255, ndb_size=64) -> VARCHAR(255) on most dbs,
>> VARCHAR(64) on ndb
>>
>> This way, we can override the String with TINYTEXT or TEXT or change the
>> size for ndb.
>
>>>
>>> oslo_db.sqlalchemy.String(255)     -> VARCHAR(255) on most dbs,
>>> TINYTEXT() on ndb
>>> oslo_db.sqlalchemy.String(255, ndb_size=64)     -> VARCHAR(255) on
>>> most dbs, VARCHAR(64) on ndb
>>> oslo_db.sqlalchemy.String(50)     -> VARCHAR(50) on all dbs
>>> oslo_db.sqlalchemy.String(64)     -> VARCHAR(64) on all dbs
>>> oslo_db.sqlalchemy.String(80)     -> VARCHAR(64) on most dbs, TINYTEXT()
>>> on ndb
>>> oslo_db.sqlalchemy.String(80, ndb_size=55)     -> VARCHAR(64) on most
>>> dbs, VARCHAR(55) on ndb
>>>
>>> don't worry about implementation, can the above declaration ->
>>> datatype mapping work ?
>>>
>>>
>> In my patch for Neutron, you'll see a lot of the AutoStringText() calls to
>> replace exceptionally long String columns (4096, 8192, and larger).
>
> MySQL supports large VARCHAR now, OK.   yeah this could be
> String(8192, ndb_type=TEXT) as well.

OK, no, sorry each time I think of this I keep seeing the verbosity of
imports etc. in the code, because if we had:

String(80, ndb_type=TEXT)

then we have to import both String and TEXT, and then what if there's
ndb.TEXT, the code is still making an ndb-specific decision, etc.

I still see that this can be mostly automated from a simple ruleset
based on the size:

length <= 64 :    VARCHAR(length) on all backends
length > 64, length <= 255:   VARCHAR(length) for most backends,
TINYTEXT for ndb
length > 4096:  VARCHAR(length) for most backends, TEXT for ndb

the one case that seems outside of this is:

String(255)  where they have an index or key on the VARCHAR, and in
fact they only need < 64 characters to be indexed.  In that case you
don't want to use TINYTEXT, right?   So one exception:

oslo_db.sqlalchemy.types.String(255, indexable=True)

e.g. a declarative hint to the oslo_db backend to not use a LOB type.

then we just need oslo_db.sqlalchemy.types.String, and virtually
nothing except the import has to change, and a few keywords.

What we're trying to do in oslo_db is as much as possible state the
intent of a structure or datatype declaratively, and leave as much of
the implementation up to oslo_db itself.