[openstack-dev] [oslo.db] [ndb] ndb namespace throughout openstack projects
mbayer at redhat.com
Tue Jul 25 18:20:41 UTC 2017
On Mon, Jul 24, 2017 at 5:41 PM, Michael Bayer <mbayer at redhat.com> wrote:
>> oslo_db.sqlalchemy.String(255, ndb_type=TINYTEXT) -> VARCHAR(255) for most
>> dbs, TINYTEXT for ndb
>> oslo_db.sqlalchemy.String(4096, ndb_type=TEXT) -> VARCHAR(4096) for most
>> dbs, TEXT for ndb
>> oslo_db.sqlalchemy.String(255, ndb_size=64) -> VARCHAR(255) on most dbs,
>> VARCHAR(64) on ndb
>> This way, we can override the String with TINYTEXT or TEXT or change the
>> size for ndb.
>>> oslo_db.sqlalchemy.String(255) -> VARCHAR(255) on most dbs,
>>> TINYTEXT() on ndb
>>> oslo_db.sqlalchemy.String(255, ndb_size=64) -> VARCHAR(255) on
>>> most dbs, VARCHAR(64) on ndb
>>> oslo_db.sqlalchemy.String(50) -> VARCHAR(50) on all dbs
>>> oslo_db.sqlalchemy.String(64) -> VARCHAR(64) on all dbs
>>> oslo_db.sqlalchemy.String(80) -> VARCHAR(64) on most dbs, TINYTEXT()
>>> on ndb
>>> oslo_db.sqlalchemy.String(80, ndb_size=55) -> VARCHAR(64) on most
>>> dbs, VARCHAR(55) on ndb
>>> don't worry about implementation, can the above declaration ->
>>> datatype mapping work ?
>> In my patch for Neutron, you'll see a lot of the AutoStringText() calls to
>> replace exceptionally long String columns (4096, 8192, and larger).
> MySQL supports large VARCHAR now, OK. yeah this could be
> String(8192, ndb_type=TEXT) as well.
OK, no, sorry each time I think of this I keep seeing the verbosity of
imports etc. in the code, because if we had:
then we have to import both String and TEXT, and then what if there's
ndb.TEXT, the code is still making an ndb-specific decision, etc.
I still see that this can be mostly automated from a simple ruleset
based on the size:
length <= 64 : VARCHAR(length) on all backends
length > 64, length <= 255: VARCHAR(length) for most backends,
TINYTEXT for ndb
length > 4096: VARCHAR(length) for most backends, TEXT for ndb
the one case that seems outside of this is:
String(255) where they have an index or key on the VARCHAR, and in
fact they only need < 64 characters to be indexed. In that case you
don't want to use TINYTEXT, right? So one exception:
e.g. a declarative hint to the oslo_db backend to not use a LOB type.
then we just need oslo_db.sqlalchemy.types.String, and virtually
nothing except the import has to change, and a few keywords.
What we're trying to do in oslo_db is as much as possible state the
intent of a structure or datatype declaratively, and leave as much of
the implementation up to oslo_db itself.
More information about the OpenStack-dev