[openstack-dev] [oslo][oslo.db] MySQL Cluster support

Morgan Fainberg morgan.fainberg at gmail.com
Mon Feb 6 20:04:34 UTC 2017


On Thu, Feb 2, 2017 at 2:28 PM, Octave J. Orgeron <octave.orgeron at oracle.com
> wrote:

> That refers to the total length of the row. InnoDB has a limit of 65k and
> NDB is limited to 14k.
>
> A simple example would be the volumes table in Cinder where the row length
> goes beyond 14k. So in the IF logic block, I change columns types that are
> vastly oversized such as status and attach_status, which by default are 255
> chars. So to determine a more appropriate size, I look through the Cinder
> code to find where the possible options/states are for those columns. Then
> I cut it down to a more reasonable size. I'm very careful when I cut the
> size of a string column to ensure that all of the possible values can be
> contained.
>
> In cases where a column is extremely large for capturing the outputs of a
> command, I will change the type to Text or TinyText depending on the length
> required. A good example of this is in the agents table of Neutron where
> there is a column for configurations that has a string length of 4096
> characters, which I change to Text. Text blobs are stored differently and
> do not count against the row length.
>

So
https://github.com/openstack/keystone/blob/master/keystone/common/sql/core.py#L117
would not be an issue with the 14k limit, simply limits for things such as
VARCHAR would be affected (in other words, we wouldn't need to change
keystone's implementation since we already use sql.text here)?

>
> I've also observed differences between Kilo, Mitaka, and tip where even
> for InnoDB some of these tables are getting wider than can be supported. So
> in the case of Cinder, some of the columns have been shifted to separate
> tables to fit within 65k. I've seen the same thing in Neutron. So I fully
> expect that some of the services that have table bloat will have to cut the
> lengths or break the tables up over time anyways. As that happens, it
> reduces the amount of work for me, which is a good thing.
>
> The most complicated database schemas to patch up are cinder, glance,
> neutron, and nova due to the size and complexity of their tables. Those
> also have a lot of churn between releases where the schema changes more
> often. Other services like keystone, heat, and ironic are considerably
> easier to work with and have well laid out tables that don't change much.
>
>
FTR: Keystone also supports "no-downtime-upgrades" (just pending some
functional tests before we apply for the tag) and we will be looking to
move towards Alembic, so make sure that the code supplied can easily be
swapped out between SQL-A-Migrate and Alembic (IIRC most projects want to
move to alembic, but it is varying levels of difficulty to do so and
therefore different priorities).

I look forward to solid NDB support; having using NDB in the past to
support another project, I always thought it could be an interesting choice
to back OpenStack (++ to what Monty said eariler).



> Thanks,
> Octave
>
>
> On 2/2/2017 1:25 PM, Mike Bayer wrote:
>
>
>
> On 02/02/2017 02:52 PM, Mike Bayer wrote:
>
>
> But more critically I noticed you referred to altering the names of
> columns to suit NDB.  How will this be accomplished?   Changing a column
> name in an openstack application is no longer trivial, because online
> upgrades must be supported for applications like Nova and Neutron.  A
> column name can't just change to a new name, both columns have to exist
> and logic must be added to keep these columns synchronized.
>
>
> correction, the phrase was "Row character length limits 65k -> 14k" - does
> this refer to the total size of a row?  I guess rows that store JSON or
> tables like keystone tokens are what you had in mind here, can you give
> specifics ?
>
>
>
> __________________________________________________________________________
>
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> --
>
> [image: Oracle] <http://www.oracle.com/>
> Octave J. Orgeron | Sr. Principal Architect and Software Engineer
> Oracle Linux OpenStack
> Mobile: +1-720-616-1550 <+17206161550>
> 500 Eldorado Blvd. | Broomfield, CO 80021
> [image: Certified Oracle Enterprise Architect: Systems Infrastructure]
> <http://www.oracle.com/us/solutions/enterprise-architecture/index.html>
> [image: Green Oracle] <http://www.oracle.com/commitment> Oracle is
> committed to developing practices and products that help protect the
> environment
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170206/452bcf28/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: oracle_sig_logo.gif
Type: image/gif
Size: 658 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170206/452bcf28/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: oea_logo.jpg
Type: image/jpeg
Size: 2398 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170206/452bcf28/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: green-for-email-sig_0.gif
Type: image/gif
Size: 356 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170206/452bcf28/attachment-0001.gif>


More information about the OpenStack-dev mailing list