<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Morgan,<br>
<br>
Comments below..<br>
<br>
Thanks,<br>
Octave<br>
<br>
<div class="moz-cite-prefix">On 2/6/2017 1:04 PM, Morgan Fainberg
wrote:<br>
</div>
<blockquote
cite="mid:CAGnj6auQ_5W=1cswXrCF38AB0+3W9895AGcxcN9Xzgzptgbzjg@mail.gmail.com"
type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Feb 2, 2017 at 2:28 PM,
Octave J. Orgeron <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:octave.orgeron@oracle.com" target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:octave.orgeron@oracle.com">octave.orgeron@oracle.com</a></a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> That refers to the total length of
the row. InnoDB has a limit of 65k and NDB is limited to
14k. <br>
<br>
A simple example would be the volumes table in Cinder
where the row length goes beyond 14k. So in the IF logic
block, I change columns types that are vastly oversized
such as status and attach_status, which by default are
255 chars. So to determine a more appropriate size, I
look through the Cinder code to find where the possible
options/states are for those columns. Then I cut it down
to a more reasonable size. I'm very careful when I cut
the size of a string column to ensure that all of the
possible values can be contained.<br>
<br>
In cases where a column is extremely large for capturing
the outputs of a command, I will change the type to Text
or TinyText depending on the length required. A good
example of this is in the agents table of Neutron where
there is a column for configurations that has a string
length of 4096 characters, which I change to Text. Text
blobs are stored differently and do not count against
the row length.<br>
</div>
</blockquote>
<div><br>
</div>
<div>So <a moz-do-not-send="true"
href="https://github.com/openstack/keystone/blob/master/keystone/common/sql/core.py#L117">https://github.com/openstack/keystone/blob/master/keystone/common/sql/core.py#L117</a>
would not be an issue with the 14k limit, simply limits
for things such as VARCHAR would be affected (in other
words, we wouldn't need to change keystone's
implementation since we already use sql.text here)?</div>
</div>
</div>
</div>
</blockquote>
<br>
Correct. Having done these patches for Kilo and Mitaka, I can say
that Keystone has been the easiest to patch up. I haven't had to
make any column changes at all. All I've had to do is change the
mysql_engine setting for each table to use the value from
mysql_storage_engine. So it hasn't had any impact on the table
schema or structure.<br>
<br>
<br>
<blockquote
cite="mid:CAGnj6auQ_5W=1cswXrCF38AB0+3W9895AGcxcN9Xzgzptgbzjg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> <br>
I've also observed differences between Kilo, Mitaka, and
tip where even for InnoDB some of these tables are
getting wider than can be supported. So in the case of
Cinder, some of the columns have been shifted to
separate tables to fit within 65k. I've seen the same
thing in Neutron. So I fully expect that some of the
services that have table bloat will have to cut the
lengths or break the tables up over time anyways. As
that happens, it reduces the amount of work for me,
which is a good thing.<br>
<br>
The most complicated database schemas to patch up are
cinder, glance, neutron, and nova due to the size and
complexity of their tables. Those also have a lot of
churn between releases where the schema changes more
often. Other services like keystone, heat, and ironic
are considerably easier to work with and have well laid
out tables that don't change much.<br>
<br>
</div>
</blockquote>
<div><br>
</div>
<div>FTR: Keystone also supports "no-downtime-upgrades"
(just pending some functional tests before we apply for
the tag) and we will be looking to move towards Alembic,
so make sure that the code supplied can easily be swapped
out between SQL-A-Migrate and Alembic (IIRC most projects
want to move to alembic, but it is varying levels of
difficulty to do so and therefore different priorities).</div>
</div>
</div>
</div>
</blockquote>
<br>
There are some things that I do like about Alembic and the way that
it heals the database. But there will probably be some tricky
conversions going from SQL Alchemy. <br>
<br>
<blockquote
cite="mid:CAGnj6auQ_5W=1cswXrCF38AB0+3W9895AGcxcN9Xzgzptgbzjg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>I look forward to solid NDB support; having using NDB
in the past to support another project, I always thought
it could be an interesting choice to back OpenStack (++ to
what Monty said eariler).</div>
</div>
</div>
</div>
</blockquote>
<br>
Thanks! I think the benefits will outweigh the investment for
everyone.<br>
<br>
<blockquote
cite="mid:CAGnj6auQ_5W=1cswXrCF38AB0+3W9895AGcxcN9Xzgzptgbzjg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> <br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> Thanks,<br>
Octave
<div>
<div class="gmail-h5"><br>
<br>
<div
class="gmail-m_-2475182059545556065moz-cite-prefix">On
2/2/2017 1:25 PM, Mike Bayer wrote:<br>
</div>
<blockquote type="cite"> <br>
<br>
On 02/02/2017 02:52 PM, Mike Bayer wrote: <br>
<blockquote type="cite"> <br>
But more critically I noticed you referred to
altering the names of <br>
columns to suit NDB. How will this be
accomplished? Changing a column <br>
name in an openstack application is no longer
trivial, because online <br>
upgrades must be supported for applications like
Nova and Neutron. A <br>
column name can't just change to a new name,
both columns have to exist <br>
and logic must be added to keep these columns
synchronized. <br>
<br>
</blockquote>
<br>
correction, the phrase was "Row character length
limits 65k -> 14k" - does this refer to the
total size of a row? I guess rows that store JSON
or tables like keystone tokens are what you had in
mind here, can you give specifics ? <br>
<br>
<br>
<br>
______________________________<wbr>______________________________<wbr>______________
<br>
OpenStack Development Mailing List (not for usage
questions) <br>
Unsubscribe: <a moz-do-not-send="true"
class="gmail-m_-2475182059545556065moz-txt-link-abbreviated"
href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe"
target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a>
<br>
<a moz-do-not-send="true"
class="gmail-m_-2475182059545556065moz-txt-link-freetext"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a>
<br>
<br>
</blockquote>
<br>
</div>
</div>
<div class="gmail-m_-2475182059545556065moz-signature">--
<br>
<br>
<p> <a moz-do-not-send="true"
href="http://www.oracle.com/" target="_blank"><img
src="cid:part5.08090205.07090901@oracle.com"
alt="Oracle" border="0" height="26" width="114"></a><br>
<font color="#666666" size="2" face="Verdana, Arial,
Helvetica, sans-serif"><span class="gmail-">Octave
J. Orgeron | Sr. Principal Architect and
Software Engineer<br>
<font color="#ff0000">Oracle</font> Linux
OpenStack<br>
</span><span class="gmail-"> 500 Eldorado Blvd. |
Broomfield, CO 80021<br>
</span><a moz-do-not-send="true"
href="http://www.oracle.com/us/solutions/enterprise-architecture/index.html"
target="_blank"><img
src="cid:part7.04040907.01060808@oracle.com"
alt="Certified Oracle Enterprise Architect:
Systems Infrastructure" border="0" height="42"
width="182"></a><br>
<a moz-do-not-send="true"
href="http://www.oracle.com/commitment"
target="_blank"><img
src="cid:part9.08020003.03000907@oracle.com"
alt="Green Oracle" align="absmiddle"
border="0" height="28" width="44"></a> <font
color="#4B7D42" size="1" face="Verdana, Arial,
Helvetica, sans-serif">Oracle is committed to
developing practices and products that help
protect the environment</font> </font></p>
<font color="#666666" size="2" face="Verdana, Arial,
Helvetica, sans-serif"> </font></div>
</div>
<br>
______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage
questions)<br>
Unsubscribe: <a moz-do-not-send="true"
href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe"
rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
<a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<br>
<p>
<a href="http://www.oracle.com/" target="_blank"><img
src="cid:part13.09040505.05000902@oracle.com" alt="Oracle"
border="0" height="26" width="114"></a><br>
<font color="#666666" size="2" face="Verdana, Arial, Helvetica,
sans-serif">Octave J. Orgeron | Sr. Principal Architect and
Software Engineer<br>
<font color="#ff0000">Oracle</font> Linux OpenStack<br>
<a href="tel:+17206161550"></a>
500 Eldorado Blvd. | Broomfield, CO 80021<br>
<a
href="http://www.oracle.com/us/solutions/enterprise-architecture/index.html"><img
src="cid:part16.03060007.01000601@oracle.com"
alt="Certified Oracle Enterprise Architect: Systems
Infrastructure" border="0" height="42" width="182"></a><br>
<a href="http://www.oracle.com/commitment" target="_blank"><img
src="cid:part18.08050607.00030000@oracle.com" alt="Green
Oracle" align="absmiddle" border="0" height="28"
width="44"></a>
<font color="#4B7D42" size="1" face="Verdana, Arial,
Helvetica, sans-serif">Oracle is committed to developing
practices and products that help protect the environment</font>
</font></p>
<font color="#666666" size="2" face="Verdana, Arial, Helvetica,
sans-serif">
</font></div>
</body>
</html>