<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Jay,<br>
<br>
Comments below..<br>
<br>
<div class="moz-cite-prefix">On 7/24/2017 8:01 AM, Jay Pipes wrote:<br>
</div>
<blockquote
cite="mid:31dc1af1-6939-73f4-9225-febfcfc5d458@gmail.com"
type="cite">+Dan Smith
<br>
<br>
Good morning Mike :) Comments inline...
<br>
<br>
On 07/23/2017 08:05 PM, Michael Bayer wrote:
<br>
<blockquote type="cite">On Sun, Jul 23, 2017 at 6:10 PM, Jay Pipes
<a class="moz-txt-link-rfc2396E" href="mailto:jaypipes@gmail.com"><jaypipes@gmail.com></a> wrote:
<br>
<blockquote type="cite">Glad you brought this up, Mike. I was
going to start a thread about this.
<br>
Comments inline.
<br>
<br>
On 07/23/2017 05:02 PM, Michael Bayer wrote:
<br>
Well, besides that point (which I agree with), that is
attempting to change
<br>
an existing database schema migration, which is a no-no in my
book ;)
<br>
</blockquote>
<br>
OK this point has come up before and I always think there's a
bit of
<br>
an over-broad kind of purism being applied (kind of like when
someone
<br>
says, "never use global variables ever!" and I say, "OK so
sys.modules
<br>
is out right ?" :) ).
<br>
</blockquote>
<br>
I'm not being a purist. I'm being a realist :) See below...
<br>
<br>
<blockquote type="cite">I agree with "never change a migration" to
the extent that you should
<br>
never change the *SQL steps emitted for a database migration*.
That
<br>
is, if your database migration emitted "ALTER TABLE foobar
foobar
<br>
foobar" on a certain target databse, that should never change.
No
<br>
problem there.
<br>
<br>
However, what we're doing here is adding new datatype logic for
the
<br>
NDB backend which are necessarily different; not to mention that
NDB
<br>
requires more manipulation of constraints to make certain
changes
<br>
happen. To make all that work, the *Python code that emits the
SQL
<br>
for the migration* needs to have changes made, mostly (I would
say
<br>
completely) in the form of new conditionals for NDB-specific
concepts.
<br>
In the case of the datatypes, the migrations will need to
refer to
<br>
a SQLAlchemy type object that's been injected with the
ndb-specific
<br>
logic when the NDB backend is present; I've made sure that when
the
<br>
NDB backend is *not* present, the datatypes behave exactly the
same as
<br>
before.
<br>
</blockquote>
<br>
No disagreement here.
<br>
<br>
<blockquote type="cite">So basically, *SQL steps do not change*,
but *Python code that emits
<br>
the SQL steps* necessarily has to change to accomodate for when
the
<br>
"ndb" flag is present - this because these migrations have to
run on
<br>
brand new ndb installations in order to create the database.
If Nova
<br>
and others did the initial "create database" without using the
<br>
migration files and instead used a create_all(), things would be
<br>
different, but that's not how things work (and also it is fine
that
<br>
the migrations are used to build up the DB).
<br>
</blockquote>
<br>
So, I see your point here, but my concern here is that if we
*modify* an existing schema migration that has already been tested
to properly apply a schema change for MySQL/InnoDB and PostgreSQL
with code that is specific to NDB, we introduce the potential for
bugs where users report that the same migration works sometimes
but fails other times.
<br>
</blockquote>
<br>
I don't think that the testing issues should be a concern here
because I've been working to make sure that the tests work with both
InnoDB and NDB. It's a pain, but again, we are only talking about a
handful of the services. Bottom line here is that if you are not
using NDB, the changes have zero effect on your setup.<br>
<br>
<blockquote
cite="mid:31dc1af1-6939-73f4-9225-febfcfc5d458@gmail.com"
type="cite">
<br>
I would much prefer to *add* a brand new schema migration that
handles conversion of the entire InnoDB schema at a certain point
to an NDB-compatible one *after* that point. That way, we isolate
the NDB changes to one specific schema migration -- and can point
users to that one specific migration in case bugs arise. This is
the reason that every release we add a number of "placeholder"
schema migration numbered files to handle situations such as
these.
<br>
</blockquote>
<br>
The only problem with this approach is that it assumes you are on
InnoDB to start out with, which is not the use case here. This is
for new installations or ones that started out with NDB, so we have
to start out with the base schema in the scripts working. <br>
<blockquote
cite="mid:31dc1af1-6939-73f4-9225-febfcfc5d458@gmail.com"
type="cite">
<br>
I understand that Oracle wants to support older versions of
OpenStack in their distribution and that's totally cool with me.
But, the proper way IMHO to do this kind of thing is to take one
of the placeholder migrations and use that as the NDB-conversion
migration. I would posit that since Oracle will need to keep some
not-insignificant amount of Python code in their distribution fork
of Nova in order to bring in the oslo.db and Nova NDB support,
that it will actually be *easier* for them to maintain a
*separate* placeholder schema migration for all NDB conversion
work instead of changing an existing schema migration with a new
patch.
<br>
</blockquote>
<br>
And this is the whole point of the work that I'm doing. Getting
upstream so that others can benefit and so that we don't have to
waste cycles maintaining custom code. Instead, we do all of the work
upstream and that will enable our customers to more easily upgrade
from one release to another. FYI, we have been using NDB since
version 2 of our product. We are working on version 4 right now.<br>
<br>
<blockquote
cite="mid:31dc1af1-6939-73f4-9225-febfcfc5d458@gmail.com"
type="cite">
<br>
All the best,
<br>
-jay
<br>
<br>
__________________________________________________________________________
<br>
OpenStack Development Mailing List (not for usage questions)
<br>
Unsubscribe:
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<br>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
<br>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<br>
<p>
<a href="http://www.oracle.com/" target="_blank"><img
src="cid:part1.07080306.00010103@oracle.com" alt="Oracle"
border="0" height="26" width="114"></a><br>
<font color="#666666" size="2" face="Verdana, Arial, Helvetica,
sans-serif">Octave J. Orgeron | Sr. Principal Architect<br>
<font color="#666666" size="2" face="Verdana, Arial,
Helvetica, sans-serif">Certified Enterprise Architect<br>
<font color="#ff0000">Oracle</font> Linux OpenStack<br>
Mobile: <a href="tel:+17206161550">+1-720-616-1550</a><br>
500 Eldorado Blvd. | Broomfield, CO 80021<br>
<a href="http://www.opengroup.org/certifications/togaf"><img
src="cid:part4.09070300.08000707@oracle.com" alt="TOGAF
9.1 Certified" border="0" height="50" width="185"></a>
<a
href="http://www.oracle.com/us/solutions/enterprise-architecture/index.html"><img
src="cid:part6.07070900.09030300@oracle.com"
alt="Certified Oracle Enterprise Architect: Systems
Infrastructure" border="0" height="42" width="182"></a><br>
<a href="http://www.oracle.com/commitment" target="_blank"><img
src="cid:part8.00050204.08080909@oracle.com" alt="Green
Oracle" align="absmiddle" border="0" height="28"
width="44"></a>
<font color="#4B7D42" size="1" face="Verdana, Arial,
Helvetica, sans-serif">Oracle is committed to developing
practices and products that help protect the environment</font>
</font></font></p>
<font color="#666666" size="2" face="Verdana, Arial, Helvetica,
sans-serif"><font color="#666666" size="2" face="Verdana, Arial,
Helvetica, sans-serif">
</font></font></div>
</body>
</html>