Open Stack

Hi Michael,

Comments below..

On 7/24/2017 9:13 AM, Michael Bayer wrote:
> On Mon, Jul 24, 2017 at 10:01 AM, Jay Pipes <jaypipes at gmail.com> wrote:
>
>> I would much prefer to *add* a brand new schema migration that handles
>> conversion of the entire InnoDB schema at a certain point to an
>> NDB-compatible one *after* that point. That way, we isolate the NDB changes
>> to one specific schema migration -- and can point users to that one specific
>> migration in case bugs arise. This is the reason that every release we add a
>> number of "placeholder" schema migration numbered files to handle situations
>> such as these.
>>
>> I understand that Oracle wants to support older versions of OpenStack in
>> their distribution and that's totally cool with me. But, the proper way IMHO
>> to do this kind of thing is to take one of the placeholder migrations and
>> use that as the NDB-conversion migration. I would posit that since Oracle
>> will need to keep some not-insignificant amount of Python code in their
>> distribution fork of Nova in order to bring in the oslo.db and Nova NDB
>> support, that it will actually be *easier* for them to maintain a *separate*
>> placeholder schema migration for all NDB conversion work instead of changing
>> an existing schema migration with a new patch.
> OK, if it is feasible for the MySQL engine to build out the whole
> schema as InnoDB and then do a migrate that changes the storage engine
> of all tables to NDB and then also changes all the datatypes, that can
> work.   If you want to go that way, then fine.

Unfortunately, to do that, you'd have to drop all of the constraints, 
foreign keys, and probably indexes before doing a change to table type. 
Then go back and put them in all into place. You also have to deal with 
changing your NDB cluster configuration to force all of the traffic to a 
single node since InnoDB tables are not replicated across an NDB 
cluster. So this is a lot more overhead for operators and introduces 
greater risks.


>
> However, I may be missing something but I'm not seeing the practical
> difference.   This new "ndb" migration still goes into the source
> tree, still gets invoked for all users, and if the "if ndb_enabled()"
> flag is somehow broken, it breaks just as well if it's in a brand new
> migration vs. if it's in an old migration.
>
> Suppose "if ndb_enabled(engine)" is somehow broken.  Either it crashes
> the migrations, or it runs inappropriately.
>
> If the conditional is in a brand new migration file that's pushed out
> in Queens, *everybody* runs it when they upgrade, as well as when they
> do fresh installation, and they get the breakage.
>
> if the conditional is in havana 216, *everybody* gets it when they do
> a fresh installation, and they get the breakage.   Upgraders do not.
>
> How is "new migration" better than "make old migration compatible" ?
>
> Again, fine by me if the other approach works, I'm just trying to see
> where I'm being dense here.
>
> Keep in mind that existing migrations *do* break and have to be fixed
> - because while the migration files don't change, the databases they
> talk to do.  The other thread I introduced about Mariadb 10.2 now
> refusing to DROP columns that have a CHECK constraint is an example,
> and will likely mean lots of old migration files across openstack
> projects will need adjustments.
>

Exactly! I've seen plenty of cases where these scripts have been patched 
to fix problems that crop up in later migrations. So doing these changes 
is not that alien to OpenStack, even for Nova:

http://git.openstack.org/cgit/openstack/nova/log/nova/db/sqlalchemy/migrate_repo/versions/216_havana.py

Also, another good point that everyone should be working on fixing is 
that with in MySQL 5.7.x you'll get warnings about duplicate keys, 
indexes, constraints, etc. that WILL NOT be supported in a future 
release. So these scripts have to be patched or MySQL support for these 
databases will be broken in the not so distant future.

>
>
>
>
>
>
>> All the best,
>> -jay
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev