[openstack-dev] [nova] unit test migration failure specific to MySQL/MariaDB - 'uuid': used in a foreign key constraint 'block_device_mapping_instance_uuid_fkey'
Mike Bayer
mbayer at redhat.com
Wed Jan 7 20:37:28 UTC 2015
OK so it’s looking like sql_mode=‘TRADITIONAL’ that allows it to work. So that is most of it. My MariaDB has no default sql_mode but oslo.db should be setting this, but in any case this seems more like a local oslo.db connection type of thing that I can track down myself, so most of the mystery solved! (at least the part that I didn’t feel like getting into….which I did anyway).
Mike Bayer <mbayer at redhat.com> wrote:
> working with sdague on IRC, the first thing I’m seeing is that my MariaDB server is disallowing a change in column that is UNIQUE and has an FK pointing to it, and this is distinctly different from a straight up MySQL server (see below).
>
> http://paste.openstack.org/raw/155896/
>
>
> old school MySQL:
>
> Welcome to the MySQL monitor. Commands end with ; or \g.
> Your MySQL connection id is 4840
> Server version: 5.6.15 Homebrew
>
> Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
>
> Oracle is a registered trademark of Oracle Corporation and/or its
> affiliates. Other names may be trademarks of their respective
> owners.
>
> Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
>
> mysql> create table foo (id int, blah int, primary key (id), unique key (blah)) engine=InnoDB;
> Query OK, 0 rows affected (0.01 sec)
>
> mysql> create table bar(id int, blah_fk int, primary key (id), foreign key (blah_fk) references foo(blah)) engine=InnoDB;
> Query OK, 0 rows affected (0.01 sec)
>
> mysql> alter table foo change column blah blah int not null;
> Query OK, 0 rows affected (0.02 sec)
> Records: 0 Duplicates: 0 Warnings: 0
>
> mysql>
>
>
>
> MariaDB 10:
>
> MariaDB [test]> create table foo (id int, blah int, primary key (id), unique key (blah));
> Query OK, 0 rows affected (0.09 sec)
>
> MariaDB [test]> create table bar(id int, blah_fk int, primary key (id), foreign key (blah_fk) references foo(blah));
> Query OK, 0 rows affected (0.12 sec)
>
> MariaDB [test]> alter table foo change column blah blah int not null;
> ERROR 1833 (HY000): Cannot change column 'blah': used in a foreign key constraint 'bar_ibfk_1' of table 'test.bar'
> MariaDB [test]>
>
> Matt Riedemann <mriedem at linux.vnet.ibm.com> wrote:
>
>> On 1/6/2015 5:40 PM, Mike Bayer wrote:
>>> Hello -
>>>
>>> Victor Sergeyev and I are both observing the following test failure which occurs with all the tests underneath nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL. This is against master with a brand new tox environment and everything at the default.
>>>
>>> It does not seem to be occurring on gates that run these tests and interestingly the tests seem to complete very quickly (under seven seconds) on the gate as well; the failures here take between 50-100 seconds to occur, not fully deterministically, and only on the MySQL backend; the Postgresql and SQLite versions of these tests pass. I’m running against MariaDB server 10.0.14 with Python 2.7.8 on Fedora 21.
>>>
>>> Below is the test just for test_walk_versions, but the warnings (not necessarily the failures themselves) here also occur for test_migration_267 as well as test_innodb_tables.
>>>
>>> I’m still looking into what the cause of this is, I’d imagine it’s something related to newer MySQL versions or perhaps MariaDB vs. MySQL, I’m just putting it up here in case someone already knows what this is or has some clue to save me some time figuring it out. I apologize if I’m just doing something dumb, I’ve only recently begun to run Nova’s test suite in full against all backends, so I haven’t yet put intelligent thought into this nor have I tried to yet look at the migration in question causing the problem. Will do that next.
>>>
>>>
>>> [mbayer at thinkpad nova]$ tox -e py27 -- nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.test_walk_versions
>>> py27 develop-inst-noop: /home/mbayer/dev/openstack/nova
>>> py27 runtests: PYTHONHASHSEED='0'
>>> py27 runtests: commands[0] | find . -type f -name *.pyc -delete
>>> py27 runtests: commands[1] | bash tools/pretty_tox.sh nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.test_walk_versions
>>> running testr
>>> running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
>>> OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
>>> OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-160} \
>>> ${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./nova/tests} --list
>>> running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
>>> OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
>>> OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-160} \
>>> ${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./nova/tests} --load-list /tmp/tmpw7zqhE
>>>
>>> 2015-01-06 18:28:12.913 32435 WARNING oslo.db.sqlalchemy.session [req-5cc6731f-00ef-43df-8aec-4914a44d12c5 ] MySQL SQL mode is '', consider enabling TRADITIONAL or STRICT_ALL_TABLES
>>> {0} nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.test_walk_versions [51.553131s] ... FAILED
>>>
>>> Captured traceback:
>>> ~~~~~~~~~~~~~~~~~~~
>>> Traceback (most recent call last):
>>> File "nova/tests/unit/db/test_migrations.py", line 151, in test_walk_versions
>>> self.walk_versions(self.snake_walk, self.downgrade)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 193, in walk_versions
>>> self.migrate_up(version, with_data=True)
>>> File "nova/tests/unit/db/test_migrations.py", line 148, in migrate_up
>>> super(NovaMigrationsCheckers, self).migrate_up(version, with_data)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 263, in migrate_up
>>> self.REPOSITORY, version)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py", line 186, in upgrade
>>> return _migrate(url, repository, version, upgrade=True, err=err, **opts)
>>> File "<string>", line 2, in _migrate
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/util/__init__.py", line 160, in with_engine
>>> return f(*a, **kw)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py", line 366, in _migrate
>>> schema.runchange(ver, change, changeset.step)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/schema.py", line 93, in runchange
>>> change.run(self.engine, step)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/script/py.py", line 148, in run
>>> script_func(engine)
>>> File "/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py", line 103, in upgrade
>>> process_null_records(meta, scan=False)
>>> File "/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py", line 89, in process_null_records
>>> table.columns.uuid.alter(nullable=False)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py", line 534, in alter
>>> return alter_column(self, *p, **k)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py", line 141, in alter_column
>>> engine._run_visitor(visitorcallable, delta)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1616, in _run_visitor
>>> conn._run_visitor(visitorcallable, element, **kwargs)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1245, in _run_visitor
>>> **kwargs).traverse_single(element)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py", line 56, in traverse_single
>>> ret = super(AlterTableVisitor, self).traverse_single(elem)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 120, in traverse_single
>>> return meth(obj, **kw)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/databases/mysql.py", line 47, in visit_column
>>> self.execute()
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py", line 44, in execute
>>> return self.connection.execute(self.buffer.getvalue())
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 721, in execute
>>> return self._execute_text(object, multiparams, params)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 870, in _execute_text
>>> statement, parameters
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 958, in _execute_context
>>> context)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1155, in _handle_dbapi_exception
>>> util.raise_from_cause(newraise, exc_info)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
>>> reraise(type(exception), exception, tb=exc_tb)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 951, in _execute_context
>>> context)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 436, in do_execute
>>> cursor.execute(statement, parameters)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
>>> self.errorhandler(self, exc, value)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
>>> raise errorclass, errorvalue
>>> OperationalError: (OperationalError) (1833, "Cannot change column 'uuid': used in a foreign key constraint 'block_device_mapping_instance_uuid_fkey' of table 'qzclautyzr.block_device_mapping'") '\nALTER TABLE instances CHANGE COLUMN uuid uuid VARCHAR(36) NOT NULL' ()
>>> Traceback (most recent call last):
>>> _StringException: Empty attachments:
>>> stdout
>>>
>>> stderr: {{{
>>> /home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:436: Warning: Duplicate index 'block_device_mapping_instance_uuid_virtual_name_device_name_idx' defined on the table 'qzclautyzr.block_device_mapping'. This is deprecated and will be disallowed in a future release.
>>> cursor.execute(statement, parameters)
>>> }}}
>>>
>>> Traceback (most recent call last):
>>> File "nova/tests/unit/db/test_migrations.py", line 151, in test_walk_versions
>>> self.walk_versions(self.snake_walk, self.downgrade)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 193, in walk_versions
>>> self.migrate_up(version, with_data=True)
>>> File "nova/tests/unit/db/test_migrations.py", line 148, in migrate_up
>>> super(NovaMigrationsCheckers, self).migrate_up(version, with_data)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 263, in migrate_up
>>> self.REPOSITORY, version)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py", line 186, in upgrade
>>> return _migrate(url, repository, version, upgrade=True, err=err, **opts)
>>> File "<string>", line 2, in _migrate
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/util/__init__.py", line 160, in with_engine
>>> return f(*a, **kw)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py", line 366, in _migrate
>>> schema.runchange(ver, change, changeset.step)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/schema.py", line 93, in runchange
>>> change.run(self.engine, step)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/script/py.py", line 148, in run
>>> script_func(engine)
>>> File "/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py", line 103, in upgrade
>>> process_null_records(meta, scan=False)
>>> File "/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py", line 89, in process_null_records
>>> table.columns.uuid.alter(nullable=False)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py", line 534, in alter
>>> return alter_column(self, *p, **k)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py", line 141, in alter_column
>>> engine._run_visitor(visitorcallable, delta)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1616, in _run_visitor
>>> conn._run_visitor(visitorcallable, element, **kwargs)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1245, in _run_visitor
>>> **kwargs).traverse_single(element)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py", line 56, in traverse_single
>>> ret = super(AlterTableVisitor, self).traverse_single(elem)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 120, in traverse_single
>>> return meth(obj, **kw)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/databases/mysql.py", line 47, in visit_column
>>> self.execute()
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py", line 44, in execute
>>> return self.connection.execute(self.buffer.getvalue())
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 721, in execute
>>> return self._execute_text(object, multiparams, params)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 870, in _execute_text
>>> statement, parameters
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 958, in _execute_context
>>> context)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1155, in _handle_dbapi_exception
>>> util.raise_from_cause(newraise, exc_info)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
>>> reraise(type(exception), exception, tb=exc_tb)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 951, in _execute_context
>>> context)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 436, in do_execute
>>> cursor.execute(statement, parameters)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
>>> self.errorhandler(self, exc, value)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
>>> raise errorclass, errorvalue
>>> OperationalError: (OperationalError) (1833, "Cannot change column 'uuid': used in a foreign key constraint 'block_device_mapping_instance_uuid_fkey' of table 'qzclautyzr.block_device_mapping'") '\nALTER TABLE instances CHANGE COLUMN uuid uuid VARCHAR(36) NOT NULL' ()
>>>
>>> Traceback (most recent call last):
>>> _StringException: Empty attachments:
>>> stdout
>>>
>>> stderr: {{{
>>> /home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:436: Warning: Duplicate index 'block_device_mapping_instance_uuid_virtual_name_device_name_idx' defined on the table 'qzclautyzr.block_device_mapping'. This is deprecated and will be disallowed in a future release.
>>> cursor.execute(statement, parameters)
>>> }}}
>>>
>>> Traceback (most recent call last):
>>> File "nova/tests/unit/db/test_migrations.py", line 151, in test_walk_versions
>>> self.walk_versions(self.snake_walk, self.downgrade)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 193, in walk_versions
>>> self.migrate_up(version, with_data=True)
>>> File "nova/tests/unit/db/test_migrations.py", line 148, in migrate_up
>>> super(NovaMigrationsCheckers, self).migrate_up(version, with_data)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 263, in migrate_up
>>> self.REPOSITORY, version)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py", line 186, in upgrade
>>> return _migrate(url, repository, version, upgrade=True, err=err, **opts)
>>> File "<string>", line 2, in _migrate
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/util/__init__.py", line 160, in with_engine
>>> return f(*a, **kw)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py", line 366, in _migrate
>>> schema.runchange(ver, change, changeset.step)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/schema.py", line 93, in runchange
>>> change.run(self.engine, step)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/script/py.py", line 148, in run
>>> script_func(engine)
>>> File "/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py", line 103, in upgrade
>>> process_null_records(meta, scan=False)
>>> File "/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py", line 89, in process_null_records
>>> table.columns.uuid.alter(nullable=False)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py", line 534, in alter
>>> return alter_column(self, *p, **k)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py", line 141, in alter_column
>>> engine._run_visitor(visitorcallable, delta)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1616, in _run_visitor
>>> conn._run_visitor(visitorcallable, element, **kwargs)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1245, in _run_visitor
>>> **kwargs).traverse_single(element)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py", line 56, in traverse_single
>>> ret = super(AlterTableVisitor, self).traverse_single(elem)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 120, in traverse_single
>>> return meth(obj, **kw)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/databases/mysql.py", line 47, in visit_column
>>> self.execute()
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py", line 44, in execute
>>> return self.connection.execute(self.buffer.getvalue())
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 721, in execute
>>> return self._execute_text(object, multiparams, params)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 870, in _execute_text
>>> statement, parameters
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 958, in _execute_context
>>> context)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1155, in _handle_dbapi_exception
>>> util.raise_from_cause(newraise, exc_info)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
>>> reraise(type(exception), exception, tb=exc_tb)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 951, in _execute_context
>>> context)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 436, in do_execute
>>> cursor.execute(statement, parameters)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
>>> self.errorhandler(self, exc, value)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
>>> raise errorclass, errorvalue
>>> OperationalError: (OperationalError) (1833, "Cannot change column 'uuid': used in a foreign key constraint 'block_device_mapping_instance_uuid_fkey' of table 'qzclautyzr.block_device_mapping'") '\nALTER TABLE instances CHANGE COLUMN uuid uuid VARCHAR(36) NOT NULL' ()
>>>
>>>
>>>
>>> Captured stderr:
>>> ~~~~~~~~~~~~~~~~
>>> /home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:436: Warning: Duplicate index 'block_device_mapping_instance_uuid_virtual_name_device_name_idx' defined on the table 'qzclautyzr.block_device_mapping'. This is deprecated and will be disallowed in a future release.
>>> cursor.execute(statement, parameters)
>>>
>>> Slowest Tests
>>> Test id Runtime (s)
>>> ----------------------------------------------------------------------------- -----------
>>> nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.test_walk_versions 51.553
>>>
>>> ==============================
>>> Failed 1 tests - output below:
>>> ==============================
>>>
>>> nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.test_walk_versions
>>> -----------------------------------------------------------------------------
>>>
>>> Captured traceback:
>>> ~~~~~~~~~~~~~~~~~~~
>>> Traceback (most recent call last):
>>> File "nova/tests/unit/db/test_migrations.py", line 151, in test_walk_versions
>>> self.walk_versions(self.snake_walk, self.downgrade)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 193, in walk_versions
>>> self.migrate_up(version, with_data=True)
>>> File "nova/tests/unit/db/test_migrations.py", line 148, in migrate_up
>>> super(NovaMigrationsCheckers, self).migrate_up(version, with_data)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 263, in migrate_up
>>> self.REPOSITORY, version)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py", line 186, in upgrade
>>> return _migrate(url, repository, version, upgrade=True, err=err, **opts)
>>> File "<string>", line 2, in _migrate
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/util/__init__.py", line 160, in with_engine
>>> return f(*a, **kw)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py", line 366, in _migrate
>>> schema.runchange(ver, change, changeset.step)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/schema.py", line 93, in runchange
>>> change.run(self.engine, step)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/script/py.py", line 148, in run
>>> script_func(engine)
>>> File "/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py", line 103, in upgrade
>>> process_null_records(meta, scan=False)
>>> File "/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py", line 89, in process_null_records
>>> table.columns.uuid.alter(nullable=False)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py", line 534, in alter
>>> return alter_column(self, *p, **k)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py", line 141, in alter_column
>>> engine._run_visitor(visitorcallable, delta)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1616, in _run_visitor
>>> conn._run_visitor(visitorcallable, element, **kwargs)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1245, in _run_visitor
>>> **kwargs).traverse_single(element)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py", line 56, in traverse_single
>>> ret = super(AlterTableVisitor, self).traverse_single(elem)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 120, in traverse_single
>>> return meth(obj, **kw)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/databases/mysql.py", line 47, in visit_column
>>> self.execute()
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py", line 44, in execute
>>> return self.connection.execute(self.buffer.getvalue())
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 721, in execute
>>> return self._execute_text(object, multiparams, params)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 870, in _execute_text
>>> statement, parameters
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 958, in _execute_context
>>> context)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1155, in _handle_dbapi_exception
>>> util.raise_from_cause(newraise, exc_info)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
>>> reraise(type(exception), exception, tb=exc_tb)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 951, in _execute_context
>>> context)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 436, in do_execute
>>> cursor.execute(statement, parameters)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
>>> self.errorhandler(self, exc, value)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
>>> raise errorclass, errorvalue
>>> OperationalError: (OperationalError) (1833, "Cannot change column 'uuid': used in a foreign key constraint 'block_device_mapping_instance_uuid_fkey' of table 'qzclautyzr.block_device_mapping'") '\nALTER TABLE instances CHANGE COLUMN uuid uuid VARCHAR(36) NOT NULL' ()
>>> Traceback (most recent call last):
>>> _StringException: Empty attachments:
>>> stdout
>>>
>>> stderr: {{{
>>> /home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:436: Warning: Duplicate index 'block_device_mapping_instance_uuid_virtual_name_device_name_idx' defined on the table 'qzclautyzr.block_device_mapping'. This is deprecated and will be disallowed in a future release.
>>> cursor.execute(statement, parameters)
>>> }}}
>>>
>>> Traceback (most recent call last):
>>> File "nova/tests/unit/db/test_migrations.py", line 151, in test_walk_versions
>>> self.walk_versions(self.snake_walk, self.downgrade)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 193, in walk_versions
>>> self.migrate_up(version, with_data=True)
>>> File "nova/tests/unit/db/test_migrations.py", line 148, in migrate_up
>>> super(NovaMigrationsCheckers, self).migrate_up(version, with_data)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 263, in migrate_up
>>> self.REPOSITORY, version)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py", line 186, in upgrade
>>> return _migrate(url, repository, version, upgrade=True, err=err, **opts)
>>> File "<string>", line 2, in _migrate
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/util/__init__.py", line 160, in with_engine
>>> return f(*a, **kw)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py", line 366, in _migrate
>>> schema.runchange(ver, change, changeset.step)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/schema.py", line 93, in runchange
>>> change.run(self.engine, step)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/script/py.py", line 148, in run
>>> script_func(engine)
>>> File "/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py", line 103, in upgrade
>>> process_null_records(meta, scan=False)
>>> File "/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py", line 89, in process_null_records
>>> table.columns.uuid.alter(nullable=False)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py", line 534, in alter
>>> return alter_column(self, *p, **k)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py", line 141, in alter_column
>>> engine._run_visitor(visitorcallable, delta)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1616, in _run_visitor
>>> conn._run_visitor(visitorcallable, element, **kwargs)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1245, in _run_visitor
>>> **kwargs).traverse_single(element)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py", line 56, in traverse_single
>>> ret = super(AlterTableVisitor, self).traverse_single(elem)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 120, in traverse_single
>>> return meth(obj, **kw)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/databases/mysql.py", line 47, in visit_column
>>> self.execute()
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py", line 44, in execute
>>> return self.connection.execute(self.buffer.getvalue())
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 721, in execute
>>> return self._execute_text(object, multiparams, params)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 870, in _execute_text
>>> statement, parameters
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 958, in _execute_context
>>> context)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1155, in _handle_dbapi_exception
>>> util.raise_from_cause(newraise, exc_info)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
>>> reraise(type(exception), exception, tb=exc_tb)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 951, in _execute_context
>>> context)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 436, in do_execute
>>> cursor.execute(statement, parameters)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
>>> self.errorhandler(self, exc, value)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
>>> raise errorclass, errorvalue
>>> OperationalError: (OperationalError) (1833, "Cannot change column 'uuid': used in a foreign key constraint 'block_device_mapping_instance_uuid_fkey' of table 'qzclautyzr.block_device_mapping'") '\nALTER TABLE instances CHANGE COLUMN uuid uuid VARCHAR(36) NOT NULL' ()
>>>
>>> Traceback (most recent call last):
>>> _StringException: Empty attachments:
>>> stdout
>>>
>>> stderr: {{{
>>> /home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:436: Warning: Duplicate index 'block_device_mapping_instance_uuid_virtual_name_device_name_idx' defined on the table 'qzclautyzr.block_device_mapping'. This is deprecated and will be disallowed in a future release.
>>> cursor.execute(statement, parameters)
>>> }}}
>>>
>>> Traceback (most recent call last):
>>> File "nova/tests/unit/db/test_migrations.py", line 151, in test_walk_versions
>>> self.walk_versions(self.snake_walk, self.downgrade)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 193, in walk_versions
>>> self.migrate_up(version, with_data=True)
>>> File "nova/tests/unit/db/test_migrations.py", line 148, in migrate_up
>>> super(NovaMigrationsCheckers, self).migrate_up(version, with_data)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/oslo/db/sqlalchemy/test_migrations.py", line 263, in migrate_up
>>> self.REPOSITORY, version)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py", line 186, in upgrade
>>> return _migrate(url, repository, version, upgrade=True, err=err, **opts)
>>> File "<string>", line 2, in _migrate
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/util/__init__.py", line 160, in with_engine
>>> return f(*a, **kw)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/api.py", line 366, in _migrate
>>> schema.runchange(ver, change, changeset.step)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/schema.py", line 93, in runchange
>>> change.run(self.engine, step)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/versioning/script/py.py", line 148, in run
>>> script_func(engine)
>>> File "/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py", line 103, in upgrade
>>> process_null_records(meta, scan=False)
>>> File "/home/mbayer/dev/openstack/nova/nova/db/sqlalchemy/migrate_repo/versions/267_instance_uuid_non_nullable.py", line 89, in process_null_records
>>> table.columns.uuid.alter(nullable=False)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py", line 534, in alter
>>> return alter_column(self, *p, **k)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/schema.py", line 141, in alter_column
>>> engine._run_visitor(visitorcallable, delta)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1616, in _run_visitor
>>> conn._run_visitor(visitorcallable, element, **kwargs)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1245, in _run_visitor
>>> **kwargs).traverse_single(element)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py", line 56, in traverse_single
>>> ret = super(AlterTableVisitor, self).traverse_single(elem)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/sql/visitors.py", line 120, in traverse_single
>>> return meth(obj, **kw)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/databases/mysql.py", line 47, in visit_column
>>> self.execute()
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/migrate/changeset/ansisql.py", line 44, in execute
>>> return self.connection.execute(self.buffer.getvalue())
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 721, in execute
>>> return self._execute_text(object, multiparams, params)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 870, in _execute_text
>>> statement, parameters
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 958, in _execute_context
>>> context)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1155, in _handle_dbapi_exception
>>> util.raise_from_cause(newraise, exc_info)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
>>> reraise(type(exception), exception, tb=exc_tb)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 951, in _execute_context
>>> context)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 436, in do_execute
>>> cursor.execute(statement, parameters)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
>>> self.errorhandler(self, exc, value)
>>> File "/home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
>>> raise errorclass, errorvalue
>>> OperationalError: (OperationalError) (1833, "Cannot change column 'uuid': used in a foreign key constraint 'block_device_mapping_instance_uuid_fkey' of table 'qzclautyzr.block_device_mapping'") '\nALTER TABLE instances CHANGE COLUMN uuid uuid VARCHAR(36) NOT NULL' ()
>>>
>>>
>>>
>>> Captured stderr:
>>> ~~~~~~~~~~~~~~~~
>>> /home/mbayer/dev/openstack/nova/.tox/py27/lib/python2.7/site-packages/sqlalchemy/engine/default.py:436: Warning: Duplicate index 'block_device_mapping_instance_uuid_virtual_name_device_name_idx' defined on the table 'qzclautyzr.block_device_mapping'. This is deprecated and will be disallowed in a future release.
>>> cursor.execute(statement, parameters)
>>>
>>>
>>>
>>> ======
>>> Totals
>>> ======
>>> Run: 1 in 51.553131 sec.
>>> - Passed: 0
>>> - Skipped: 0
>>> - Failed: 1
>>>
>>> ==============
>>> Worker Balance
>>> ==============
>>> - Worker 0 (1 tests) => 0:00:51.553131s
>>> ERROR: InvocationError: '/usr/bin/bash tools/pretty_tox.sh nova.tests.unit.db.test_migrations.TestNovaMigrationsMySQL.test_walk_versions'
>>> ____________________________________________________________________________________ summary ____________________________________________________________________________________
>>> ERROR: py27: commands failed
>>> [mbayer at thinkpad nova]$
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> We're running the migrations with Tempest in an internal CI with MariaDB and RHEL 7 and haven't had migration issues like this.
>>
>> The deprecation warning is odd, unless that's coming from the 216 migration where the duplicate index is defined, but it's removed in the 249 migration so that shouldn't be a problem.
>>
>> As for the 267 migration, it's failing to set the instances.uuid column to nullable=False. Before it does this, it scans the tables looking for tables with an instance_uuid column that has a foreign key back to the instances table and where the instance_uuid value is NULL, it will fail it if finds one of those (on purpose, you have to manually purge those rows).
>>
>> Once it's checked that there aren't any null instance_uuid entries with fkeys back to instances, it alters the instances.uuid column to be non-nullable so the UniqueConstraint on instances.uuid can be created.
>>
>> There is a known issue with a unique index being created on instances.uuid in the 216 migration and the 267 migration adds the unique constraint, so there is some duplication there, but I don't think that's what's causing your failure.
>>
>> --
>>
>> Thanks,
>>
>> Matt Riedemann
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
More information about the OpenStack-dev
mailing list