Open Stack

On Tue, Jul 16, 2013 at 11:51 AM, Roman Podolyaka
<rpodolyaka at mirantis.com>wrote:

> Hello, stackers!
>
> Most of you who is interested in work around DB in OpenStack must have
> read this thread [1] started by Boris Pavlovic. Boris made an overview of
> the work our team is doing to make DB code better.
>
> One of our main goals is to switch from sqlalchemy-migrate to Alembic for
> applying of DB schema migrations. sqlalchemy-migrate was abandoned for a
> long time, and even now when it's become adopted by OpenStack community,
> we'd better use a project which is supported by upstream (especially in the
> case when the author of this project is the same person who also authored
> SQLAlchemy).
>
> The switch isn't going to be simple though. We have a few problems:
>
> 1) stable releases must be supported for some time, so we can't switch
> from migrate to alembic immediately
>
> The switch should probably be made when previous migrations scripts are
> "compacted", so all new migrations scripts will use alembic. Switching of
> such big projects as Nova is hard, so we decided to gain some experience
> with porting of smaller ones first. Alexei Kornienko is currently working
> on adding support of Alembic migrations in Ceilometer [3].
>

I like the idea of switching when we do a major release. I don't think we
need to port the old migrations to alembic, though, as I said to Alexei on
the review. We should be able to continue to have sqlalchemy-migrate
distributed as part of OpenStack for the legacy migrations until those
migrations can be dropped entirely. Updating would mean running
sqlalchemy-migrate, and then alembic, to apply the full set of migrations.
>From what I understand the policy to be, since we have sqlalchemy-migrate
scripts in Havana we could stop creating new ones in Icehouse and drop the
use of sqlalchemy-migrate entirely in the J release when the Havana
migrations are removed.

Doug


>
> Our long term goal is to switch all projects from using of
> sqlalchemy-migrate to Alembic.
>
> 2) we rely on schema migrations to set up an SQLite database for running
> tests
>
> Nova and possibly other projects use schema migrations to set up an SQLite
> database for running tests. Unfortunately, we can't use models definitions
> for generation of initial DB schema, because those definitions do not
> correspond migration scripts. Our team is working on fixing of this issue
> [2].
>

> As you may now SQLite has limited support of ALTER DDL statements [4].
> Nova code contains a few auxiliary functions to make ALTER work in SQLite.
> Unfortunately, Alembic doesn't support ALTER in SQLite on purpose [5]. In
> order to run our tests on SQLite right now using Alembic as a schema
> migration tool, we should add ALTER support to it first.
>
> We are going to implement ALTER support in Alembic for SQLite in the next
> few weeks.
>
> As always, your comments in ML and reviews are always welcome.
>
> Thanks,
> Roman
>
> [1]
> http://lists.openstack.org/pipermail/openstack-dev/2013-July/011253.html
> [2]
> https://blueprints.launchpad.net/nova/+spec/db-sync-models-with-migrations
> [3]
> https://review.openstack.org/#/q/status:open+project:openstack/ceilometer+branch:master+topic:bp/convert-to-alembic,n,z
> [4] http://www.sqlite.org/lang_altertable.html
> [5] https://bitbucket.org/zzzeek/alembic
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130716/43368da5/attachment.html>