Open Stack

Hi!

It is nice to see so careful attention for my change requests. In fact they
are related the same topic that Johannes mentioned. The idea of this is
described there
https://blueprints.launchpad.net/neutron/+spec/db-sync-models-with-migrations
.

Problems that mentioned in https://review.openstack.org/#/c/82073/
https://review.openstack.org/#/c/80518/ were discovered with the help of
this test https://review.openstack.org/74081 that I adopted for Neutron
https://review.openstack.org/76520. It helps to find a lot of bugs in
Neutron database and models. Now as this change
https://review.openstack.org/74081 is going to move to oslo.db all of this
work is frozen.

Regards,
Ann



On Tue, May 20, 2014 at 9:02 AM, Johannes Erdfelt <johannes at erdfelt.com>wrote:

> On Tue, May 20, 2014, Collins, Sean <Sean_Collins2 at cable.comcast.com>
> wrote:
> > I've been looking at two reviews that Ann Kamyshnikova has proposed
> >
> > https://review.openstack.org/#/c/82073/
> >
> > https://review.openstack.org/#/c/80518/
> >
> > I think the changes are fundamentally a Good Thing™  - they appear to
> > reduce the differences between the database models and their
> > corresponding migrations – as well as fixing differences in the
> > generated DDL between Postgres and MySQL.
> >
> > The only thing I'm concerned about, is how to prevent these
> > inconsistencies from sneaking into the codebase in the future. The
> > one review that fixes ForeignKey constraints that are missing a name
> > argument which ends up failing to create indexes in Postgres – I can
> > see myself repeating that mistake, it's very subtle.
> >
> > Should we have some sort of HACKING for database models and
> > migrations? Are these problems subtle enough that they warrant
> > changes to SQLAlchemy/Alembic?
>
> On the Nova side of things, there has been similar concerns.
>
> There is a nova-spec that is proposing adding a unit test to check the
> schema versus the model:
>
> https://review.openstack.org/#/c/85325/
>
> This should work, but I think the underlying problem is DRY based. We
> should not need to declare a schema in a model and then a set of
> imperative tasks to get to that point. All too often they get
> unsynchronized.
>
> I informally proposed a different solution, moving schema migrations to
> a declarative model. I wrote a proof of concept to show how something
> like this would work:
>
> https://github.com/jerdfelt/muscle
>
> We already have a model written (but need some fixes to make it accurate
> wrt to existing migrations), we should be able to emit ALTER TABLE
> statements based on the existing schema to bring it into line with the
> model.
>
> This also has the added benefit of allowing certain schema migrations to
> be done online, while services are still running. This can significantly
> reduce downtime during deploys (a big concern for large deployments of
> Nova).
>
> There are some corner cases that do cause problems (renaming columns,
> changing column types, etc). Those can either remain as traditional
> migrations and/or discouraged.
>
> Data migrations would still remain with sqlalchemy-migrate/alembic, but
> there have some proposals about solving that problem too.
>
> JE
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140520/4caff980/attachment.html>