Open Stack

On Wed, Jun 24, 2020 at 10:22 PM Rodolfo Alonso Hernandez
<ralonsoh at redhat.com> wrote:
>
> Hello all:
>
> Along this years we have increased the number of DB migrations each time we needed a new DB schema. This is good because that means the project is evolving and adding new features.
>
> Although this is not a problem per se, there are some inconvenients:
> - Every time a system is deployed (for example in the CI using devstack), the initial DB schema is created. Then, each migration is applied sequentially.
> - Some FT tests are still checking the sanity of some migrations [1] implemented a few releases ago.
> - We are still testing the contract DB migrations. Of course, this is something supported before and we still need to apply those revisions.
> - "TestWalkMigrationsMysql" and "TestModelsMigrationsMysql", both using MySQL backend, are still affected by LP#1687027.
>
> The proposal is to remove some DB migrations, starting from Liberty; of course, because all migrations must be applied in a specific order, we should begin from the initial revision, "kilo". The latest migration to be removed should be decided depending on the stable releases support.
>
> Apart from mitigating or solving some of the commented problems, we can "group" the DB model definition in one place. E.g.: "subnetpools" table is created in "other_extensions_init_ops". This file contains the first table. However is modified in at least two migrations:
> - 1b4c6e320f79_address_scope_support_in_subnetpool: added "address_scope_id" field
> - 13cfb89f881a_add_is_default_to_subnetpool: added "is_default" field
>
> Instead of having (at least) three places where the "subnetpools" DB schema is defined, we can remove the Mitaka migration and group this definition in just one place.
>
> One possible issue: some migrations add dependencies on other tables. That means the table the dependency is referring should be created in advance. That implies that, in some cases, the table creation order should be modified. That should never affect subsequent created tables or migrations.
>
> Do you see any inconvenience on this proposal? Am I missing something that I didn't consider?
>
> Thank you and regards.
>
> [1]https://github.com/openstack/neutron/blob/9fd60ffaac6b178de62dab169c826d52f7bfbb2d/neutron/tests/functional/db/test_migrations.py

Hi,

Simplification sounds good in general.

Previously (up to Liberty release or some), we squashed all
migrationed up to a specific past release.
If you look at the git log of
neutron/db/migration/alembic_migrations/versions/kilo_initial.py, you
can see an example.
However, it was stopped as squashing migrations needs to be done very
carefully and even if we don't squash migrations
the overhead of alembic migrations is not so high.
You now raise this again, so it might be time to revisit it, so I am
not against your proposal in general.

I am not sure what you mean by "remove some DB migrations".
Squashing migrations only related to some tables potentially
introduces some confusion.

A simpler approach looks like to merge all migrations up to a specific
release (queens or rocky?).
I think this approach addresses the problems you mentioned above.

Thought?

Akihiro