Open Stack

Salvatore,

Thanks for your answer. I was on holidays, so I clearly read your response
just now.

As I understand from your comments there is no reason for me to continue
working on making migrations run unconditionally for service plugins.
Making all migrations run unconditionally based on current state of
database will be rather long process, because I'll have to find out which
tables are in database for each plugin and then create migration that will
recreate them, if they won't exist.

While I was working on this change https://review.openstack.org/87935 I
researched different opportunities how to check database state without
breaking offline migrations. The only suggestion is to use op.execute() and
SQL statement 'CREATE TABLE IF NOT EXIST'. The problem with that is that if
table uses enum type it can't be done for PostgreSQL. I described this
problem in earlier mail with topic 'Fix migration that breaks Grenade jobs'

If we won't care about offline migrations this could be done by rerunning
some migrations or recreating some tables, if running the whole migration
is not reliable.

Regards,
Ann


On Wed, Apr 30, 2014 at 6:03 PM, Salvatore Orlando <sorlando at nicira.com>wrote:

> Anna,
>
> It's good to see progress being made on this blueprint.
> I have some comments inline.
>
> Also, I would recommend keeping in mind the comments Mark had regarding
> migration generation and plugin configuration in hist post on the email
> thread I started.
>
> Salvatore
>
>
> On 30 April 2014 14:16, Anna Kamyshnikova <akamyshnikova at mirantis.com>wrote:
>
>> Hi everyone!
>>
>> I'm working on blueprint
>> https://blueprints.launchpad.net/neutron/+spec/neutron-robust-db-migrations.
>> This topic was discussed earlier by Salvatore in ML thread "Migrations,
>> service plugins and Grenade jobs". I'm researching how to make migrations
>> for service plugins run unconditionally. In fact it is enough to change
>> should_run method for migrations -  to make it return True if this
>> migration is for service plugin
>> https://review.openstack.org/#/c/91323/1/neutron/db/migration/__init__.py
>> .
>>
>
> I think running migrations unconditionally for service plugins is an
> advancement but not yet a final solution.
> I would insist on the path you've been pursuing of running unconditionally
> all migrations. We should strive to solve the issues you found there so far
>
>
>> It is almost working except for conflicts of VNPaaS service plugin with
>> Brocade and Nuage plugins http://paste.openstack.org/show/77946/.
>> 1) Migration for Brocade plugin fails, because 2c4af419145b_l3_support
>> doesn't run (it adds necessary table 'routers') It can be fixed by adding
>> Brocade plugin to list migration_for_plugins in 2c4af419145b_l3_support.
>>
> 2) Migration for Nuage plugin fails, because e766b19a3bb_nuage_initial
>> runs after 52ff27f7567a_support_for_vpnaas, and there is no necessary table
>> 'routers'.  It can be fixed by adding Nuage plugin to list
>> migration_for_plugins in 2c4af419145b_l3_support and removing addition of
>> these tables in e766b19a3bb_nuage_initial.
>>
>> I noticed that too in the past.
> However there are two aspects to this fix. The one you're mentioning is
> for fixing migrations in new deployments; on the other hand migrations
> should be fixed also for existing deployments. This kind of problem seems
> to me more concerning the work for removing schema auto-generation. Indeed
> the root problem here is probably that l3 schemas for these two plugins are
> being created only because of schema auto-generation.
>
> Also I researched opportunity to make all migrations run unconditionally,
>> but this could not be done as there is going to be a lot of conflicts
>> http://paste.openstack.org/show/77902/. Mostly this conflicts take place
>> in initial migrations for core plugins as they create basical tables that
>> was created before. For example, mlnx_initial create tables securitygroups,
>> securitygroupportbindings, networkdhcpagentbindings, portbindingports that
>> were already created in 3cb5d900c5de_securitygroups,
>> 4692d074d587_agent_scheduler, 176a85fc7d79_add_portbindings_db migrations.
>> Besides I was thinking about 'making migrations smart' - to make them check
>> whether all necessary migrations has been run or not, but the main problem
>> about that is we need to store information about applied migrations, so
>> this should have been planned from the very beginning.
>>
>>
> I agree that just changing migrations to run unconditionally is not a
> viable solution.
> Rather than changing the existing migration path, I was thinking more
> about adding "corrective" unconditional migrations to make the database
> state the same regardless of the plugin configuration. The difficult part
> here is that these migrations would need to be smart enough to understand
> whether a previous migration was executed or skipped; this might also break
> offline migrations (but perhaps this might be tolerable).
>
> I look forward for any suggestions or thoughts on this topic.
>>
>> Regards, Ann
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140505/f79dc8c6/attachment.html>