[openstack-dev] [neutron][db] online schema upgrades

Anna Kamyshnikova akamyshnikova at mirantis.com
Thu Jun 18 09:54:22 UTC 2015


On Wed, Jun 17, 2015 at 8:23 PM, Mike Bayer <mbayer at redhat.com> wrote:

>
>
> On 6/17/15 12:40 PM, Ihar Hrachyshka wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 06/17/2015 11:27 AM, Anna Kamyshnikova wrote:
>
>  Ihar, thanks for bringing this up!
>
> This is very interesting and I think it worth trying. I'm +1 on
> that and want to participate in this work.
>
>
>  Awesome.
>
>
>  In fact a lot *not strict* migrations are removed with
> juno_initial, so I hope it won't be so hard for now to apply
> stricter rules for migration. But what is the plan for those
> migrations that are *not strict* now?
>
>  Still, we have several live data migrations from Juno to Kilo:
>
> - - 14be42f3d0a5_default_sec_group_table.py: populates db with default
> security groups;
>
> - - 16cdf118d31d_extra_dhcp_options_ipv6_support.py: populates
> extraqdhcpopts with default ip_version = 4;
>
> - - 2d2a8a565438_hierarchical_binding.py: populates db with
> port_binding_levels objects, then drops old tables;
>
> - - 35a0f3365720_add_port_security_in_ml2.py: port security field is
> populated with True for ports and networks;
>
> - - 034883111f_remove_subnetpool_allow_overlap.py: drops allow_overlap
> column from subnetpools: probably unused so we can be ok with it?..
>
> In any case, the plan for existing migration rules is: don't touch
> them. Their presence in N release just indicates that we cannot get
> online db migration in N+1. That's why we should adopt strict rules
> the earlier the better, so that opportunity does not slip to N+X where
> X is too far.
>
> The patches currently in review that look suspicious in this regard are:
> - - I4ff7db0f5fa12b0fd1c6b10318e9177fde0210d7: moves data from one table
> into another;
> - - Iecb3e168a805fc5f3d59d894c3f0d9298505e872: fills new columns with
> default server values (why is it even needed?..);
> - - Icde55742aa78ed995bac0896c01c80c9d28aa0cf: alter_column(). Not sure
> we are ok with it;
> - - I66b3ee8c2f9fa6f04b9e89dc49d1a3d277d63191: probably not an issue
> though since it touches existing live data impact rule?
>
>
> I made a list of migrations from juno-> kilo that are non expansive or do
> data migrations:
>
> *these contain "drop column":*
>
>  034883111f_remove_subnetpool_allow_overlap.py
> 2d2a8a565438_hierarchical_binding.py
>
>  *these contain "drop table":*
>
>  28c0ffb8ebbd_remove_mlnx_plugin.py
> 2b801560a332_remove_hypervneutronplugin_tables.py
> 408cfbf6923c_remove_ryu_plugin.py
> 57086602ca0a_scrap_nsx_adv_svcs_models.py
>
>  *these contain data migrations:*
>
>  14be42f3d0a5_default_sec_group_table.py
> 16cdf118d31d_extra_dhcp_options_ipv6_support.py
> 2b801560a332_remove_hypervneutronplugin_tables.py
> 2d2a8a565438_hierarchical_binding.py
> 35a0f3365720_add_port_security_in_ml2.py
>
>  *Example of failure:*
>
>  neutron/db/migration/alembic_migrations/versions/2d2a8a565438_hierarchical_binding.py
> - drops the following columns:
>
>      op.drop_constraint(fk_name_dvr[0], 'ml2_dvr_port_bindings',
> 'foreignkey')
>     op.drop_column('ml2_dvr_port_bindings', 'cap_port_filter')
>     op.drop_column('ml2_dvr_port_bindings', 'segment')
>     op.drop_column('ml2_dvr_port_bindings', 'driver')
>
>      op.drop_constraint(fk_name[0], 'ml2_port_bindings', 'foreignkey')
>     op.drop_column('ml2_port_bindings', 'driver')
>     op.drop_column('ml2_port_bindings', 'segment')
>
>     which then causes a failure in Juno:
>
>     OperationalError: (OperationalError) (1054, "Unknown column
> 'ml2_port_bindings_1.driver' in 'field list'")
>
>
In M release we will be able to create kilo_initial migration that will
hide all these migrations(juno -> kilo), so I as there is nothing we can do
we won't touch them as Ihar said. The problem about migrations that are on
review is more serious, as we should adopt strict rules as early as
possible and all core reviewer should be aware about that.

To Ihar's list of migrations on review  I can add :
I3426b13eede8bfa29729cf3efea3419fb91175c4 - insert some data,
I9cf36e1fd3a009c175e0d475af407a30f4e5c408 (almost ready to be merged!) -
drop tables.
And there are a lot changes with altering columns. One is merged in
neutron-vpnaas.
So we should decide these things quickly.




>
>
>
>  (the list can be incomplete)
>
>
>  I think that we should try to use Alembic as much we could as Mike
> is going to support us in that and we have time to make some change
> in Alembic directly.
>
>  Yes, sure, I'm looking forward to see Mike's proposal in public.
>
>
>  We should undoubtedly plan this work for M release because there
> will be some issues that will appear in the process.
>
>
>  Sure.
>
> Ihar
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iQEcBAEBCAAGBQJVgaL4AAoJEC5aWaUY1u57oZgH/34pgV7AqOiq4XnWOOmQ9HA9
> jL+8E9jv8pUSW3X4v0Rm5mDuWJyWscrgy61Om+sWsmqBFAmm/gmLWm+NNADbYM5e
> 6hsoaO5WmuvRc03MwIwsa0NEgfPc8EhT5JiZmYRjOBc85ZCs6+UOKUHBAI2EVTg8
> t8YKdTdzxlrZQEOng1lbsUQYkHnNUZTbsREnpangfaHXBk3xmilH/ebGsz3CRUCe
> OBrpp6q8N7mgZgK/UQKb04eS5bCna7eVmv6q7PvIO0SlYhhDbrL3+dv/SZpqQWZ/
> Hek2Oig0IYyPygVrGc4BpT9MIaKisGoxXMn1rRB2g8us8jM58VyzqgXwEH2H4Aw=
> =TqHb
> -----END PGP SIGNATURE-----
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Regards,
Ann Kamyshnikova
Mirantis, Inc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150618/507b01a2/attachment.html>


More information about the OpenStack-dev mailing list