[openstack-dev] [neutron] Plethora of dbase migration questions...

Paul Michali pc at michali.net
Tue Jul 7 12:00:32 UTC 2015


Thanks Salvatore for the responses. See @PCM in-line...



On Tue, Jul 7, 2015 at 6:14 AM Salvatore Orlando <sorlando at nicira.com>
wrote:

> Some comments inline.
>
> Salvatore
>
> On 6 July 2015 at 20:00, Paul Michali <pc at michali.net> wrote:
>
>> Hi,
>>
>> I have some urgent requests about migration that I'm hoping to get some
>> info on. I'm working on a bug where I need to add two (related) fields to a
>> table for VPNaaS. Here's the objectives related to migration...
>>
>> 1) create local_v4_ip and lcoal_v6_ip fields in the vpnservice table
>> 2) for each entry in the vpnservice table:
>>     2.1) Get the router.gw_port.fixed_ips list
>>     2.2) Determine the version of each fixed IP and store the first of
>> each version (if any) into the appropriate new field.
>>
>> I have created a migration file, and I changed the down_revision to be
>> the number of the revision that is the first in the migration chain in the
>> VPN repo.
>>
>> Here are the many questions I have...
>>
>> When I look in the VPN repo, the HEAD file has the version 'kilo', which
>> is not the current head.
>>
>
>> Shouldn't it the version number of the first file in the migration chain?
>>
>
> It should indeed. How are you generating the revision script? Using
> neutron-db-manage it should be updated automatically [1]
>

@PCM I ran neutron-db-manage, when in the neutron repo, and it assigned
some version, but it was not the latest in the neutron-vpnaas repo.

I checked the VPN repo and there were a chain of versions, which I used to
determine what the head should be and have set the version accordingly.
However, in the current repo, head is set to "kilo", which appears to be
incorrect.  The versions are:

56893333aa52
kilo   <<< HEAD
3ea02b2a773e
start_neutron_vpnaas
None

Should I do a separate commit that fixes the HEAD file, or just fix it as
part of the bug fix I'm working on.

BTW, at one point, after having correctly set the HEAD and versions in my
new migration file, I think I ran neutron-db-manage check_migration, and I
think it set the HEAD to my version, but it did that in the neutron repo,
and not the VPN repo.  I might have been running from the wrong repo?



> For my commit, I'm assuming I change the HEAD file to use my migration
>> file's version?
>>
>
> You can do that manually too, yes.
>
>
>>
>> I set HEAD to my migration file, and my file has a down revision of the
>> previous head's revision. If I run 'neutron-db-manage --config-file
>> ../neutron/etc/neutron.conf --config-file
>> ../neutron/etc/neutron/plugins/ml2/ml2_conf.ini check_migration' there is
>> no output so I guess that is OK.
>>
>> As I develop my new migration file, is there a way that I can test it
>> (running neutron-db-migration, maybe)?
>>
>
> When I test migrations I usually dump the database, run the migration with
> neutron-db-manage upgrade HEAD (I think it's not necessary to specify
> HEAD), and restore the db from the dump if the migration fails.
>
>
>> Is there a way to run the migration file under the debugger, as well
>> (importing pdb, for example)?
>>
>
> The migration process is just like any python application, so I guess you
> can debug it with pdb.
>

@PCM Ah, so use "neutron-db-manage upgrade HEAD". That was the piece that
was missing. I take it there are no specific unit tests of the migration
files?



>
>>
>> In the migration, I can add the columns needed. What's the best way to
>> fill out those fields - using raw SQL queries or create a Session object
>> and access the VpnService object's router object?
>>
>
> If the default value for the column is not enough, and you need to specify
> a value which depends on other values in the same row I would prefer plain
> SQL statements, but if that become cumbersome I guess it's ok to use
> sqlalchemy's session.
>
>
>> I see there is some op.bind() call and then engine.execute(), but could
>> use some help on the best way to extract the needed queries (I need to
>> access the vpnservice's router, and then access the (Port) gw_port
>> relationship, and from that access the (IPAllocation) fixed_ips list).
>>
>
> Perhaps you can point us to the review pages on gerrit, and we can provide
> detailed comments there.
>

@PCM Yeah, I haven't pushed it up yet. I have a few more changes to make,
and should be able to get it up in a few days. The LP bug is 1464387.

Essentially, in the vpnservices table, I'm adding an IPv4 and/or IPv6
addresses for the  "local" end of VPN connections that will be established.
This is to allow alternative VPN implementation (appliances, separate S/W,
H/W, VM based VPN, etc) to specify addresses different than what is
available on the Neutron router.

However, for the reference implementation, we'll use the Neutron router's
fixed_ips list (as is done today), and to handle the migration, I'm
thinking the following is needed:

1) create the new columns.
2) Identify the router for that service and obtain it's GW fixed_ips list.
3) Pick first IPv4 address (if any) and IPv6 address (if any), and store in
new columns.

So I need to form a query and code to do this.



>
>>
>> Appreciate any advise here on how to debug the migration stuff...
>>
>> Paul Michali (pc_m)
>>
>
> [1]
> http://git.openstack.org/cgit/openstack/neutron/tree/neutron/db/migration/cli.py#n124
>
>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150707/3de1ffcd/attachment.html>


More information about the OpenStack-dev mailing list