Open Stack

2015-07-07 21:39 GMT+09:00 Henry Gessau <gessau at cisco.com>:

>  On Tue, Jul 07, 2015, Paul Michali <pc at michali.net> <pc at michali.net>
> wrote:
>
> Thanks Salvatore for the responses. See @PCM in-line...
>
>
>
>  On Tue, Jul 7, 2015 at 6:14 AM Salvatore Orlando <sorlando at nicira.com>
> wrote:
>
>> Some comments inline.
>>
>>  Salvatore
>>
>>    On 6 July 2015 at 20:00, Paul Michali < <pc at michali.net>pc at michali.net
>> > wrote:
>>
>>> Hi,
>>>
>>>  I have some urgent requests about migration that I'm hoping to get
>>> some info on. I'm working on a bug where I need to add two (related) fields
>>> to a table for VPNaaS. Here's the objectives related to migration...
>>>
>>>  1) create local_v4_ip and lcoal_v6_ip fields in the vpnservice table
>>> 2) for each entry in the vpnservice table:
>>>     2.1) Get the router.gw_port.fixed_ips list
>>>     2.2) Determine the version of each fixed IP and store the first of
>>> each version (if any) into the appropriate new field.
>>>
>>>  I have created a migration file, and I changed the down_revision to be
>>> the number of the revision that is the first in the migration chain in the
>>> VPN repo.
>>>
>>>  Here are the many questions I have...
>>>
>>>  When I look in the VPN repo, the HEAD file has the version 'kilo',
>>> which is not the current head.
>>>
>>
>>>  Shouldn't it the version number of the first file in the migration
>>> chain?
>>>
>>
>>      It should indeed. How are you generating the revision script? Using
>> neutron-db-manage it should be updated automatically [1]
>>
>
>  @PCM I ran neutron-db-manage, when in the neutron repo, and it assigned
> some version, but it was not the latest in the neutron-vpnaas repo.
>
> neutron-db-manage does not handle alembic branches in separate repos very
> well at all yet. I am working on updating it with
> <https://review.openstack.org/198524>https://review.openstack.org/198524
> but I have quite a lot left to do.
>

Yes, at now we have implicit order of running alembic migrations.
First run neutron db migration and then advanced service migrations.

I do not fully understand how alembic branch mechanism works, but
I think we can have a common ancestor and have multiple branches,
and each branch can evolve independently.


>
>  I checked the VPN repo and there were a chain of versions, which I used
> to determine what the head should be and have set the version accordingly.
> However, in the current repo, head is set to "kilo", which appears to be
> incorrect.  The versions are:
>
>  56893333aa52
> kilo   <<< HEAD
> 3ea02b2a773e
> start_neutron_vpnaas
> None
>
> Ouch. That is an error, because <https://review.openstack.org/190569>
> https://review.openstack.org/190569 should have updated HEAD but didn't.
>
> The version sequence (you can see it in any devstack run) is:
>
> INFO  [alembic.migration] Running upgrade  -> start_neutron_vpnaas, start
> neutron-vpnaas chain
> INFO  [alembic.migration] Running upgrade start_neutron_vpnaas ->
> 3ea02b2a773e, add_index_tenant_id
> INFO  [alembic.migration] Running upgrade 3ea02b2a773e -> kilo, kilo
> INFO  [alembic.migration] Running upgrade kilo -> 56893333aa52, fix
> identifier map fk
>

It seems we don't have an appropriate check for HEAD revision in at least
VPNaaS repo.
Paul and I just discussed it. We need to improve the check too.


> Should I do a separate commit that fixes the HEAD file, or just fix it as
> part of the bug fix I'm working on.
>
> Yes, you should immediately submit a patch to change HEAD to 56893333aa52.
>
>
>  BTW, at one point, after having correctly set the HEAD and versions in
> my new migration file, I think I ran neutron-db-manage check_migration, and
> I think it set the HEAD to my version, but it did that in the neutron repo,
> and not the VPN repo.  I might have been running from the wrong repo?
>
> I working on updating the devref docs for this process. Things have
> changed quite a bit with the alembic branches in separate repos.
>
>
>
>
>>   For my commit, I'm assuming I change the HEAD file to use my migration
>>> file's version?
>>>
>>
>>      You can do that manually too, yes.
>>
>>
>>>
>>>  I set HEAD to my migration file, and my file has a down revision of
>>> the previous head's revision. If I run 'neutron-db-manage --config-file
>>> ../neutron/etc/neutron.conf --config-file
>>> ../neutron/etc/neutron/plugins/ml2/ml2_conf.ini check_migration' there is
>>> no output so I guess that is OK.
>>>
>>>  As I develop my new migration file, is there a way that I can test it
>>> (running neutron-db-migration, maybe)?
>>>
>>
>>      When I test migrations I usually dump the database, run the
>> migration with neutron-db-manage upgrade HEAD (I think it's not necessary
>> to specify HEAD), and restore the db from the dump if the migration fails.
>>
>>
>>>  Is there a way to run the migration file under the debugger, as well
>>> (importing pdb, for example)?
>>>
>>
>>      The migration process is just like any python application, so I
>> guess you can debug it with pdb.
>>
>
>  @PCM Ah, so use "neutron-db-manage upgrade HEAD". That was the piece
> that was missing. I take it there are no specific unit tests of the
> migration files?
>
>
>
>>
>>>
>>>  In the migration, I can add the columns needed. What's the best way to
>>> fill out those fields - using raw SQL queries or create a Session object
>>> and access the VpnService object's router object?
>>>
>>
>>      If the default value for the column is not enough, and you need to
>> specify a value which depends on other values in the same row I would
>> prefer plain SQL statements, but if that become cumbersome I guess it's ok
>> to use sqlalchemy's session.
>>
>>
>>>  I see there is some op.bind() call and then engine.execute(), but
>>> could use some help on the best way to extract the needed queries (I need
>>> to access the vpnservice's router, and then access the (Port) gw_port
>>> relationship, and from that access the (IPAllocation) fixed_ips list).
>>>
>>
>>      Perhaps you can point us to the review pages on gerrit, and we can
>> provide detailed comments there.
>>
>
>  @PCM Yeah, I haven't pushed it up yet. I have a few more changes to
> make, and should be able to get it up in a few days. The LP bug is 1464387.
>
>  Essentially, in the vpnservices table, I'm adding an IPv4 and/or IPv6
> addresses for the  "local" end of VPN connections that will be established.
> This is to allow alternative VPN implementation (appliances, separate S/W,
> H/W, VM based VPN, etc) to specify addresses different than what is
> available on the Neutron router.
>
>  However, for the reference implementation, we'll use the Neutron
> router's fixed_ips list (as is done today), and to handle the migration,
> I'm thinking the following is needed:
>
>  1) create the new columns.
> 2) Identify the router for that service and obtain it's GW fixed_ips list.
> 3) Pick first IPv4 address (if any) and IPv6 address (if any), and store
> in new columns.
>
>  So I need to form a query and code to do this.
>
>
>
>>
>>>
>>>  Appreciate any advise here on how to debug the migration stuff...
>>>
>>>  Paul Michali (pc_m)
>>>
>>
>>      [1]
>> http://git.openstack.org/cgit/openstack/neutron/tree/neutron/db/migration/cli.py#n124
>>
>>
>>>
>>>
>>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150707/d288d37d/attachment.html>