Open Stack

Use "neutron-db-manage history" to check what your alembic migration
current status is.

On Mon, Nov 28, 2022 at 12:03 PM Eugen Block <eblock at nde.ag> wrote:

> How do I check what the latest applied migration file was?
>
> Zitat von Rodolfo Alonso Hernandez <ralonsoh at redhat.com>:
>
> > Yes, but you should also be sure what is the status of the DB schema.
> That
> > means to check what is the latest migration file applied and set that
> > revision ID on the "neutron.alembic_version" table.
> >
> > On Mon, Nov 28, 2022 at 11:31 AM Eugen Block <eblock at nde.ag> wrote:
> >
> >> Hi,
> >>
> >> not really, no. I have no explanation how those files got there, to be
> >> honest. We're using openSUSE Leap (currently 15.2) and the respective
> >> repos from openSUSE. By the way, I only see those files on one of the
> >> control nodes, that's irritating me even more.
> >> But if those files are not known, maybe I should just delete them and
> >> the contract directories as well? Because during the next upgrade I'll
> >> probably have the same issue again. So if I see it correctly the two
> >> "contract" directories should be removed
> >>
> >>
> >>
> /usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/train/contract
> >>
> >>
> /usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/ussuri/contract
> >>
> >> as well as this revision file:
> >>
> >>
> >>
> /usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/train/expand/633d74ebbc4b_.py
> >>
> >> Comparing with the "native" V installation (and the other control
> >> node) I should only keep two of these files:
> >>
> >> controller01:~ # ll
> >>
> >>
> /usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/train/expand/
> >> insgesamt 16
> >> -rw-r--r-- 1 root root  900 30. Mär 2021  633d74ebbc4b_.py  <-- delete
> >> -rw-r--r-- 1 root root 1694 14. Nov 16:07
> 63fd95af7dcd_conntrack_helper.py
> >> -rw-r--r-- 1 root root  900 30. Mär 2021  6c9eb0469914_.py  <-- delete
> >> -rw-r--r-- 1 root root 1134 14. Nov 16:07
> >> c613d0b82681_subnet_force_network_id.py
> >> drwxr-xr-x 2 root root  312 23. Nov 11:09 __pycache__
> >>
> >> I believe that should clean it up. Then I'll import the merged neutron
> >> database and run the upgrade commands again. Does that make sense?
> >>
> >> Thanks!
> >> Eugen
> >>
> >> Zitat von Rodolfo Alonso Hernandez <ralonsoh at redhat.com>:
> >>
> >> > Hi Eugen:
> >> >
> >> > Please check the code you have. Those revisions (633d74ebbc4b,
> >> > bebe95aae4d4) do not exist in the Neutron repository. File [1] (or
> >> > something similar with the same prefix) does not exist. Are you using
> a
> >> > customized Neutron repository?
> >> >
> >> > Regards.
> >> >
> >> >
> >>
> [1]/usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/train/expand/633d74ebbc4b_.py
> >> >
> >> > On Fri, Nov 25, 2022 at 8:57 PM Eugen Block <eblock at nde.ag> wrote:
> >> >
> >> >> Hi,
> >> >>
> >> >> I believe they are neutron revisions, here's the output from
> >> >> yesterday's neutron-db-manage attempt:
> >> >>
> >> >> ---snip---
> >> >> Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO
> >> >> [alembic.runtime.migration] Context impl MySQLImpl.
> >> >> Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO
> >> >> [alembic.runtime.migration] Will assume non-transactional DDL.
> >> >> Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO
> >> >> [alembic.runtime.migration] Context impl MySQLImpl.
> >> >> Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO
> >> >> [alembic.runtime.migration] Will assume non-transactional DDL.
> >> >> Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO
> >> >> [alembic.runtime.migration] Running upgrade 5c85685d616d ->
> c43a0ddb6a03
> >> >> Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO
> >> >> [alembic.runtime.migration] Running upgrade c43a0ddb6a03 ->
> b5344a66e818
> >> >> Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO
> >> >> [alembic.runtime.migration] Running upgrade b5344a66e818 ->
> bebe95aae4d4
> >> >> Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO
> >> >> [alembic.runtime.migration] Running upgrade c613d0b82681 ->
> 6c9eb0469914
> >> >> Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO
> >> >> [alembic.runtime.migration] Running upgrade 6c9eb0469914 ->
> 633d74ebbc4b
> >> >> Nov 23 12:51:52 controller01 neutron-db-manage[25913]: Running
> upgrade
> >> >> for neutron ...
> >> >> Nov 23 12:51:52 controller01 neutron-db-manage[25913]: OK
> >> >> ---snip---
> >> >>
> >> >> And here's where they are located, apparently from train version:
> >> >>
> >> >> ---snip---
> >> >> controller01:~ # grep -r 633d74ebbc4b
> /usr/lib/python3.6/site-packages/
> >> >> Übereinstimmungen in Binärdatei
> >> >>
> >> >>
> >>
> /usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/train/expand/__pycache__/633d74ebbc4b_.cpython-36.pyc
> >> >>
> >>
> /usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/train/expand/633d74ebbc4b_.py:Revision
> >> >> ID:
> >> >> 633d74ebbc4b
> >> >>
> >>
> /usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/train/expand/633d74ebbc4b_.py:revision
> >> >> =
> >> >> '633d74ebbc4b'
> >> >> ---snip---
> >> >>
> >> >> Next week we'll try it with merging fresh victoria schema with our
> >> >> production data, then run the upgrade command again.
> >> >>
> >> >> Thanks,
> >> >> Eugen
> >> >> Zitat von Rodolfo Alonso Hernandez <ralonsoh at redhat.com>:
> >> >>
> >> >> > Hi Eugen:
> >> >> >
> >> >> > I don't know how it is possible that you have 3 registers in this
> >> table.
> >> >> > And the first two are not IDs of any Neutron revision. I would
> suggest
> >> >> you
> >> >> > to (1) check the DB schema deployed against a fresh deployed system
> >> (in
> >> >> > Victoria version) and (2) fix this table to point to the correct
> >> revision
> >> >> > numbers.
> >> >> >
> >> >> > Regards.
> >> >> >
> >> >> >
> >> >> > On Fri, Nov 25, 2022 at 3:51 PM Eugen Block <eblock at nde.ag> wrote:
> >> >> >
> >> >> >> Hi,
> >> >> >>
> >> >> >> thanks for your quick response.
> >> >> >>
> >> >> >> > In Neutron we don't support contract operations since Newton.
> >> >> >> >
> >> >> >> > If you are in Victoria and you correctly finished the DB
> migration,
> >> >> your
> >> >> >> > HEADs should be:
> >> >> >> > * contract: 5c85685d616d (from Newton)
> >> >> >> > * expand: I38991de2b4 (from the last DB change in Victoria,
> >> >> >> > source_and_destination_ip_prefix_neutron_metering_rule)
> >> >> >>
> >> >> >> That explains why I saw the newton revision in a new victoria
> cluster
> >> >> :-)
> >> >> >>
> >> >> >> > Please check what you have in the DB table
> neutron.alembic_version.
> >> >> The
> >> >> >> > first register should be the expand number, the second the
> contract
> >> >> one.
> >> >> >> If
> >> >> >> > not, update them with the ones I've provided.
> >> >> >>
> >> >> >> The table alembic_versions contains the three versions I provided
> at
> >> >> >> the end of my email:
> >> >> >>
> >> >> >> MariaDB [neutron]> select * from alembic_version;
> >> >> >> +--------------+
> >> >> >> | version_num  |
> >> >> >> +--------------+
> >> >> >> | 633d74ebbc4b |
> >> >> >> | bebe95aae4d4 |
> >> >> >> | I38991de2b4  |
> >> >> >> +--------------+
> >> >> >>
> >> >> >> I already tried to manipulate the table so I would only have those
> >> two
> >> >> >> versions you already mentioned, but then the upgrade --expand
> command
> >> >> >> alternates the database again with the mentioned error message
> >> >> >> ("Multiple heads are present").
> >> >> >>
> >> >> >> > Before executing the
> >> >> >> > migration tool again, be sure the DB schema matches the latest
> >> >> migration
> >> >> >> > patch for your version. You can deploy a VM with devstack and
> run
> >> this
> >> >> >> > version.
> >> >> >>
> >> >> >> That's what I wanted to try next, export only the db schema (no
> data)
> >> >> >> from a working victoria neutron database, then export only data
> from
> >> >> >> our production db and merge those, then import that into the
> >> >> >> production and try to run upgrade --expand and --contract again.
> But
> >> I
> >> >> >> didn't want to fiddle around too much in the production, that's
> why I
> >> >> >> wanted to ask for your guidance first.
> >> >> >> But IIUC even if I changed the table alembic_versions again and
> >> import
> >> >> >> the merged db, wouldn't upgrade --expand somehow try to alternate
> the
> >> >> >> table again? I don't see where the train revision comes from
> exactly,
> >> >> >> could you clarify, please? It seems like I always get back to
> square
> >> >> >> one when running the --expand command.
> >> >> >>
> >> >> >> Thanks!
> >> >> >> Eugen
> >> >> >>
> >> >> >> Zitat von Rodolfo Alonso Hernandez <ralonsoh at redhat.com>:
> >> >> >>
> >> >> >> > Hi Eugen:
> >> >> >> >
> >> >> >> > In Neutron we don't support contract operations since Newton.
> >> >> >> >
> >> >> >> > If you are in Victoria and you correctly finished the DB
> migration,
> >> >> your
> >> >> >> > HEADs should be:
> >> >> >> > * contract: 5c85685d616d (from Newton)
> >> >> >> > * expand: I38991de2b4 (from the last DB change in Victoria,
> >> >> >> > source_and_destination_ip_prefix_neutron_metering_rule)
> >> >> >> >
> >> >> >> > Please check what you have in the DB table
> neutron.alembic_version.
> >> >> The
> >> >> >> > first register should be the expand number, the second the
> contract
> >> >> one.
> >> >> >> If
> >> >> >> > not, update them with the ones I've provided. Before executing
> the
> >> >> >> > migration tool again, be sure the DB schema matches the latest
> >> >> migration
> >> >> >> > patch for your version. You can deploy a VM with devstack and
> run
> >> this
> >> >> >> > version.
> >> >> >> >
> >> >> >> > Regards.
> >> >> >> >
> >> >> >> >
> >> >> >> > On Fri, Nov 25, 2022 at 1:58 PM Eugen Block <eblock at nde.ag>
> wrote:
> >> >> >> >
> >> >> >> >> Hi *,
> >> >> >> >>
> >> >> >> >> I'd like to ask you for advice on how to clean up my neutron
> db.
> >> At
> >> >> >> >> some point (which I don't know exactly, probably train) my
> neutron
> >> >> >> >> database got inconsistent, apparently one of the upgrades did
> not
> >> go
> >> >> >> >> as planned. The interesting thing is that the database still
> >> works, I
> >> >> >> >> just upgraded from ussuri to victoria where that issue popped
> up
> >> >> again
> >> >> >> >> during 'neutron-db-manage upgrade --expand', I'll add the
> >> information
> >> >> >> >> at the end of this email. Apparently, I have multiple heads,
> and
> >> one
> >> >> >> >> of them is from train, it seems as if I never ran --contract
> (or
> >> it
> >> >> >> >> failed and I didn't notice).
> >> >> >> >> Just some additional information what I did with this database:
> >> this
> >> >> >> >> cloud started out as a test environment with a single control
> node
> >> >> and
> >> >> >> >> then became a production environment. About two and a half
> years
> >> ago
> >> >> >> >> we decided to reinstall this cloud with version ussuri and
> import
> >> the
> >> >> >> >> databases. I had a virtual machine in which I upgraded the
> >> database
> >> >> >> >> dump from production to the latest versions at that time. That
> all
> >> >> >> >> worked quite well, I only didn't notice that something was
> >> missing.
> >> >> >> >> Now that I finished the U --> V upgrade I want to fix this
> >> >> >> >> inconsistency, I just have no idea how to do it. As I'm not
> sure
> >> how
> >> >> >> >> all the neutron-db-manage commands work exactly I'd like to ask
> >> for
> >> >> >> >> some guidance. For example, could the "stamp" command possibly
> >> help?
> >> >> >> >> Or how else can I get rid of the train head and/or how to get
> the
> >> >> >> >> train revision to "contract" so I can finish the upgrade and
> >> contract
> >> >> >> >> the victoria revision? I can paste the whole neutron-db
> history if
> >> >> >> >> necessary (neutron-db-manage history), please let me know what
> >> >> >> >> information would be required to get to the bottom of this.
> >> >> >> >> Any help is greatly appreciated!
> >> >> >> >>
> >> >> >> >> Thanks!
> >> >> >> >> Eugen
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> ---snip---
> >> >> >> >> controller01:~ # neutron-db-manage upgrade --expand
> >> >> >> >> [...]
> >> >> >> >> alembic.script.revision.MultipleHeads: Multiple heads are
> present
> >> for
> >> >> >> >> given argument 'expand at head'; 633d74ebbc4b, I38991de2b4
> >> >> >> >>
> >> >> >> >> controller01:~ # neutron-db-manage current --verbose
> >> >> >> >>    Running current for neutron ...
> >> >> >> >> INFO  [alembic.runtime.migration] Context impl MySQLImpl.
> >> >> >> >> INFO  [alembic.runtime.migration] Will assume non-transactional
> >> DDL.
> >> >> >> >> Current revision(s) for
> >> mysql+pymysql://neutron:XXXXX@controller.fqdn
> >> >> >> >> /neutron:
> >> >> >> >> Rev: bebe95aae4d4 (head)
> >> >> >> >> Parent: b5344a66e818
> >> >> >> >> Branch names: contract
> >> >> >> >> Path:
> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> /usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/ussuri/contract/bebe95aae4d4_.py
> >> >> >> >>
> >> >> >> >> Rev: 633d74ebbc4b (head)
> >> >> >> >> Parent: 6c9eb0469914
> >> >> >> >> Branch names: expand
> >> >> >> >> Path:
> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> /usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/train/expand/633d74ebbc4b_.py
> >> >> >> >>
> >> >> >> >> Rev: I38991de2b4 (head)
> >> >> >> >> Parent: 49d8622c5221
> >> >> >> >> Branch names: expand
> >> >> >> >> Path:
> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >>
> >>
> /usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/victoria/expand/I38991de2b4_source_and_destination_ip_prefix_neutron_metering_rule.py
> >> >> >> >>
> >> >> >> >>    OK
> >> >> >> >> ---snip---
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >>
> >>
> >>
> >>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20221128/36104343/attachment-0001.htm>