<div dir="ltr"><div>Hi Eugen:</div><div><br></div><div>Please check the code you have. Those revisions (633d74ebbc4b, bebe95aae4d4) do not exist in the Neutron repository. File [1] (or something similar with the same prefix) does not exist. Are you using a customized Neutron repository?</div><div><br></div><div>Regards.</div><div><br></div><div>[1]/usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/train/expand/633d74ebbc4b_.py</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Nov 25, 2022 at 8:57 PM Eugen Block <<a href="mailto:eblock@nde.ag">eblock@nde.ag</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br>
<br>
I believe they are neutron revisions, here's the output from <br>
yesterday's neutron-db-manage attempt:<br>
<br>
---snip---<br>
Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO <br>
[alembic.runtime.migration] Context impl MySQLImpl.<br>
Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO <br>
[alembic.runtime.migration] Will assume non-transactional DDL.<br>
Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO <br>
[alembic.runtime.migration] Context impl MySQLImpl.<br>
Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO <br>
[alembic.runtime.migration] Will assume non-transactional DDL.<br>
Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO <br>
[alembic.runtime.migration] Running upgrade 5c85685d616d -> c43a0ddb6a03<br>
Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO <br>
[alembic.runtime.migration] Running upgrade c43a0ddb6a03 -> b5344a66e818<br>
Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO <br>
[alembic.runtime.migration] Running upgrade b5344a66e818 -> bebe95aae4d4<br>
Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO <br>
[alembic.runtime.migration] Running upgrade c613d0b82681 -> 6c9eb0469914<br>
Nov 23 12:51:51 controller01 neutron-db-manage[25913]: INFO <br>
[alembic.runtime.migration] Running upgrade 6c9eb0469914 -> 633d74ebbc4b<br>
Nov 23 12:51:52 controller01 neutron-db-manage[25913]: Running upgrade <br>
for neutron ...<br>
Nov 23 12:51:52 controller01 neutron-db-manage[25913]: OK<br>
---snip---<br>
<br>
And here's where they are located, apparently from train version:<br>
<br>
---snip---<br>
controller01:~ # grep -r 633d74ebbc4b /usr/lib/python3.6/site-packages/<br>
Übereinstimmungen in Binärdatei <br>
/usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/train/expand/__pycache__/633d74ebbc4b_.cpython-36.pyc<br>
/usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/train/expand/633d74ebbc4b_.py:Revision ID: <br>
633d74ebbc4b<br>
/usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/train/expand/633d74ebbc4b_.py:revision = <br>
'633d74ebbc4b'<br>
---snip---<br>
<br>
Next week we'll try it with merging fresh victoria schema with our <br>
production data, then run the upgrade command again.<br>
<br>
Thanks,<br>
Eugen<br>
Zitat von Rodolfo Alonso Hernandez <<a href="mailto:ralonsoh@redhat.com" target="_blank">ralonsoh@redhat.com</a>>:<br>
<br>
> Hi Eugen:<br>
><br>
> I don't know how it is possible that you have 3 registers in this table.<br>
> And the first two are not IDs of any Neutron revision. I would suggest you<br>
> to (1) check the DB schema deployed against a fresh deployed system (in<br>
> Victoria version) and (2) fix this table to point to the correct revision<br>
> numbers.<br>
><br>
> Regards.<br>
><br>
><br>
> On Fri, Nov 25, 2022 at 3:51 PM Eugen Block <<a href="mailto:eblock@nde.ag" target="_blank">eblock@nde.ag</a>> wrote:<br>
><br>
>> Hi,<br>
>><br>
>> thanks for your quick response.<br>
>><br>
>> > In Neutron we don't support contract operations since Newton.<br>
>> ><br>
>> > If you are in Victoria and you correctly finished the DB migration, your<br>
>> > HEADs should be:<br>
>> > * contract: 5c85685d616d (from Newton)<br>
>> > * expand: I38991de2b4 (from the last DB change in Victoria,<br>
>> > source_and_destination_ip_prefix_neutron_metering_rule)<br>
>><br>
>> That explains why I saw the newton revision in a new victoria cluster :-)<br>
>><br>
>> > Please check what you have in the DB table neutron.alembic_version. The<br>
>> > first register should be the expand number, the second the contract one.<br>
>> If<br>
>> > not, update them with the ones I've provided.<br>
>><br>
>> The table alembic_versions contains the three versions I provided at<br>
>> the end of my email:<br>
>><br>
>> MariaDB [neutron]> select * from alembic_version;<br>
>> +--------------+<br>
>> | version_num |<br>
>> +--------------+<br>
>> | 633d74ebbc4b |<br>
>> | bebe95aae4d4 |<br>
>> | I38991de2b4 |<br>
>> +--------------+<br>
>><br>
>> I already tried to manipulate the table so I would only have those two<br>
>> versions you already mentioned, but then the upgrade --expand command<br>
>> alternates the database again with the mentioned error message<br>
>> ("Multiple heads are present").<br>
>><br>
>> > Before executing the<br>
>> > migration tool again, be sure the DB schema matches the latest migration<br>
>> > patch for your version. You can deploy a VM with devstack and run this<br>
>> > version.<br>
>><br>
>> That's what I wanted to try next, export only the db schema (no data)<br>
>> from a working victoria neutron database, then export only data from<br>
>> our production db and merge those, then import that into the<br>
>> production and try to run upgrade --expand and --contract again. But I<br>
>> didn't want to fiddle around too much in the production, that's why I<br>
>> wanted to ask for your guidance first.<br>
>> But IIUC even if I changed the table alembic_versions again and import<br>
>> the merged db, wouldn't upgrade --expand somehow try to alternate the<br>
>> table again? I don't see where the train revision comes from exactly,<br>
>> could you clarify, please? It seems like I always get back to square<br>
>> one when running the --expand command.<br>
>><br>
>> Thanks!<br>
>> Eugen<br>
>><br>
>> Zitat von Rodolfo Alonso Hernandez <<a href="mailto:ralonsoh@redhat.com" target="_blank">ralonsoh@redhat.com</a>>:<br>
>><br>
>> > Hi Eugen:<br>
>> ><br>
>> > In Neutron we don't support contract operations since Newton.<br>
>> ><br>
>> > If you are in Victoria and you correctly finished the DB migration, your<br>
>> > HEADs should be:<br>
>> > * contract: 5c85685d616d (from Newton)<br>
>> > * expand: I38991de2b4 (from the last DB change in Victoria,<br>
>> > source_and_destination_ip_prefix_neutron_metering_rule)<br>
>> ><br>
>> > Please check what you have in the DB table neutron.alembic_version. The<br>
>> > first register should be the expand number, the second the contract one.<br>
>> If<br>
>> > not, update them with the ones I've provided. Before executing the<br>
>> > migration tool again, be sure the DB schema matches the latest migration<br>
>> > patch for your version. You can deploy a VM with devstack and run this<br>
>> > version.<br>
>> ><br>
>> > Regards.<br>
>> ><br>
>> ><br>
>> > On Fri, Nov 25, 2022 at 1:58 PM Eugen Block <<a href="mailto:eblock@nde.ag" target="_blank">eblock@nde.ag</a>> wrote:<br>
>> ><br>
>> >> Hi *,<br>
>> >><br>
>> >> I'd like to ask you for advice on how to clean up my neutron db. At<br>
>> >> some point (which I don't know exactly, probably train) my neutron<br>
>> >> database got inconsistent, apparently one of the upgrades did not go<br>
>> >> as planned. The interesting thing is that the database still works, I<br>
>> >> just upgraded from ussuri to victoria where that issue popped up again<br>
>> >> during 'neutron-db-manage upgrade --expand', I'll add the information<br>
>> >> at the end of this email. Apparently, I have multiple heads, and one<br>
>> >> of them is from train, it seems as if I never ran --contract (or it<br>
>> >> failed and I didn't notice).<br>
>> >> Just some additional information what I did with this database: this<br>
>> >> cloud started out as a test environment with a single control node and<br>
>> >> then became a production environment. About two and a half years ago<br>
>> >> we decided to reinstall this cloud with version ussuri and import the<br>
>> >> databases. I had a virtual machine in which I upgraded the database<br>
>> >> dump from production to the latest versions at that time. That all<br>
>> >> worked quite well, I only didn't notice that something was missing.<br>
>> >> Now that I finished the U --> V upgrade I want to fix this<br>
>> >> inconsistency, I just have no idea how to do it. As I'm not sure how<br>
>> >> all the neutron-db-manage commands work exactly I'd like to ask for<br>
>> >> some guidance. For example, could the "stamp" command possibly help?<br>
>> >> Or how else can I get rid of the train head and/or how to get the<br>
>> >> train revision to "contract" so I can finish the upgrade and contract<br>
>> >> the victoria revision? I can paste the whole neutron-db history if<br>
>> >> necessary (neutron-db-manage history), please let me know what<br>
>> >> information would be required to get to the bottom of this.<br>
>> >> Any help is greatly appreciated!<br>
>> >><br>
>> >> Thanks!<br>
>> >> Eugen<br>
>> >><br>
>> >><br>
>> >> ---snip---<br>
>> >> controller01:~ # neutron-db-manage upgrade --expand<br>
>> >> [...]<br>
>> >> alembic.script.revision.MultipleHeads: Multiple heads are present for<br>
>> >> given argument 'expand@head'; 633d74ebbc4b, I38991de2b4<br>
>> >><br>
>> >> controller01:~ # neutron-db-manage current --verbose<br>
>> >> Running current for neutron ...<br>
>> >> INFO [alembic.runtime.migration] Context impl MySQLImpl.<br>
>> >> INFO [alembic.runtime.migration] Will assume non-transactional DDL.<br>
>> >> Current revision(s) for mysql+pymysql://neutron:XXXXX@controller.fqdn<br>
>> >> /neutron:<br>
>> >> Rev: bebe95aae4d4 (head)<br>
>> >> Parent: b5344a66e818<br>
>> >> Branch names: contract<br>
>> >> Path:<br>
>> >><br>
>> >><br>
>> /usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/ussuri/contract/bebe95aae4d4_.py<br>
>> >><br>
>> >> Rev: 633d74ebbc4b (head)<br>
>> >> Parent: 6c9eb0469914<br>
>> >> Branch names: expand<br>
>> >> Path:<br>
>> >><br>
>> >><br>
>> /usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/train/expand/633d74ebbc4b_.py<br>
>> >><br>
>> >> Rev: I38991de2b4 (head)<br>
>> >> Parent: 49d8622c5221<br>
>> >> Branch names: expand<br>
>> >> Path:<br>
>> >><br>
>> >><br>
>> /usr/lib/python3.6/site-packages/neutron/db/migration/alembic_migrations/versions/victoria/expand/I38991de2b4_source_and_destination_ip_prefix_neutron_metering_rule.py<br>
>> >><br>
>> >> OK<br>
>> >> ---snip---<br>
>> >><br>
>> >><br>
>> >><br>
>><br>
>><br>
>><br>
>><br>
<br>
<br>
<br>
</blockquote></div>