Nova Ussuri: DB online migration cannot migrate (some) populate_user_id

melanie witt melwittt at gmail.com
Mon Jul 24 23:57:29 UTC 2023


On 07/20/23 22:45, Michel Jouvin wrote:
> Hi,
> 
> We upgraded one of our clouds to Ussuri (we are late!) a couple of
> months ago. After the upgrade 'nova-manage db online_data_migration' was
> run. fill_virtual_interface_list and populate_user_id where non zero and
> after running the command fill_virtual_interface_list was 0 but
> populate_user_id stayed 24. Looking in nova-manage.log, I found the
> follwoing message (twice every time the command is run):
> 
> Some instance mappings were not migratable. This may be transient due to
> in-flight instance builds, or could be due to stale data that will be
> cleaned up after running "nova-manage db archive_deleted_rows --purge".
> 
> I ran the suggest command but the result is that
> fill_virtual_interface_list is non zero again (15) and there is no
> change to populate_user_i. If I run again online_data_migration, it
> clears again  fill_virtual_interface_list... and so on.
> 
> After 2 months the situation remains the same, no increase in
> populate_user_id but no possibility to fix them. We were wondering if it
> could have been caused by the fact that we had not yet upgraded all the
> compute nodes when we first ran the online_data_migration. We are
> planning to upgrade to Victoria and we'd like to fix the situation
> before as we understand that online data migration not done can be
> blocking for the next upgrade.
> 
> We don't have any clue on the reason this happened. It is a test cloud
> with a very low usage. On our production cloud we have a somewhat
> similar problem but populate_user_id=1 and running the purge doesn't
> increase fill_virtual_interface_list.

I'm not sure whether it's related but your post reminded me of this past 
bug:

https://bugs.launchpad.net/nova/+bug/1824435

so it might be worth checking whether you have a duplicate 
'security_groups' table record in your database. If so, there is a 
workaround described in comment #28 on that bug report.

-melwitt



More information about the openstack-discuss mailing list