[nova][placement] Incomplete Consumers return negative value after upgrade
melanie witt
melwittt at gmail.com
Thu May 5 16:58:10 UTC 2022
On Thu May 05 2022 02:31:16 GMT-0700 (Pacific Daylight Time), Jan
Wasilewski <finarffin at gmail.com> wrote:
> Hi,
>
> after an upgrade from Stein to Train, I hit an issue with negative value
> during upgrade check for placement:
>
> # placement-status upgrade check
> +------------------------------------------------------------------+
> | Upgrade Check Results |
> +------------------------------------------------------------------+
> | Check: Missing Root Provider IDs |
> | Result: Success |
> | Details: None |
> +------------------------------------------------------------------+
> | Check: Incomplete Consumers |
> | Result: Warning |
> | Details: There are -20136 incomplete consumers table records for |
> | existing allocations. Run the "placement-manage db |
> | online_data_migrations" command. |
> +------------------------------------------------------------------+
>
>
> Seems that negative value is a result that I get such values from tables
> consumer and allocations:
>
> mysql> select count(id), consumer_id from allocations group by consumer_id;
> ...
> 1035 rows in set (0.00 sec)
>
> mysql> select count(*) from consumers;
> +----------+
> | count(*) |
> +----------+
> | 21171 |
> +----------+
> 1 row in set (0.04 sec)
>
> Unfortunately such warning cannot be solved by execution of suggested
> command( |placement-manage db online_data_migrations|) as it seems it
> adds records to consumers table - not to allocations, which looks like
> to be a problem here. I was following recommendations from this
> discussion:
> http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018536.html <http://lists.openstack.org/pipermail/openstack-discuss/2020-November/018536.html> but unfortunately it doesn't solve the issue(even not changing a negative value). I'm just wondering if I skipped something important and you can suggest some (obvious?) solution.
I think you need the following patch which changes the logic of the
upgrade check to ignore orphaned consumers which are not relevant to the
upgrade:
https://review.opendev.org/c/openstack/placement/+/840704
The above originally landed in Xena but was overlooked for backport to
older versions. I have proposed backports just now. If you can apply
this patch for 'placement-status' and the check passes afterward, you
are good.
You may still have orphaned consumers table records in the placement
database but they don't hurt anything. If you want/need to clean them
up, it has to be done manually, something like (disclaimer I did not
test this):
delete from placement.consumers where placement.consumers.uuid not in
(select nova_api.instance_mappings.instance_uuid from
nova_api.instance_mappings where
nova_api.instance_mappings.queued_for_delete = true);
HTH,
-melanie
More information about the openstack-discuss
mailing list