Open Stack

I created a nova bug [1] to track a request that came up in the upgrades 
SIG room at the PTG today [2] and would like to see if there is any 
feedback from other operators/developers that weren't part of the 
discussion.

The basic problem is that failing to archive/purge deleted records* from 
the database can make upgrades much slower during schema migrations. 
Anecdotes from the room mentioned that it can be literally impossible to 
complete upgrades for keystone and heat in certain scenarios if you 
don't purge the database first.

The request was that a configurable limit gets added to each service 
which is checked as part of the service's pre-upgrade check routine [3] 
and warn if the number of records to purge is over that limit.

For example, the nova-status upgrade check could warn if there are over 
100000 deleted records total across all cells databases. Maybe cinder 
would have something similar for deleted volumes. Keystone could have 
something for revoked tokens.

Another idea in the room was flagging on records over a certain age 
limit. For example, if there are deleted instances in nova that were 
deleted >1 year ago.

How do people feel about this? It seems pretty straight-forward to me. 
If people are generally in favor of this, then the question is what 
would be sane defaults - or should we not assume a default and force 
operators to opt into this?

* nova delete doesn't actually delete the record from the instances 
table, it flips a value to hide it - you have to archive/purge those 
records to get them out of the main table.

[1] https://bugs.launchpad.net/nova/+bug/1791824
[2] https://etherpad.openstack.org/p/upgrade-sig-ptg-stein
[3] https://governance.openstack.org/tc/goals/stein/upgrade-checkers.html

-- 

Thanks,

Matt