Open Stack

👋

I'm planning on bringing this up in the nova rooms at the PTG in a few weeks,
but I'm also raising it here since this potentially affects other service
projects and I can't attend all of those room :)

Many projects use the concept of "soft delete" in their database models. A soft
deletable model typically has two additional columns, 'deleted' and
'deleted_at'. When deleting such a model, instead of actually deleting the
database row (i.e. 'DELETE FROM table WHERE condition'), we set 'deleted' to
'True' and populate the 'deleted_at' column. This is helpful for auditing
purposes (e.g. you can inspect all resources ever created, even after they've
been "deleted") but bad for database performance (your tables can grow without
bound). To work around the performance issues, most projects implement some kind
of archive or purge command that will allow operators to periodically clean up
these deleted resources. However, at least in nova, we've long since come to the
conclusion that soft deleting isn't as useful as initially suspected and the
need to run these commands is additional work for no benefit. We've moved toward
not using it for all new models.

With this said, it's going to be difficult to get away from soft-delete quickly.
Not only are there database migrations involved, but operators will need to
rework their tooling to adapt to a new, no-soft-delete world. As such, I'd like
to propose a half-way measure of making soft-delete configurable. To do this,
I'd like to add a new flag in oslo.db, '[database] enable_soft_delete'. When set
to 'False' anyone using the 'SoftDeleteMixin' from oslo.db would see these
models hard deleted rather than soft deleted when calling 'soft_delete'. This
would avoid the need for operators to run the various project-specific purge
tooling. The RFC patch for this is available for review [1]. I can also do this
on a project-specific basis and have proposed a similar patch for nova [2],
however, doing it in oslo.db means every project that uses 'SoftDeleteMixin' in
their models will get this for free. Projects that don't (glance, cinder) can
switch to using this mixin and also get it for free.

As noted above, I intend to discuss this in the nova room at the PTG, but I'd be
interested in people's thoughts ahead of time. Do you think this is a good idea?
Should we proceed with it? Perhaps there are there better ways to do this? Let
me know!

Cheers,
Stephen

[1] https://review.opendev.org/c/openstack/oslo.db/+/860407
[2] https://review.opendev.org/c/openstack/nova/+/860401