Open Stack

I would like to strongly second this: not having soft delete, or an
equivalent for audit purposes (I am not attached to the actual
implementation), would be a great loss. We actually have a long standing
task to add soft delete to Blazar, which I am hoping will be merged in
Antelope.

As an operator, I also get annoyed by the lack of soft delete in Neutron,
for example to answer the question: who was using this specific floating IP
at this specific time?

On Thu, 6 Oct 2022 at 07:28, Dmitriy Rabotyagov <noonedeadpunk at gmail.com>
wrote:

> Not having soft delete in the database is really quite bad for operators
> and it's not about tooling, but it's about audit purposes.
>
> If take nova as example, this also means that once server is deleted,
> event log will be also wiped with no way to see who and when has performed
> delete action. And that is really used feature, as we got requests like
> "why my VM has disappeared" at very least one a week.
>
> For other services having deleted_at at least points to the datetime where
> to search in the logs.
>
> At the same time I don't see any issue in having soft delete. It's just a
> matter of one systemd-timer, and too concerned about performance can set it
> to 1 day, thus almost no impact on db performance.
>
> So from operator perspective I can say this is very valuable feature and I
> personally do struggle regularly with neutron services where it's absent.
> And I would hate this to disappear at all, as it would be really a
> nightmare.
>
> ср, 5 окт. 2022 г., 14:48 Stephen Finucane <stephenfin at redhat.com>:
>
>> 👋
>>
>> I'm planning on bringing this up in the nova rooms at the PTG in a few
>> weeks,
>> but I'm also raising it here since this potentially affects other service
>> projects and I can't attend all of those room :)
>>
>> Many projects use the concept of "soft delete" in their database models.
>> A soft
>> deletable model typically has two additional columns, 'deleted' and
>> 'deleted_at'. When deleting such a model, instead of actually deleting the
>> database row (i.e. 'DELETE FROM table WHERE condition'), we set 'deleted'
>> to
>> 'True' and populate the 'deleted_at' column. This is helpful for auditing
>> purposes (e.g. you can inspect all resources ever created, even after
>> they've
>> been "deleted") but bad for database performance (your tables can grow
>> without
>> bound). To work around the performance issues, most projects implement
>> some kind
>> of archive or purge command that will allow operators to periodically
>> clean up
>> these deleted resources. However, at least in nova, we've long since come
>> to the
>> conclusion that soft deleting isn't as useful as initially suspected and
>> the
>> need to run these commands is additional work for no benefit. We've moved
>> toward
>> not using it for all new models.
>>
>> With this said, it's going to be difficult to get away from soft-delete
>> quickly.
>> Not only are there database migrations involved, but operators will need
>> to
>> rework their tooling to adapt to a new, no-soft-delete world. As such,
>> I'd like
>> to propose a half-way measure of making soft-delete configurable. To do
>> this,
>> I'd like to add a new flag in oslo.db, '[database] enable_soft_delete'.
>> When set
>> to 'False' anyone using the 'SoftDeleteMixin' from oslo.db would see these
>> models hard deleted rather than soft deleted when calling 'soft_delete'.
>> This
>> would avoid the need for operators to run the various project-specific
>> purge
>> tooling. The RFC patch for this is available for review [1]. I can also
>> do this
>> on a project-specific basis and have proposed a similar patch for nova
>> [2],
>> however, doing it in oslo.db means every project that uses
>> 'SoftDeleteMixin' in
>> their models will get this for free. Projects that don't (glance, cinder)
>> can
>> switch to using this mixin and also get it for free.
>>
>> As noted above, I intend to discuss this in the nova room at the PTG, but
>> I'd be
>> interested in people's thoughts ahead of time. Do you think this is a
>> good idea?
>> Should we proceed with it? Perhaps there are there better ways to do
>> this? Let
>> me know!
>>
>> Cheers,
>> Stephen
>>
>> [1] https://review.opendev.org/c/openstack/oslo.db/+/860407
>> [2] https://review.opendev.org/c/openstack/nova/+/860401
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20221006/b0d3591f/attachment.htm>