Open Stack

On 11/18/2015 7:32 AM, Andrew Laski wrote:
> On 11/17/15 at 07:43pm, Jay Pipes wrote:
>> On 11/17/2015 05:43 PM, Matt Riedemann wrote:
>>> I found some time to work on a reverse sort of nova's tables for the db
>>> archive command, that looks like [1].  It works fine in the unit tests,
>>> but fails because the deleted instances are referenced by
>>> instance_actions that aren't deleted.  I see any DB APIs for deleting
>>> instance actions.
>>>
>>> Were we just planning on instance_actions living forever in the
>>> database?
>>
>> Not as far as I understand.
>
> They were never intended to live forever.  However there is a use case
> for holding on to the deleted action so that someone could query when or
> by whom their instance was deleted.  But the current API does not
> provide a good way to query for that so this may be something better
> left to the growing list of things that Tasks could address.
>
>>
>>> Should we soft delete instance_actions when we delete the referenced
>>> instance?
>>
>> No.
>>
>>> Or should we (hard) delete instance_actions when we archive (move to
>>> shadow tables) soft deleted instances?
>>
>> Yes.
>>
>> Best,
>> -jay
>>
>>> This is going to be a blocker to getting nova-manage db
>>> archive_deleted_rows working.
>>>
>>> [1] https://review.openstack.org/#/c/246635/
>>>
>>
>> __________________________________________________________________________
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

OK, so we talked a bit about this in IRC today.  I'll try to summarize 
here.  There are really two options it sounds like, either soft delete 
instance_actions when we soft delete instances, or just hard delete the 
instance_actions when archiving/purging soft deleted instances (to avoid 
the referential constraint failure when hard deleting the instances).

We have to note that the os-instance-actions API only works on instances 
that are not deleted [1]. So once you delete an instance (soft deleted 
in this case, i.e. instances.deleted != 0 in the DB), then the 
os-instance-actions API is useless. We could change that with a 
microversion to read deleted instances...but I'm not sure if anyone 
wants that (operators might?).

Even if we start soft deleting instance_actions, the archive/purge code 
will still have to account for existing resources in the database and 
handle them, i.e. when archiving an instance, find it's related 
instance_actions and if they are not soft deleted, soft delete and 
archive them first before archiving the instance. We could probably also 
provide a DB migration to do this, but that would be a lot of queries 
and updates potentially during an offline upgrade. We could also provide 
a nova-manage command to do it explicitly.

So hard-deleting instance_actions when archiving/purging instances is 
easier from a development perspective, it's much more straight forward 
(fewer bugs, yay!).  The downside to hard delete is they are gone, they 
wouldn't be in shadow tables or anything in the nova database.

The proposed remedy for wanting to store instance_actions after 
hard-delete is to basically have a monitoring service setup that is 
storing the instance delete notificiations, i.e. StackTach, gnocchi, 
probably others (monasca?). This is kind of a cop out for the nova dev 
team since it puts the burden on the operators to have this extra 
service running, but I suspect most already have something like this 
running anyway.

I'll cross post this to the operators ML and have it on this weeks' nova 
meeting agenda to see if we can find some agreement on a path forward here.

[1] 
https://github.com/openstack/nova/blob/12.0.0/nova/api/openstack/compute/instance_actions.py#L68

-- 

Thanks,

Matt Riedemann