Open Stack

I *think* deleted flavours used to be needed as there could still be instances running against them and the flavour definition was used by the quota calculations.  Not sure if this is still the case, or if the data now comes straight from the instances table.    Some aspects of a flavour (e.g. rxtx_factor) could be useful to a scheduler, and that data currently isn't saved into the instance.

I guess the usage audit type functionality (i.e. tell me about all instances that have run sometime in this audit period) may be another case where deleted instances are required at the moment.



-----Original Message-----
From: openstack-bounces+philip.day=hp.com at lists.launchpad.net [mailto:openstack-bounces+philip.day=hp.com at lists.launchpad.net] On Behalf Of Pitucha, Stanislaw Izaak
Sent: 03 October 2012 13:09
To: openstack at lists.launchpad.net
Subject: Re: [Openstack] Discussion / proposal: deleted column marker

Hi Johannes,
I know the names collide here, but since this technique is known as soft-deletes... We need more namespaces :)

Thanks for the idea of grepping for read_deleted. Fortunately I think the situation isn't as bad as it would seem. Let me group the places which change read_deleted in the code (many results from grep are comments).
Reading only deleted entries, or all:

- xenserver (instance) - cleanup tool - I don't do xen, so I'm not sure how useful is it. Anyone?
- tests - can be ignored - if there is no functionality, tests can be killed
- sqlalchemy api (instance) - fixed ip can reference a deleted instance (tricky situation; from the commit message: "It adds a test  to verify that the code works with a duplicate deleted floating_ip" - this seems very
wrong...)
- sqlalchemy api (iscsi) - getting deleted iscsi targets which are still referenced by volume
- sqlalchemy api (various) - instance migration, s3image, bandwidth, storage manager, flavors (only available from nova-manage)
- compute manager (instance) - reaping deleted instances - I can't see why the same logic wouldn't apply if the rows are actually missing (needs analysis, maybe there's a reason)
- compute instance_types (flavour) - apparently flavours are usually read even if they're deleted
- network manager (instance) - making sure that ips/networks can be removed even if the instance is already deleted

So here's what I can see: pretty much all the usage is about deleting instances or making sure parts connected to instances go away if the instance is deleted earlier. It doesn't seem right, but could be progressively fixed. It looks like another "state" of the instance, which could be integrated into the other state fields.

Nothing else uses the deleted column explicitly (unless I missed something - possible). Ips, networks, keys, anything that actually goes away permanently (and doesn't involve a big chain of cleanup events) is never read back once it's marked as deleted.
So maybe a better approach is not to remove the deleted column completely, but to start stripping it from places where it's not needed (fixed, floating ips, networks, ssh keys being good candidates). This could be done by creating a new layer over NovaBase and removing the "deleted" marker from NovaBase itself. Or maybe even by creating a SoftDeleteMixin and applying it to all current models, then removing it where unnecessary? That would keep the current behaviour where it's currently needed, but at the same time it would provide a known migration path to get rid of it.
We could start stripping the new layer then table by table and adding unique constraints where they make sense, before trying to tackle the really tricky parts (for instances table, maybe the marker actually makes sense? maybe not? - it's definitely not going to be an easy decision/fix)

Regards,
Stanisław Pitucha
Cloud Services
Hewlett Packard


-----Original Message-----
From: openstack-bounces+stanislaw.pitucha=hp.com at lists.launchpad.net
[mailto:openstack-bounces+stanislaw.pitucha=hp.com at lists.launchpad.net] On
Behalf Of Johannes Erdfelt
Sent: Tuesday, October 02, 2012 6:12 PM
To: openstack at lists.launchpad.net
Subject: Re: [Openstack] Discussion / proposal: deleted column marker

On Tue, Oct 02, 2012, Pitucha, Stanislaw Izaak <stanislaw.pitucha at hp.com>
wrote:
> Does anyone know why soft-delete is still in place?
> Are there any reasons it can't / shouldn't be removed at this time?
> If it's possible to remove it, would you miss it?

I'm certainly not a fan of the database soft-delete for many of the same
reasons you've described, but there are some places where removing it would
require code changes.

Off the top of my head would be pretty much anywhere a context sets
read_deleted to 'yes' or 'only', which is a surprising number of places now
that I've done a grep.

I suspect at least a handful of those cases don't need the functionality and
others probably use it as a crutch around other problems.

JE


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack at lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp