Open Stack

Hi Johannes,
I know the names collide here, but since this technique is known as
soft-deletes... We need more namespaces :)

Thanks for the idea of grepping for read_deleted. Fortunately I think the
situation isn't as bad as it would seem. Let me group the places which
change read_deleted in the code (many results from grep are comments).
Reading only deleted entries, or all:

- xenserver (instance) - cleanup tool - I don't do xen, so I'm not sure how
useful is it. Anyone?
- tests - can be ignored - if there is no functionality, tests can be killed
- sqlalchemy api (instance) - fixed ip can reference a deleted instance
(tricky situation; from the commit message: "It adds a test  to verify that
the code works with a duplicate deleted floating_ip" - this seems very
wrong...)
- sqlalchemy api (iscsi) - getting deleted iscsi targets which are still
referenced by volume
- sqlalchemy api (various) - instance migration, s3image, bandwidth, storage
manager, flavors (only available from nova-manage)
- compute manager (instance) - reaping deleted instances - I can't see why
the same logic wouldn't apply if the rows are actually missing (needs
analysis, maybe there's a reason)
- compute instance_types (flavour) - apparently flavours are usually read
even if they're deleted
- network manager (instance) - making sure that ips/networks can be removed
even if the instance is already deleted

So here's what I can see: pretty much all the usage is about deleting
instances or making sure parts connected to instances go away if the
instance is deleted earlier. It doesn't seem right, but could be
progressively fixed. It looks like another "state" of the instance, which
could be integrated into the other state fields.

Nothing else uses the deleted column explicitly (unless I missed something -
possible). Ips, networks, keys, anything that actually goes away permanently
(and doesn't involve a big chain of cleanup events) is never read back once
it's marked as deleted.
So maybe a better approach is not to remove the deleted column completely,
but to start stripping it from places where it's not needed (fixed, floating
ips, networks, ssh keys being good candidates). This could be done by
creating a new layer over NovaBase and removing the "deleted" marker from
NovaBase itself. Or maybe even by creating a SoftDeleteMixin and applying it
to all current models, then removing it where unnecessary? That would keep
the current behaviour where it's currently needed, but at the same time it
would provide a known migration path to get rid of it.
We could start stripping the new layer then table by table and adding unique
constraints where they make sense, before trying to tackle the really tricky
parts (for instances table, maybe the marker actually makes sense? maybe
not? - it's definitely not going to be an easy decision/fix)

Regards,
Stanisław Pitucha
Cloud Services 
Hewlett Packard


-----Original Message-----
From: openstack-bounces+stanislaw.pitucha=hp.com at lists.launchpad.net
[mailto:openstack-bounces+stanislaw.pitucha=hp.com at lists.launchpad.net] On
Behalf Of Johannes Erdfelt
Sent: Tuesday, October 02, 2012 6:12 PM
To: openstack at lists.launchpad.net
Subject: Re: [Openstack] Discussion / proposal: deleted column marker

On Tue, Oct 02, 2012, Pitucha, Stanislaw Izaak <stanislaw.pitucha at hp.com>
wrote:
> Does anyone know why soft-delete is still in place?
> Are there any reasons it can't / shouldn't be removed at this time?
> If it's possible to remove it, would you miss it?

I'm certainly not a fan of the database soft-delete for many of the same
reasons you've described, but there are some places where removing it would
require code changes.

Off the top of my head would be pretty much anywhere a context sets
read_deleted to 'yes' or 'only', which is a surprising number of places now
that I've done a grep.

I suspect at least a handful of those cases don't need the functionality and
others probably use it as a crutch around other problems.

JE


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack at lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6216 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20121003/2f6fbe4b/attachment.bin>