[openstack-dev] Resources owned by a project/tenant are not cleaned up after that project is deleted from keystone

Matthew Treinish mtreinish at kortar.org
Mon Feb 2 18:16:44 UTC 2015


On Mon, Feb 02, 2015 at 11:46:53AM -0600, Matt Riedemann wrote:
> This came up in the operators mailing list back in June [1] but given the
> subject probably didn't get much attention.
> 
> Basically there is a really old bug [2] from Grizzly that is still a problem
> and affects multiple projects.  A tenant can be deleted in Keystone even
> though other resources in other projects are under that project, and those
> resources aren't cleaned up.

I agree this probably can be a major pain point for users. We've had to work around it
in tempest by creating things like:

http://git.openstack.org/cgit/openstack/tempest/tree/tempest/cmd/cleanup_service.py
and
http://git.openstack.org/cgit/openstack/tempest/tree/tempest/cmd/cleanup.py

to ensure we aren't dangling resources after a run. But, this doesn't work in
all cases either. (like with tenant isolation enabled)

I also know there is a stackforge project that is attempting something similar
here:

http://git.openstack.org/cgit/stackforge/ospurge/

It would be much nicer if the burden for doing this was taken off users and this
was just handled cleanly under the covers.

> 
> Keystone implemented event notifications back in Havana [3] but the other
> projects aren't listening on them to know when a project has been deleted
> and act accordingly.
> 
> The bug has several people saying "we should talk about this at the summit"
> for several summits, but I can't find any discussion or summit sessions
> related back to the bug.
> 
> Given this is an operations and cross-project issue, I'd like to bring it up
> again for the Vancouver summit if there is still interest (which I'm
> assuming there is from operators).

I'd definitely support having a cross-project session on this.

> 
> There is a blueprint specifically for the tenant deletion case but it's
> targeted at only Horizon [4].
> 
> Is anyone still working on this? Is there sufficient interest in a
> cross-project session at the L summit?
> 
> Thinking out loud, even if nova doesn't listen to events from keystone, we
> could at least have a periodic task that looks for instances where the
> tenant no longer exists in keystone and then take some action (log a
> warning, shutdown/archive/, reap, etc).
> 
> There is also a spec for L to transfer instance ownership [5] which could
> maybe come into play, but I wouldn't depend on it.
> 
> [1] http://lists.openstack.org/pipermail/openstack-operators/2014-June/004559.html
> [2] https://bugs.launchpad.net/nova/+bug/967832
> [3] https://blueprints.launchpad.net/keystone/+spec/notifications
> [4] https://blueprints.launchpad.net/horizon/+spec/tenant-deletion
> [5] https://review.openstack.org/#/c/105367/
> 

-Matt Treinish
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150202/7685cc05/attachment.pgp>


More information about the OpenStack-dev mailing list