---- On Wed, 23 Jan 2019 08:21:27 +0900 Adrian Turjak adriant@catalyst.net.nz wrote ----
Thanks for the input! I'm willing to bet there are many people excited about this goal, or will be when they realise it exists!
The 'dirty' state I think would be solved with a report API in each service (tell me everything a given project has resource wise). Such an API would be useful without needing to query each resource list, and potentially could be an easy thing to implement to help a purge library figure out what to delete. I know right now our method for checking if a project is 'dirty' is part of our quota checking scripts, and it has to query a lot of APIs per service to build an idea of what a project has.
As for using existing code, OSPurge could well be a starting point, but the major part of this goal has to be that each OpenStack service (that creates resources owned by a project) takes ownership of their own deletion logic. This is why a top level library for cross project logic, with per service plugin libraries is possibly the best approach. Each library would follow the same template and abstraction layers (as inherited from the top level library), but how each service implements their own deletion is up to them. I would also push for them using the SDK only as their point of interaction with the APIs (lets set some hard requirements and standards!), because that is the python library we should be using going forward. In addition such an approach could mean that anyone can write a plugin for the top level library (e.g. internal company only services) which will automatically get picked up if installed.
+100 for not making keystone as Actor. Leaving purge responsibility to service side is the best way without any doubt.
Instead of accepting Purge APIs from each service, I am thinking we should consider another approach also which can be the plugin-able approach. Ewe can expose the plugin interface from purge library/tool. Each service implements the interface with purge functionality(script or command etc). On discovery of each service's purge plugin, purge library/tool will start the deletion in required order etc.
This can give 2 simple benefits 1. No need to detect the service availability before requesting them to purge the resources. I am not sure OSpurge check the availability of services or not. But in plugin approach case, that will not be required. For example, if Congress is not installed in my env then, congress's purge plugin will not be discovered so no need to check Congress service availability.
2. purge all resources interface will not be exposed to anyone except the Purge library/tool. In case of API, we are exposing the interface to user(admin/system scopped etc) which can delete all the resources of that service which is little security issue may be. This can be argued with existing delete API but those are per resource not all. Other side we can say those can be taken care by RBAC but still IMO exposing anything to even permissiable user(especially human) which can destruct the env is not a good idea where only right usage of that interface is something else (Purge library/tool in this case).
Plugin-able can also have its cons but Let's first discuss all those possibilities.
-gmann
We would need robust and extensive testing for this, because deletion is critical, and we need it to work, but also not cause damage in ways it shouldn't.
And you're right, purge tools purging outside of the scope asked for is a worry. Our own internal logic actually works by having the triggering admin user add itself to the project (and ensure no admin role), then scope a token to just that project, and delete resources form the point of view of a project user. That way it's kind of like a user deleting their own resources, and in truth having a nicer way to even do that (non-admin clearing of project) would be amazing for a lot of people who don't want to close their account or disable their project, but just want to delete stray resources and not get charged.
On 23/01/19 4:03 AM, Tobias Urdin wrote:
Thanks for the thorough feedback Adrian.
My opinion is also that Keystone should not be the actor in executing this functionality but somewhere else whether that is Adjutant or any other form (application, library, CLI etc).
I would also like to bring up the point about knowing if a project is "dirty" (it has provisioned resources). This is something that I think all business logic would benefit from, we've had issue with knowing when resources should be deleted, our solution is pretty much look at metrics the last X minutes, check if project is disabled and compare to business logic that says it should be deleted.
While the above works it kills some of logical points of disabling a project since the only thing that knows if the project should be deleted or is actually disabled is the business logic application that says they clicked the deleted button and not disabled.
Most of the functionality you are mentioning is things that the ospurge project has been working to implement and the maintainer even did a full rewrite which improved the dependency arrangement for resource removal.
I think the biggest win for this community goal would be the developers of the projects would be available for input regarding the project specific code that does purging. There has been some really nasty bugs in ospurge in the past that if executed with the admin user you would wipe everything and not only that project, which is probably a issue that makes people think twice about using a purging toolkit at all.
We should carefully consider what parts of ospurge could be reused, concept, code or anything in between that could help derive what direction we wan't to push this goal.
I'm excited :)
Best regards Tobias