Open Stack

Regarding gmann's #1. Existing OSPurge doesn't have any specific logic 
per-se
but would just return no resources based on the has_service() method 
which I would
assume is checking endpoints. [1]

+1 On having a pluggable approach and to Adrian's feedback on having a 
strict policy
on how they should be implemented.

Best regards

[1] https://review.openstack.org/#/c/600919/1/ospurge/resources/heat.py

On 01/30/2019 07:36 AM, Ghanshyam Mann wrote:
>   ---- On Wed, 23 Jan 2019 08:21:27 +0900 Adrian Turjak <adriant at catalyst.net.nz> wrote ----
>   > Thanks for the input! I'm willing to bet there are many people excited
>   > about this goal, or will be when they realise it exists!
>   >
>   > The 'dirty' state I think would be solved with a report API in each
>   > service (tell me everything a given project has resource wise). Such an
>   > API would be useful without needing to query each resource list, and
>   > potentially could be an easy thing to implement to help a purge library
>   > figure out what to delete. I know right now our method for checking if a
>   > project is 'dirty' is part of our quota checking scripts, and it has to
>   > query a lot of APIs per service to build an idea of what a project has.
>   >
>   > As for using existing code, OSPurge could well be a starting point, but
>   > the major part of this goal has to be that each OpenStack service (that
>   > creates resources owned by a project) takes ownership of their own
>   > deletion logic. This is why a top level library for cross project logic,
>   > with per service plugin libraries is possibly the best approach. Each
>   > library would follow the same template and abstraction layers (as
>   > inherited from the top level library), but how each service implements
>   > their own deletion is up to them. I would also push for them using the
>   > SDK only as their point of interaction with the APIs (lets set some hard
>   > requirements and standards!), because that is the python library we
>   > should be using going forward. In addition such an approach could mean
>   > that anyone can write a plugin for the top level library (e.g. internal
>   > company only services) which will automatically get picked up if installed.
>
> +100 for not making keystone as Actor. Leaving purge responsibility to service
> side is the best way without any doubt.
>
> Instead of accepting Purge APIs from each service, I am thinking
> we should consider another approach also which can be the plugin-able approach.
> Ewe can expose the plugin interface from purge library/tool. Each service implements
> the interface with purge functionality(script or command etc).
> On discovery of each service's purge plugin, purge library/tool will start the deletion
> in required order etc.
>
> This can give 2 simple benefits
> 1. No need to detect the service availability before requesting them to purge the resources.
> I am not sure OSpurge check the availability of services or not. But in plugin approach case,
> that will not be required. For example, if Congress is not installed in my env then,
> congress's purge plugin will not be discovered so no need to check Congress service availability.
>
> 2. purge all resources interface will not be exposed to anyone except the Purge library/tool.
> In case of API, we are exposing the interface to user(admin/system scopped etc) which can
> delete all the resources of that service which is little security issue may be. This can be argued
> with existing delete API but those are per resource not all. Other side we can say those can be
> taken care by RBAC but still IMO exposing anything to even permissiable user(especially human)
> which can destruct the env is not a good idea where only right usage of that interface is something
> else (Purge library/tool in this case).
>
> Plugin-able can also have its cons but Let's first discuss all those possibilities.
>
> -gmann
>
>   >
>   > We would need robust and extensive testing for this, because deletion is
>   > critical, and we need it to work, but also not cause damage in ways it
>   > shouldn't.
>   >
>   > And you're right, purge tools purging outside of the scope asked for is
>   > a worry. Our own internal logic actually works by having the triggering
>   > admin user add itself to the project (and ensure no admin role), then
>   > scope a token to just that project, and delete resources form the point
>   > of view of a project user. That way it's kind of like a user deleting
>   > their own resources, and in truth having a nicer way to even do that
>   > (non-admin clearing of project) would be amazing for a lot of people who
>   > don't want to close their account or disable their project, but just
>   > want to delete stray resources and not get charged.
>   >
>   > On 23/01/19 4:03 AM, Tobias Urdin wrote:
>   > > Thanks for the thorough feedback Adrian.
>   > >
>   > > My opinion is also that Keystone should not be the actor in executing
>   > > this functionality but somewhere else
>   > > whether that is Adjutant or any other form (application, library, CLI
>   > > etc).
>   > >
>   > > I would also like to bring up the point about knowing if a project is
>   > > "dirty" (it has provisioned resources).
>   > > This is something that I think all business logic would benefit from,
>   > > we've had issue with knowing when
>   > > resources should be deleted, our solution is pretty much look at
>   > > metrics the last X minutes, check if project
>   > > is disabled and compare to business logic that says it should be deleted.
>   > >
>   > > While the above works it kills some of logical points of disabling a
>   > > project since the only thing that knows if
>   > > the project should be deleted or is actually disabled is the business
>   > > logic application that says they clicked the
>   > > deleted button and not disabled.
>   > >
>   > > Most of the functionality you are mentioning is things that the
>   > > ospurge project has been working to implement and the
>   > > maintainer even did a full rewrite which improved the dependency
>   > > arrangement for resource removal.
>   > >
>   > > I think the biggest win for this community goal would be the
>   > > developers of the projects would be available for input regarding
>   > > the project specific code that does purging. There has been some
>   > > really nasty bugs in ospurge in the past that if executed with the admin
>   > > user you would wipe everything and not only that project, which is
>   > > probably a issue that makes people think twice about
>   > > using a purging toolkit at all.
>   > >
>   > > We should carefully consider what parts of ospurge could be reused,
>   > > concept, code or anything in between that could help derive
>   > > what direction we wan't to push this goal.
>   > >
>   > > I'm excited :)
>   > >
>   > > Best regards
>   > > Tobias
>   > >
>   >
>   >
>
>
>
>