OH! As someone who tried to champion project termination at the API layer as a community goal during the Berlin summit, I love that someone ran with our idea of implementing it in the SDK as a starting point!

I still think we should champion a purge API for each service, but the SDK is definitely the place to handle dependencies, and I love that you are doing it in threads!

I'd be happy to review/test any code related to this both in the OSC and the SDK. I sadly haven't had time to help implement anything like this, but I will somehow find the time to review/test! And I look forward to one day throwing away our internal termination logic in favor of what's in the SDK!

On 20/01/21 2:16 am, Artem Goncharov wrote:
Hi Christian.

Actually the patch stuck due to lack of reviewers. Idea here was not to replace “openstack project purge”, but to add a totally new implementation (hopefully later dropping project purge as such). From my POV at the moment there is nothing else I was thinking to mandatorily implement on OSC side (sure, for future I would like to give possibility to limit services to cleanup, to add cleanup of key-airs, etc). SDK part is completely independent of that. Here we definitely need to add dropping of private images. Also on DNS side we can do cleanup. Other services are tricky (while swift we can still implement relatively easy).

All in all - we can merge the PR in it’s current form (assuming we get some positive reviews).

BG,
Artem

On 19. Jan 2021, at 13:33, Christian Rohmann <christian.rohmann@inovex.de> wrote:

Hey Artem,

thank you very much for your quick reply and pointer to the patchset you work on!



On 18/01/2021 20:14, Artem Goncharov wrote:
Ha, thats exactly the case, the whole logic sits in sdk and is spread across the supported services:
- https://opendev.org/openstack/openstacksdk/src/branch/master/openstack/compute/v2/_proxy.py#L1798 - for compute. KeyPairs not dropped, since they belong to user, and not to the “project”;
- https://opendev.org/openstack/openstacksdk/src/branch/master/openstack/network/v2/_proxy.py#L4130 - the most complex one in order to give possibility to clean “old” resource without destroying everything else

Adding image is few lines of code (never had enough time to add it), identity is a bit tricky, since also here mostly resources does not belong to Project. DNS would be also easy to do. OSC here is only providing I/F, while the logic sits in SDK and can be very easy extended for other services.

On 18. Jan 2021, at 19:52, Thomas Goirand <zigo@debian.org> wrote:

On 1/18/21 6:56 PM, Artem Goncharov wrote:
What do you mean it doesn’t implement anything at all? It does clean up compute, network, block_storage, orchestrate resources. Moreover it gives you possibility to clean “old” resources (created before or last updated before).

Oh really? With that few lines of code? I'll re-read the patch then,
sorry for my bad assumptions.

Can you point at the part that's actually deleting the resources?

If I understood correctly, the cleanup relies on the SDK functionality / requirement for each resource type to provide a corresponding function( https://github.com/openstack/openstacksdk/blob/master/openstack/cloud/openstackcloud.py#L762) ?

Reading through the (SDK) code this even covers depending resources, nice!


I certainly will leave some feedback and comments in your change  (https://review.opendev.org/c/openstack/python-openstackclient/+/734485).
But what are your immediate plans moving forward on with this then, Artem?

There is a little todo list in the description on your change .. is there anything you yourself know that is still missing before taking this to a full review and finally merging it?

Only code that is shipped and then actively used will improve further and people will notice other required functionality or switches for later iterations. With the current state of having a somewhat working but unmaintained ospurge and a non feature complete "project purge"  (supports only Block Storage v1, v2; Compute v2; Image v1, v2) this will only cause people to start hacking away on the ospurge codebase or worse building their own tools and scripts to implement project cleanup for their environments over and over again.



Regards,


Christian