Open Stack

On 04/03/16 04:35, Johannes Grassler wrote:
> Hello,
>
> I am currently looking into https://bugs.launchpad.net/heat/+bug/1442121
> and
> not quite sure how to tackle it, since the problematic code is used for
> lots of
> things[0]: the root cause of the problem are API clients in resource
> plugins
> that do not anticipate a resource with an entry in Heat's database
> having been
> deleted in the implementing service's database[1]. Here's an example:
>
> https://github.com/openstack/heat/blob/e4dc942ce1a8c79b450345c7afae326c80d8a5d6/heat/engine/resources/openstack/neutron/floatingip.py#L179
>
>
> If that happens[1] an uncaught exception will be thrown that among other
> things
> breaks the very operations one would need for cleaning up the mess.
>
> As far as I can see it, the cleanest way would be to go through all
> resources
> with a fine comb and add exception handling to the API calls in the
> add_dependencies() method where it is missing (just return False for any
> resource that no longer exists). Or is there a better way?

Yes, you're right and this sucks. That's not the only problem we've had 
in this area recently - for example there was also:

https://bugs.launchpad.net/heat/+bug/1536515.

The fact that we have to have these hacked in implicit dependencies at 
all is terrible, but we really need to make sure they can't break basic 
operations like loading a stack from the DB so we can show or delete it. 
The ideal would be:

* We can guarantee to catch all (non-exit) exceptions, no matter what 
kind of crazy stuff people write in add_dependencies()
* The plugin developer doesn't have to do anything special to get this 
behaviour
* The code remains backwards compatible with any third-party resource 
plugins circulating out there
* We always add as many dependencies as possible (i.e. all 
non-exception-raising dependencies are added)
* Genuine dependency problems (e.g. non-existent target of 
get_resource/get_attr) are still surfaced, preferably on CREATE only

I'm pretty sure getting all of those is impossible. I'd be very 
interested in evaluating different tradeoffs we could make among them 
though.

In the meantime, we need to find and squash every instance of this 
problem wherever we can like you said.

cheers,
Zane.

>
> Cheers,
>
> Johannes
>
> Footnotes:
>
> [0] Whenever a stack's resources are being listed using
>      heat.engine.service.list_stack_resources(). resource-list and
> stack-delete,
>      all invoke list_stack_resources()). stack-abandon does so
> indirectly (it
>      appears to trigger stack-delete judging by the log, but it yields the
>      desired output, at least in Liberty). These are just the ones I
> tested,
>      there are probably more.
>
> [1] It can happen for a number of reasons, either due to resource
> dependency
>      problems upon stack-delete as it happened in the original bug
> report or due
>      to an operator accidently deleting resources that are managed by Heat.
>
>