[openstack-dev] [heat] heat delete woes in Juno

Steve Baker sbaker at redhat.com
Sun Mar 29 20:35:16 UTC 2015


On 28/03/15 09:05, Matt Fischer wrote:
> Pavlo,
>
> Here is a link to one of the stacks. It is fairly simple just some 
> routers/nets/subnets. The description is a bit odd perhaps, but legal. 
> I've changed the template to not point at IPs at internal DNS.
>
> http://paste.ubuntu.com/10690759/
>
I've not been able to reproduce with this template running my local Juno 
2014.2.2 or a recent devstack, but I'm sure you can.
> I created and deleted this in a loop about 5 times and it finally 
> failed to delete on the last run. Now that it is stuck in 
> DELETE_FAILED no amount of deleting will help. I'm concerned that a 
> template this simple can get stuck like this.
>
You should be able to delete the underlying resources using the neutron 
command, then delete the stack.
> I will have stack_abandon enabled next week as it just landed in 
> Puppet: https://review.openstack.org/#/c/168157/ and will plan on 
> trying that then.
>
We've needed a series of workarounds for neutron resources as there are 
often implicit dependencies created which are not declared in REST 
create calls.  Its likely you've discovered some more implicit 
dependencies which we need to handle. Could you please raise a bug [1] 
with the following?

- the above pasted template
- the heat event-list of the DELETE_FAILED stack
- the heat stack-show of the DELETE_FAILED stack

[1] https://bugs.launchpad.net/heat/+filebug



>
> On Thu, Mar 26, 2015 at 12:40 PM, Pavlo Shchelokovskyy 
> <pshchelokovskyy at mirantis.com <mailto:pshchelokovskyy at mirantis.com>> 
> wrote:
>
>     Hi Matt,
>
>     if it would be feasible/appropriate, could you provide us with
>     templates for stacks that show this behavior (try to get them with
>     "heat template-show <stack-name-or-id>")? This would help us to
>     test and understand the problem better.
>
>     And yes, just the day before I was contacted by one of my
>     colleagues who seems to experience similar problems with
>     Juno-based OpenStack deployment (though I did not had a chance to
>     look through the issue yet).
>
>     Best regards,
>
>     Pavlo Shchelokovskyy
>     Software Engineer
>     Mirantis Inc
>     www.mirantis.com <http://www.mirantis.com>
>
>     On Thu, Mar 26, 2015 at 8:17 PM, Matt Fischer
>     <matt at mattfischer.com <mailto:matt at mattfischer.com>> wrote:
>
>         Nobody on the operators list had any ideas on this, so
>         re-posting here.
>
>         We've been having some issues with heatdelete-stack in Juno.
>         The issues generally fall into three categories:
>
>         1) it takes multiple calls to heat to delete a stack.
>         Presumably due to heat being unable to figure out the ordering
>         on deletion and resources being in use.
>
>         2) undeleteable stacks. Stacks that refuse to delete, get
>         stuck in DELETE_FAILED state. In this case, they show up in
>         stack-list and stack-show, yet resource-list and
>         stack-delete deny their existence. This means I can't be sure
>         whether they have any real resources very easily.
>
>         3) As a corollary to item 1, stacks for which heat can never
>         unwind the dependencies and stay in DELETE_IN_PROGRESS forever.
>
>         Does anyone have any work-arounds for these or recommendations
>         on cleanup? My main worry is removing a stack from the
>         database that is still consuming the customer's resources. I
>         also don't just want to remove stacks from the database and
>         leave orphaned records in the DB.
>
>         __________________________________________________________________________
>         OpenStack Development Mailing List (not for usage questions)
>         Unsubscribe:
>         OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>         <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>         http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>     __________________________________________________________________________
>     OpenStack Development Mailing List (not for usage questions)
>     Unsubscribe:
>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150330/69bb7994/attachment.html>


More information about the OpenStack-dev mailing list