<div dir="ltr">Steve,<div><br></div><div>I have two environments now, and although they have some other differences, one of them is Juno.2 and I've also been unable to repro it there. That env also lacks the same scale, load, and HA config so I was trying to figure out whether it was Juno.2 or not, so thanks for the extra data point.</div><div><br></div><div>I am rolling out stack-abandon soon and will try to also get Juno.2 heat out soon.</div><div><br></div><div>If I hit more examples, I'll file bugs.</div><div><br></div><div>Thanks for the assistance all.</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Mar 29, 2015 at 2:35 PM, Steve Baker <span dir="ltr"><<a href="mailto:sbaker@redhat.com" target="_blank">sbaker@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><span class="">
<div>On 28/03/15 09:05, Matt Fischer wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Pavlo,
<div><br>
</div>
<div>Here is a link to one of the stacks. It is fairly simple
just some routers/nets/subnets. The description is a bit odd
perhaps, but legal. I've changed the template to not point at
IPs at internal DNS.</div>
<div><br>
</div>
<div><a href="http://paste.ubuntu.com/10690759/" target="_blank">http://paste.ubuntu.com/10690759/</a><br>
</div>
<div><br>
</div>
</div>
</blockquote></span>
I've not been able to reproduce with this template running my local
Juno 2014.2.2 or a recent devstack, but I'm sure you can.<span class=""><br>
<blockquote type="cite">
<div dir="ltr">
<div>I created and deleted this in a loop about 5 times and it
finally failed to delete on the last run. Now that it is stuck
in DELETE_FAILED no amount of deleting will help. I'm
concerned that a template this simple can get stuck like this.</div>
<div><br>
</div>
</div>
</blockquote></span>
You should be able to delete the underlying resources using the
neutron command, then delete the stack.<span class=""><br>
<blockquote type="cite">
<div dir="ltr">
<div>I will have stack_abandon enabled next week as it just
landed in Puppet: <a href="https://review.openstack.org/#/c/168157/" target="_blank">https://review.openstack.org/#/c/168157/</a>
and will plan on trying that then.</div>
<div><br>
</div>
</div>
</blockquote></span>
We've needed a series of workarounds for neutron resources as there
are often implicit dependencies created which are not declared in
REST create calls. Its likely you've discovered some more implicit
dependencies which we need to handle. Could you please raise a bug
[1] with the following?<br>
<br>
- the above pasted template<br>
- the heat event-list of the DELETE_FAILED stack<br>
- the heat stack-show of the DELETE_FAILED stack<br>
<br>
[1] <a href="https://bugs.launchpad.net/heat/+filebug" target="_blank">https://bugs.launchpad.net/heat/+filebug</a><div><div class="h5"><br>
<br>
<br>
<br>
<blockquote type="cite">
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Mar 26, 2015 at 12:40 PM, Pavlo
Shchelokovskyy <span dir="ltr"><<a href="mailto:pshchelokovskyy@mirantis.com" target="_blank">pshchelokovskyy@mirantis.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hi Matt,
<div><br>
</div>
<div>if it would be feasible/appropriate, could you
provide us with templates for stacks that show this
behavior (try to get them with "heat template-show
<stack-name-or-id>")? This would help us to test
and understand the problem better.</div>
<div><br>
</div>
<div>And yes, just the day before I was contacted by one
of my colleagues who seems to experience similar
problems with Juno-based OpenStack deployment (though I
did not had a chance to look through the issue yet).</div>
<div><br>
</div>
<div>Best regards,</div>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div>
<div dir="ltr">Pavlo Shchelokovskyy
<div>Software Engineer</div>
<div>Mirantis Inc</div>
<div><a href="http://www.mirantis.com" target="_blank">www.mirantis.com</a></div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div>
<div>On Thu, Mar 26, 2015 at 8:17 PM, Matt
Fischer <span dir="ltr"><<a href="mailto:matt@mattfischer.com" target="_blank">matt@mattfischer.com</a>></span>
wrote:<br>
</div>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div>
<div dir="ltr"><span style="font-size:12.8000001907349px">Nobody on
the operators list had any ideas on this, so
re-posting here.</span>
<div style="font-size:12.8000001907349px"><br>
</div>
<div style="font-size:12.8000001907349px"><span style="font-size:12.8000001907349px">We've
been having some issues with </span><span style="font-size:12.8000001907349px">heat</span><span style="font-size:12.8000001907349px"> </span><span style="font-size:12.8000001907349px">delete</span><span style="font-size:12.8000001907349px">-stack
in Juno. The issues generally fall into
three categories:</span>
<div style="font-size:12.8000001907349px"><br>
</div>
<div style="font-size:12.8000001907349px">1)
it takes multiple calls to heat to delete a
stack. Presumably due to heat being unable
to figure out the ordering on deletion and
resources being in use.</div>
<div style="font-size:12.8000001907349px"><br>
</div>
<div style="font-size:12.8000001907349px">2)
undeleteable stacks. Stacks that refuse
to delete, get stuck in DELETE_FAILED state.
In this case, they show up in stack-list and
stack-show, yet resource-list and
stack-delete deny their existence. This
means I can't be sure whether they have any
real resources very easily.</div>
<div style="font-size:12.8000001907349px"><br>
</div>
<div style="font-size:12.8000001907349px">3)
As a corollary to item 1, stacks for
which heat can never unwind the dependencies
and stay in DELETE_IN_PROGRESS forever.</div>
<div style="font-size:12.8000001907349px"><br>
</div>
<div style="font-size:12.8000001907349px">Does
anyone have any work-arounds for these or
recommendations on cleanup? My main worry is
removing a stack from the database that is
still consuming the customer's resources. I
also don't just want to remove stacks from
the database and leave orphaned records in
the DB.</div>
</div>
</div>
<br>
</div>
</div>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage
questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
</div></div></div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>