Open Stack

On 06/09/2017 03:10 PM, Zane Bitter wrote:
> History lesson: a long, long time ago we made a very big mistake. We
> treated stack outputs as things that would be resolved dynamically when
> you requested them, instead of having values fixed at the time the
> template was created or updated. This makes performance of reading
> outputs slow, especially for e.g. large stacks, because it requires
> making ReST calls, and it can result in inconsistencies between Heat's
> internal model of the world and what it actually outputs.
>
> As unfortunate as this is, it's difficult to change the behaviour and be
> certain that no existing users will get broken. For that reason, this
> issue has never been addressed. Now is the time to address it.
>
> Here's the tracker bug: https://bugs.launchpad.net/heat/+bug/1660831
>
> It turns out that the correct fix is to store the attributes of a
> resource in the DB - this accounts for the fact that outputs may contain
> attributes of multiple resources, and that these resources might get
> updated at different times. It also solves a related consistency issue,
> which is that during a stack update a resource that is not updated may
> nevertheless report new attribute values, and thus cause things
> downstream to be updated, or to fail, unexpectedly (e.g.
> https://bugzilla.redhat.com/show_bug.cgi?id=1430753#c13).
>
> The proposal[1] is to make this change in Pike for convergence stacks
> only. This is to allow some warning for existing users who might be
> relying on the current behaviour - at least if they control their own
> cloud then they can opt to keep convergence disabled, and even once they
> opt to enable it for new stacks they can keep using existing stacks in
> legacy mode until they are ready to convert them to convergence or
> replace them. In addition, it avoids the difficulty of trying to get
> consistency out of the legacy path's crazy backup-stack shenanigans -
> there's basically no way to get the outputs to behave in exactly the
> same way in the legacy path as they will in convergence.
>
> This topic was raised at the Forum, and there was some feedback that:
>
> 1) There are users who require the old behaviour even after they move to
> convergence.
> 2) Specifically, there are users who don't have public API endpoints for
> services other than Heat, and who rely on Heat proxying requests to
> other services to get any information at all about their resources o.O
> 3) There are users still using the legacy path (*cough*TripleO) that
> want the performance benefits of quick output resolution.
>
> The suggestion is that instead of tying the change to the convergence
> flag, we should make it configurable by the user on a per-stack basis.
>
> I am vehemently opposed to this suggestion.
>
> It's a total cop-out to make the user decide. The existing behaviour is
> clearly buggy and inconsistent. Users are not, and should not have to
> be, sufficiently steeped in the inner workings of Heat to be able to
> decide whether and when to subject themselves to random inconsistencies
> and hope for the best. If we make the change the default then we'll
> still break people, and if we don't we'll still be saying "OMG, you
> forgot to enable the --not-suck option??!" 10 years from now.
>
> Instead, this is what I'm proposing as the solution to the above feedback:
>
> 1) The 'show' attribute of each resource will be marked CACHE_NONE[2].
> This ensures that the live data is always available via this attribute.
> 2) When showing a resource's attributes via the API (as opposed to
> referencing them from within a template), always return live values.[3]
> Since we only store the attribute values that are actually referenced in
> the template anyway, we more or less have to do this if we want the
> attributes output through this API to be consistent with each other.
> 3) Move to convergence. Seriously, the memory and database usage are
> much improved, and there are even more memory improvements in the
> pipeline,[4] and they might even get merged in Pike as long as we don't
> have to stop and reimplement the attribute storage patches that they
> depend on. If TripleO were to move to convergence in Queens, which I
> believe is 100% feasible, then it would get the performance improvements
> at least as soon as it would if we tried to implement attribute storage
> in the legacy path.

I think we wanted to move to convergence anyway so I don't see a problem 
with this.  I know there was some discussion about starting to test with 
convergence in tripleo-ci, does anyone know what, if anything, happened 
with that?

>
> Is anyone still dissatisfied? Speak now or... you know the drill.
>
> cheers,
> Zane.
>
> [1]
> https://review.openstack.org/#/q/status:open+project:openstack/heat+branch:master+topic:bug/1660831
>
> [2] https://review.openstack.org/#/c/422983/33/heat/engine/resource.py
> [3] https://review.openstack.org/472501
> [4]
> https://review.openstack.org/#/q/status:open+project:openstack/heat+topic:bp/stack-definition
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev