Open Stack

On 23-Sep-14 09:42, Clint Byrum wrote:
> Excerpts from Angus Salkeld's message of 2014-09-22 20:15:43 -0700:
>> On Tue, Sep 23, 2014 at 1:09 AM, Anant Patil <anant.patil at hp.com> wrote:
>>
>>> Hi,
>>>
>>> One of the steps in the direction of convergence is to enable Heat
>>> engine to handle concurrent stack operations. The main convergence spec
>>> talks about it. Resource versioning would be needed to handle concurrent
>>> stack operations.
>>>
>>> As of now, while updating a stack, a backup stack is created with a new
>>> ID and only one update runs at a time. If we keep the raw_template
>>> linked to it's previous completed template, i.e. have a back up of
>>> template instead of stack, we avoid having backup of stack.
>>>
>>> Since there won't be a backup stack and only one stack_id to be dealt
>>> with, resources and their versions can be queried for a stack with that
>>> single ID. The idea is to identify resources for a stack by using stack
>>> id and version. Please let me know your thoughts.
>>>
>>
>> Hi Anant,
>>
>> This seems more complex than it needs to be.
>>
>> I could be wrong, but I thought the aim was to simply update the goal state.
>> The backup stack is just the last working stack. So if you update and there
>> is already an update you don't need to touch the backup stack.
>>
>> Anyone else that was at the meetup want to fill us in?
>>
> 
> The backup stack is a device used to collect items to operate on after
> the current action is complete. It is entirely an implementation detail.
> 
> Resources that can be updated in place will have their resource record
> superseded, but retain their physical resource ID.
> 
> This is one area where the resource plugin API is particularly sticky,
> as resources are allowed to raise the "replace me" exception if in-place
> updates fail. That is o-k though, at that point we will just comply by
> creating a replacement resource as if we never tried the in-place update.
> 
> In order to facilitate this, we must expand the resource data model to
> include a version. Replacement resources will be marked as "current" and
> to-be-removed resources marked for deletion. We can also keep all current
> - 1 resources around to facilitate rollback until the stack reaches a
> "complete" state again. Once that is done, we can remove the backup stack.
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

Backup stack is a good way to take care of rollbacks or cleanups after
the stack action is complete. By cleanup I mean the deletion of
resources that are no longer needed after the new update. It works very
well when one engine is processing the stack request and the stacks are
in memory.

As a step towards distributing the stack request processing and making
it fault-tolerant, we need to persist the dependency task graph. The
backup stack can also be persisted along with the new graph, but then
the engine has to traverse both the graphs to proceed with the operation
and later identify the resources to be cleaned-up or rolled back using
the stack id. There would be many resources for the same stack but
different stack ids.

In contrast, when we store the current dependency task graph(from the
latest request) in DB, and version the resources, we can identify those
resources that need to be rolled-back or cleaned up after the stack
operations is done, by comparing their versions. With versioning of
resources and template, we can avoid creating a deep stack of backup
stacks. The processing of stack operation can happen from multiple
engines, and IMHO, it is simpler when all the engines just see one stack
and versions of resources, instead of seeing many stacks with many
resources for each stack.

- Anant