Open Stack

Excerpts from Angus Salkeld's message of 2014-09-08 17:15:04 -0700:
> On Mon, Sep 8, 2014 at 11:22 PM, Tyagi, Ishant <ishant.tyagi at hp.com> wrote:
> 
> >  Hi All,
> >
> >
> >
> > As per the heat mid cycle meetup whiteboard, we have created the
> > flowchart and sequence diagram for the convergence . Can you please review
> > these diagrams and provide your feedback?
> >
> >
> >
> > https://www.dropbox.com/sh/i8qbjtgfdxn4zx4/AAC6J-Nps8J12TzfuCut49ioa?dl=0
> >
> >
> Great! Good to see something.
> 
> 
> I was expecting something like:
> engine ~= like nova-conductor (it's the only process that talks to the db -
> make upgrading easier)

This complicates things immensely. The engine can just be the workers
too, we're just not going to do the observing and converging in the same
greenthread.

> observer - purely gets the actual state/properties and writes then to the
> db (via engine)

If you look closely at the diagrams, thats what it does.

> worker - has a "job" queue and grinds away at running those (resource
> actions)
> 

The convergence worker is just another set of RPC API calls that split
out work into isolated chunks.

> Then engine then "triggers" on differences on goal vs. actual state and
> create a job and sends it to the job queue.

Remember, we're not targeting continuous convergence yet. Just
convergence when we ask for things.

> - so, on create it sees there is no actual state so it sends a create job
> for the first resource to the worker queue

The diagram shows that, but confusingly says "is difference = 1". In
the original whiteboard this is 'if diff = DNE'. DNE stands for Does
Not Exist.

> - when the observer writes the new state for that resource it triggers the
> next resource create in the dependency tree.

Not the next resource create, but the next resource convergence. And not
just one either. I think one of the graphs was forgotten, it goes like
this:

https://www.dropbox.com/s/1h2ee151iriv4i1/resolve_graph.svg?dl=0

That is what we called "return happy" because we were at hour 9 or so of
talking and we got a bit punchy. I've renamed it 'resolve_graph'.

> - like any system that relies on notifications we need timeouts and each
> stack needs a periodic "notification" to make sure


This is, again, the continuous observer model.

https://review.openstack.org/#/c/100012/

>   that progress is been made or notify the user that no progress is being
> made.
> 
> One question about the observer (in either my setup or the one in the
> diagram).
> - If we are relying on rpc notifications all the observer processes will
> receive a copy of the same notification

Please read that spec. We talk about a filter.