Open Stack

Thanks Angus for your comments.

Your design is almost same as this one. I also agree that only engine should have DB access will DB rpc api’s. I will update the diagrams with this change.

Regarding the worker communicating with the observer, flow would be like this:

·         Engine tells worker to create or update a resource.

·         Worker then just calls resource plugin  handle_create / handle_update etc and calls observer rpc api to observer the resource ( check_create_complete ) and then exits.

·         Observer then checks the resource status until it comes to the desired state.

·         Main engine then gets back the notification from observer and then schedule next parent resource to converge.

If observer and worker are independent entities then who will invoke observer to check resource state ?

-Ishant
From: Angus Salkeld [mailto:asalkeld at mirantis.com]
Sent: Tuesday, September 9, 2014 5:45 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Heat] convergence flow diagrams

On Mon, Sep 8, 2014 at 11:22 PM, Tyagi, Ishant <ishant.tyagi at hp.com<mailto:ishant.tyagi at hp.com>> wrote:
Hi All,

As per the heat mid cycle meetup whiteboard, we have created the flowchart and sequence diagram for the convergence . Can you please review these diagrams and provide your feedback?

https://www.dropbox.com/sh/i8qbjtgfdxn4zx4/AAC6J-Nps8J12TzfuCut49ioa?dl=0

Great! Good to see something.

I was expecting something like:
engine ~= like nova-conductor (it's the only process that talks to the db - make upgrading easier)
observer - purely gets the actual state/properties and writes then to the db (via engine)
worker - has a "job" queue and grinds away at running those (resource actions)

Then engine then "triggers" on differences on goal vs. actual state and create a job and sends it to the job queue.
- so, on create it sees there is no actual state so it sends a create job for the first resource to the worker queue
- when the observer writes the new state for that resource it triggers the next resource create in the dependency tree.
- like any system that relies on notifications we need timeouts and each stack needs a periodic "notification" to make sure
  that progress is been made or notify the user that no progress is being made.

One question about the observer (in either my setup or the one in the diagram).
- If we are relying on rpc notifications all the observer processes will receive a copy of the same notification
  (say nova create end) how are we going to decide on which one does anything with it?
  We don't want 10 observers getting more detailed info from nova and then writing to the db

In your diagram worker is communicating with observer, which seems odd to me. I thought observer and worker were very
independent entities.

In my setup there are less API to worry about too:
- RPC api for the engine (access to the db)
- RPC api for sending a job to the worker
- the plugin API
- the observer might need an api just for the engine to tell it to start/stop observing a stack
-Angus


Thanks,
Ishant


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140909/1a69c9f9/attachment.html>