Open Stack

On 05/24/2013 08:36 PM, Doug Hellmann wrote:
>
>
>
> On Fri, May 24, 2013 at 12:39 PM, Tomas Sedovic <tsedovic at redhat.com
> <mailto:tsedovic at redhat.com>> wrote:
>
>     Hello all,
>
>     I wrote a small proof of concept for the realtime communication in
>     Horizon[1][2]. The code is on Gerrit awaiting your feedback. It's
>     just two smallish commits:
>
>     https://review.openstack.org/__30462
>     <https://review.openstack.org/30462>
>     https://review.openstack.org/__30463
>     <https://review.openstack.org/30463>
>
>     I would very much like anyone interested to try it out and send
>     their thoughts. Below are notes on the implementation and a few
>     things that we should hash out.
>
>
>     ## The Realtime library
>
>     The PoC is written in Python and integrated with Horizon. We're all
>     familiar with Python and we can leverage the existing infrastructure
>     (views and templates, the Request objects for API authentication,
>     deploy both the realtime portion and the traditional code on a
>     single server and the common RPC API).
>
>     As for the specific transport implementation, I picked
>     gevent-socketio[3]. I wanted to use a Socket.IO-compatible solution
>     because it supports several fallbacks for WebSockets and allows us
>     to switch to Node.js later if necessary.
>
>     There are a few Python socket.io <http://socket.io> implementations
>     out there:
>
>     https://github.com/abourget/__gevent-socketio
>     <https://github.com/abourget/gevent-socketio>
>     https://github.com/MrJoes/__tornadio2
>     <https://github.com/MrJoes/tornadio2>
>     https://github.com/stephenmcd/__django-socketio
>     <https://github.com/stephenmcd/django-socketio>
>
>     django-socketio currently supports an out-of-date version of the
>     Socket.IO protocol. We may want to revisit it once they get up to date.
>
>     The development of tornadio2 seems to be less active, and
>     gevent-socketio seems to be more readily integrated into various web
>     frameworks. But it seems either would work.
>
>
>     ## Deployment
>
>     No matter what backend we end up using, the deployment will get more
>     complicated. The WSGI specification doesn't support websockets-like
>     transfer so this (or any other solution I could find) won't work
>     with vanilla WSGI servers (most notably mod_wsgi).
>
>     The socket.io <http://socket.io> part of Horizon can be served via
>     Gevent or Gunicorn and since both support WSGI, we can serve the
>     rest of the app alongside. An alternative is to keep using mod_wsgi
>     for the non-realtime part and a separate process for realtime.
>
>     While this complicates things, it isn't worse than using Node.js:
>     there we must have two servers: one for Django and one for Node.
>
>
>     ## Functionality
>
>     The code doesn't do much at the moment. It's more about laying the
>     ground work and having something tangible we can discuss. So the
>     only functionality right now is that in the /project/instances/
>     page, the change in the instances' statuses are continuously updated
>     using socket.io <http://socket.io> instead of the existing ajax polling.
>
>     If you have a few instances running, when you visit the page and
>     then change their status out of band (e.g. via the command line),
>     this should be reflected in the UI.
>
>     I purposefully avoided two thornier issues that need to be resolved
>     before this is of any use. These should be discussed and decided
>     before we jump in and start to write more code:
>
>     1. The current way of updating the UI is to send the new chunk of
>     HTML to the browser. For the purposes of the realtime communication,
>     it would (I think) make more sense to send just the updated data and
>     update the UI client-side.
>
>     We can of course keep sending the HTML chunks, if that's the way you
>     all want to go, but it limits what we can do.
>
>     2. Horizon is mostly a request/response UI. To be able to push
>     changes to the browsers, the server needs to figure out when things
>     actually changed. We could receive the notifications via RPC or we
>     poll for the changes internally and then broadcast them to the browsers.
>
>     I wonder if we could leverage Ceilometer here since it already
>     listens in on the message bus. Could it be configured to send
>     interesting information to Horizon (as opposed to being polled for it)?
>
>
> This really feels like a different use case that what ceilometer is made
> for. Gabriel and I talked at the grizzly summit about the code we have
> in ceilometer for listening to notifications, and we agreed to try to
> put that in oslo somewhere to make it easier to create a similar
> listening service in horizon that redirects the notifications to the
> browser(s). I haven't had a chance to do that. If you want to tackle it,
> I can help figure out which bits are interesting/useful.

Thanks, I thought that might be the case but I didn't know enough to be 
sure.

Putting the bits to oslo sounds like a neat idea. I'm certainly willing 
to look into it, but I'd like to get some feedback on this first.

>
> Doug
>
>
>
>     I'm not sure whether it makes sense to push the code or not, but I
>     tried to make sure I didn't break the standard set up.
>
>     Please let me know what you think.
>
>     Thanks,
>     Tomas Sedovic
>
>
>     [1]:
>     https://blueprints.launchpad.__net/horizon/+spec/realtime-__communication
>     <https://blueprints.launchpad.net/horizon/+spec/realtime-communication>
>     [2]: https://etherpad.openstack.__org/havana-horizon-realtime
>     <https://etherpad.openstack.org/havana-horizon-realtime>
>     [3]: https://github.com/abourget/__gevent-socketio
>     <https://github.com/abourget/gevent-socketio>
>
>     _________________________________________________
>     OpenStack-dev mailing list
>     OpenStack-dev at lists.openstack.__org
>     <mailto:OpenStack-dev at lists.openstack.org>
>     http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>