Open Stack

Thanks to everyone who gave great feedback on the first queue service
thread. I've updated the wiki page to include the suggestions.

http://wiki.openstack.org/QueueService

With a decent vision of what we want to build, the next step is
figuring out how. In a previous thread it was suggested that the
preferred languages for OpenStack projects are Python, C, and
C++. Since there is an emphasis on speed and efficiency for the
queue service, I suggest we use C++. I expect this service to be
CPU bound and would benefit being able to leverage multiple cores
efficiently (within the same process), so I don't think Python
is a good fit. I think C++ is a better fit than C due to the need
for modular interfaces. While this can obviously be done in C, C++
APIs are more concise and much less error prone. The OO style will
also make it easier for Python developers who also want to learn and
assist with C++ projects.

Erlang is not on the preferred lists, but I would also put it out
there as an option. While it may be a great fit for a project like
this, I worry it won't attract the developer resources since Erlang
isn't really a first-class language yet.

If we decide to take the C++ path, I propose using a modular
application framework I've been working on over the past year (mostly
in my spare time). It provides a simple module programming interface
with dependency tracking (kind of like Linux kernel modules). It
already provides a multi-threaded event module (currently based on
libevent, but this is pluggable) with simple networking abstractions
built on top of it. We should be able to dive in an start writing the
HTTP protocol module and queue processing modules. You can check out
the current project at:

https://launchpad.net/scalestack
http://scalestack.org/

The intention of using a framework like this is so we can easily reuse
the other modules (auth, HTTP, logging, ...) for other OpenStack
services in the future. Much like we use Eventlet, WSGI, etc. (and
eventually openstack-common) for Python, we could prefer using the
modules in Scale Stack for lower level projects.

Thoughts?
-Eric