Open Stack

Hi Angus,

Looking at the service infrastructure stuff proposed for
openstack-common again:

  https://blueprints.launchpad.net/openstack-common/+spec/service-infrastructure

I always get a bit lost with all the classes and abstractions we have
going on this space, so I spent some time thinking about how it might be
improved.

None of this is necessarily stuff that needs fixing before your patches
can be merged, but I felt we needed to have a better picture of where
we're going.

So ... my attempt at simplifying the abstractions:

 - A Launcher class with the following methods:

     launch_service(service)
     stop()
     wait()

   A launcher can be responsible for multiple services.

   There are two subclasses of this. The first - ServiceLauncher - 
   handles a TERM/INT signal by stopping the services. The second -
   ProcessLauncher - will start a number of child processes, each child
   process running the service.

 - A Service class with the following methods:

     start()
     stop()
     wait()

   A service is associated with a Server object.

   A service also has a thread group whose members are stopped when the
   service is stopped.

 - A Server base class with the following methods:

     init(service)

   Server sub-classes PeriodicTasks, so it supports the @periodic_task
   decorator on its methods. The init() method of Server will spawn a
   timer thread from the service's thread group to run the periodic
   tasks from.

   There are two further sub-classes of Server which themselves are also
   base classes - rpc.Server and wsgi.Server. Sub-classes of rpc.Server
   and wsgi.Server will typically hook into the init() method and add
   methods which are decorated with @periodic_task.

 - The rpc.Server base class creates an RPC dispatcher which exposes all
   of the server's public methods over RPC via a given topic. All RPC
   calls will be handled in a thread spawned from the service's thread
   group.

 - The wsgi.Server base class exposes a WSGI over application a given 
   port and spawns threads from the service's thread group to handle
   requests.

The differences between this and what we currently have is:

 - We no longer have these fairly similar "manager" and "server"
   concepts that are completely different but mostly about WSGI vs RPC.

 - The functionality of the init_host() hook and periodic tasks are 
   available to both WSGI and RPC servers.

 - The Service class doesn't know anything about WSGI vs RPC servers, 
   all that knowledge is moved into the Server base classes.

 - init_host() is renamed to init(), since the "host" bit is fairly
   meaningless.

 - All green threads are spawned from the service's thread group.

But, actually, I'm now starting to struggle with justifying the
Server/Service split ... why do we need both?

Cheers,
Mark.