Open Stack

I've mentioned this in passing a few times, but I want to lay it out 
here in a bit more detail for comment. Basically we're implementing 
convergence at a time when we still have a lot of 'unit' tests that are 
really integration tests, and we don't want to have to rewrite them to 
anticipate this new architecture, nor wait until they have all been 
converted into functional tests. And of course it goes without saying 
that we have to land all of these changes without breaking anything for 
users.

To those ends, my proposal is that we (temporarily) support two code 
paths: the existing, legacy in-memory path and the new, distributed 
convergence path. Stacks will contain a field indicating which code path 
they were created with, and each stack will be operated on only by that 
same code path throughout its lifecycle (i.e. a stack created in legacy 
mode will always use the legacy code). We'll add a config option, off by 
default, to enable the new code path. That way users can switch over at 
a time of their choosing. When we're satisfied that it's stable enough 
we can flip the default (note: IMHO this would have to happen before 
kilo-3 in order to make it for the Kilo release).

Based on this plan, I had a go at breaking the work down into discrete 
tasks, and because it turned out to be really long I put it in an 
etherpad rather than include it here:

https://etherpad.openstack.org/p/heat-convergence-tasks

If anyone has additions/changes then I suggest adding them to that 
etherpad and replying to this message to flag your changes.

To be clear, it's unlikely I will have time to do the implementation 
work on any of these tasks myself (although I will be trying to review 
as many of them as possible). So the goal here is to get as many 
contributors involved in doing stuff in parallel as we can.

There are obviously dependencies between many of these tasks, so my plan 
is to raise each one as a blueprint so we can see the magic picture that 
Launchpad shows. I want to get feedback first though, because there are 
18 of them so far, and rejigging everything in response to feedback 
would be a bit of a pain.

I'm also prepared to propose specs for all of these _if_ people think 
that would be helpful. I see three options here:
  - Propose 18 fairly minimal specs (maybe in a single review?)
  - Propose 1 large spec (essentially the contents of that etherpad)
  - Just discuss in the etherpad rather than Gerrit

Obviously that's in decreasing order of the amount of work required, but 
I'll do whatever folks think best for discussion.

cheers,
Zane.