Open Stack

We've been talking about this since before summit without much 
consensus. I think a large part of the problem is that very few people 
have deep knowledge of both Heat and Versioned Objects. However, I think 
we are at a point where we should be able to settle on an approach at 
least for the API<->engine RPC interface. I've been talking to Dan Smith 
about the Nova team's plan for upgrades, which goes something like this:

* Specify a max RPC API version in the config file
* In the RPC client lib, add code to handle versions as far back as the 
previous release
* The operator rolls out the updated code, keeping the existing config 
file with the pin to the previous release's RPC version
* Once all services are upgraded, the operator rolls out a new config 
file shifting the pin
* The backwards compat code to handle release N-1 is removed in the N+1 
release

This is, I believe, sufficient to solve our entire problem. 
Specifically, we have no need for an indirection API that rebroadcasts 
messages that are too new (since that can't happen with pinning) and no 
need for Versioned Objects in the RPC layer. (Versioned objects for the 
DB are still critical, and we are very much better off for all the hard 
work that Michal and others have put into them. Thanks!)

The nature of Heat's RPC API is that it is effectively user-facing - the 
heat-api process is essentially a thin proxy between ReST and RPC. We 
already have a translation layer between the internal representation(s) 
of objects and the user-facing representation, in the form of 
heat.engine.api, and the RPC API is firmly on the user-facing side. The 
requirements for the content of these messages are actually much 
stricter than anything we need for RPC API stability, since they need to 
remain compatible not just with heat-api but with heatclient - and we 
have *zero* control over when that gets upgraded. Despite that, we've 
managed quite nicely for ~3 years without breaking changes afaik.

Versioned Objects is a great way of retaining control when you need to 
share internal data structures between processes. Fortunately the 
architecture of Heat makes that unnecessary. That was a good design 
decision. We are not going to reverse that design decision in order to 
use Versioned Objects. (In the interest of making sure everyone uses 
their time productively, perhaps I should clarify that to: "your patch 
is subject to -2 on sight if it introduces internal engine data 
structures to heat-api/heat-cfn-api".)

Hopefully I've convinced you of the sufficiency of this plan for the 
API<->engine interface specifically. If anyone disagrees, let them speak 
now, &c.


I think there is still a case that could be made for a different 
approach to the RPC API for convergence, which is engine->engine and 
(probably) doesn't yet have a formal translation layer of the same kind. 
At a minimum, obviously, we should do the same stuff listed above 
(though I don't think we need to declare that interface stable until the 
first release where we enable convergence by default).

There's probably places where versioned objects could benefit us. For 
example, when we trigger a check on a resource we pass it a bundle of 
data containing all the attributes and IDs it might need from resources 
it depends on. It definitely makes sense to me that that bundle would be 
a Versioned Object. (In fact, that data gets stored in the DB - as 
SyncPoint in the prototype - so we wouldn't even need to create a new 
object type. This seems like a clear win.)

What I do NOT want to do is to e.g. replace the resource_id of the 
resource to check with a versioned object containing the DB 
representation of a resource. In most cases that doesn't save a DB 
lookup, it just moves it to the other end of the RPC call. And as long 
as primary keys are unique, this can never create a compatibility problem :)

Grzegorz mentioned in https://review.openstack.org/#/c/196670/ the 
possibility of using the indirection API to handle changes to the 
message format - I guess this is an alternative to the version pinning 
approach mentioned by Dan? I'm currently leaning toward thinking that 
this is unnecessary, but I'm not sure I understand it completely so I'd 
need to see a patch to form an opinion. I'm wide open to persuasion on 
this part. However I'd be opposed to this *replacing* (rather than 
supplementing) the version pinning behaviour described above - the whole 
retransmission of too-new messages thing sounds like it could have any 
number of pathological corner cases and I'd prefer to avoid it.

cheers,
Zane.