Open Stack

On 5/21/15 2:57 PM, Mike Bayer wrote:
>
>
> On 5/21/15 2:50 PM, Mike Bayer wrote:
>>
>> Put another way, if I want to do a migration that someday, "over 
>> time", eventually, etc., will drop the "widget_status" table and 
>> merge it into the "widget" table, how do I write the "get_widget()" 
>> DB API call ?     Same for just dropping a column.   If we have to 
>> write code that will return the value of the old table / old column 
>> while it exists, how does that part work?   Somewhere, there has to 
>> be code that in one case says: "query(Widget).join(WidgetStatus)" and 
>> in other case says "query(Widget)".   You have to maintain *both* SQL 
>> queries at the same time?
> OK I guess it is this: "Kilo queries Widget JOIN WidgetStatus while 
> Liberty queries just Widget" - e.g. the two versions of the query are 
> just in two different releases of the application. You need to run 
> exclusively the old sqlalchemy/api.py version of the code before 
> "contract" is run, that is, the new API / nova.objects runs against 
> the old model code.
>
> OK!   let me chew on that.
hmm, still not grokking this.


"Kilo" application has:

     query(Widget).join(WidgetStatus)  # for queries, and:

     session.add(Widget)
     session.add(WidgetStatus) # for persistence.

It has no knowledge that Widget and WidgetStatus are merged.  Or does 
it?  (in which case, yuk?)

"Liberty" application has:

     query(Widget)  # for queries, and:

     session.add(Widget) # for persistence.


It has no knowledge that WidgetStatus exists.  Or does it?  (in which 
case, yuk?)

It seems like the two possibilities are that either a database API 
somewhere needs to know of both "endpoint" states of the schema, or the 
oslo.objects API needs to ensure that it communicates data across two 
different versions of the SQL API at the same time in order to do data 
migrations.     Either way I don't know how that works.

My concern that given a series of migrations A, B, C, D, and E, the 
states of B, C and D *do* disappear as new states are added, is true.   
It's just that the data migrations here are not simplistic scripts but 
are instead embedded into the oslo.objects and possibly the model 
objects as well, and these must be maintained as new states are added to 
accommodate the new reality of the schema from a POV of only the 
previous major release to the next major release.

I understand that this is all towards the notion that an operator can 
have dozens of apps hitting the same database and all those apps are in 
a mixture of "Kilo" / "Liberty" and all work the same.   That is a hard 
problem to solve.    I only regret that I haven't had the opportunity to 
put any thinking of my own into this problem; while it was part of my 
original mandate starting work at Red Hat, the versioned objects 
approach was already moving 100 miles an hour from my first day on the 
job and this is not a train I've been able to catch.   I'm really hoping 
to be involved with it, as it is my job, though each time I attempt to 
go there it feels a bit impenetrable.

A side-channel convo with Jay confirms that he had all these same 
questions / confusion as well - somewhere, his confusion was cleared up, 
though I don't know where that is.   Might this be a really good reason 
for this architecture to somewhere be documented really, really 
completely and thoroughly, with detailed workflow examples and such?  If 
that's been done, can someone link me please?