[openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

Clint Byrum clint at fewbar.com
Mon Sep 5 01:26:07 UTC 2016


Excerpts from Mike Bayer's message of 2016-09-02 17:58:42 -0400:
> 
> On 09/02/2016 01:53 PM, Doug Hellmann wrote:
> > Excerpts from Thierry Carrez's message of 2016-09-02 12:15:33 +0200:
> >> Sean Dague wrote:
> >>> Putting DB trigger failure analysis into the toolkit required to manage
> >>> an upgrade failure is a really high bar for new ops.
> >>
> >> I agree with Sean: increasing the variety of technologies used increases
> >> the system complexity, which in turn requires more skills to fully
> >> understand and maintain operationally. It should only be done as a last
> >> resort, with pros and cons carefully weighted. We really should involve
> >> operators in this discussion to get the full picture of arguments for
> >> and against.
> >>
> >
> > Yes, I would like to understand better what aspect of the approach
> > taken elsewhere is leading to the keystone team exploring other
> > options. So far I'm not seeing much upside to being different, and I'm
> > hearing a lot of cons.
> 
> I continue to maintain that the problems themselves being discussed at 
> https://review.openstack.org/#/c/331740/ are different than what has 
> been discussed in detail before.   To be "not different", this spec 
> would need to no longer discuss the concept of "we need N to be reading 
> from and writing to the old column to be compatible with N-1 as shown in 
> the below diagram...Once all the N-1 services are upgraded, N services 
> should be moved out of compatibility mode to use the new column. ". 
> To my knowledge, there are no examples of code in Openstack that 
> straddles table and column changes directly in the SQL access layer as 
> this document describes.    There's still a handful of folks including 
> myself that think this is a new kind of awkwardness we've not had to 
> deal with yet.   My only ideas on how to reduce it is to put the N-1/N 
> differences on the write side, not the read side, and triggers are *not* 
> the only way to do it.   But if "being different" means, "doing it on 
> the write side", then it seems like that overall concept is being 
> vetoed.  Which I actually appreciate knowing up front before I spend a 
> lot of time on it.
> 

The example for glance shows where two entirely new objects have been
created for the database (community and shared images). The compatibility
mode flag in config is cool, I think operators deal with things like
that all the time, like when a new API version arrives and they might
not be ready to support it. I'd hope that having it turned off would
also restrict the API microversion if such a thing exists so that the
community/shared image types aren't allowed yet. This seems straight
forward, and I feel like the spec was good except for the addition of
extra layers.

In this case, I'd just create the new column nullable, and maintain
both.

* Add visibility column to schema (in spec, 'glance-manage db_expand')

* upgrade all API nodes

* run the migration code to resolve the null visibility columns
  (traditional "glance-manage db_migrate")

* advance compatibility mode to lowest commit that exists running
  against DB

* set visibility to be not null (I think this would be 'glance-manage
  db_contract latest_commit_desired')

Where, in this scheme, do triggers come in?



More information about the OpenStack-dev mailing list