[openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

Jay Pipes jaypipes at gmail.com
Fri Oct 14 20:55:39 UTC 2016


Alex, so sorry for the long delayed response! :( This just crept to
the back of my inbox unfortunately. Answer inline...

On 09/14/2016 07:24 PM, Bashmakov, Alexander wrote:
>> Glance and Keystone do not participate in a rolling upgrade,
>> because Keystone and Glance do not have a distributed component
>> architecture. Online data migrations will reduce total downtime
>> experienced during an *overall upgrade procedure* for an OpenStack
>> cloud, but Nova, Neutron and Cinder are the only parts of OpenStack
>> that are going to participate in a rolling upgrade because they are
>> the services that are distributed across all the many compute
>> nodes.
>
> Hi Jay, I'd like to better understand why your definition of rolling
> upgrades excludes Glance and Keystone? Granted they don't run
> multiple disparate components over distributed systems, however, they
> can still run the same service on multiple distributed nodes. So a
> rolling upgrade can still be applied on a large cloud that has, for
> instance 50 Glance nodes.

If you've seen a cloud with 50 Glance nodes, I would be astonished :) 
That said, the number 50 doesn't really have to do with my definition of 
rolling... lemme explain.

The primary thing that, to me at least, differentiates rolling upgrades 
of distributed software is that different nodes can contain multiple 
versions of the software and continue to communicate with other nodes in 
the system without issue.

In the case of Glance, you cannot have different versions of the Glance 
service running simultaneously within an environment, because those 
Glance services each directly interface with the Glance database and 
therefore expect the Glance DB schema to look a particular way for a 
specific version of the Glance service software.

In contrast, Nova's distributed service nodes -- the nova-compute 
services and (mostly) the nova-api services do *not* talk directly to 
the Nova database. If those services need to get or set data in the 
database, they communicate with the nova-conductor services which are 
responsible for translating (called back-versioning) the most updated 
object model schema that matches the Nova database to the schema that 
the calling node understands. This means that Nova deployers can update 
the Nova database schema and not have to at the same time update the 
software on the distributed compute nodes. In this way deployers can 
"roll out" an upgrade of the Nova software across many hundreds of 
compute nodes over an extended period of time without needing to 
restart/upgrade services all at once.

Hope this clarifies things.

Best,
-jay

p.s. I see various information on the web referring to "rolling updates" 
or "rolling releases" as simply the process of continuously applying new 
versions of software to a deployment. This is decidedly *not* what I 
refer to as a "rolling upgrade". Perhaps we should invent a different 
term from "rolling upgrade" to refer to the attributes involved in being 
able to run multiple versions of distributed software with no impact on 
the control plane? Is that what folks call a "partial upgrade"? Not sure...

  > In this case different versions of the
> same service will run on different nodes simultaneously. Regards,
> Alex





More information about the OpenStack-dev mailing list