Open Stack

Hi Jay,

Thanks for the explanation. While I agree that there is a distinction between a distributed architecture like Nova and a centralized one like Glance, I would respectfully disagree with the statement that Glance cannot participate in rolling upgrades in a very similar fashion. We are currently working on a rolling upgrade POC in Glance (https://review.openstack.org/331740/). To date, we've successfully been able to run through a simple scenario with two Glance nodes running Newton and Ocata code base respectively. The latter introduces schema changes which are reconciled in the DB via a two-way trigger.

Regards,
Alex

-----Original Message-----
From: Jay Pipes [mailto:jaypipes at gmail.com] 
Sent: Friday, October 14, 2016 1:56 PM
To: openstack-dev at lists.openstack.org
Subject: Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

Alex, so sorry for the long delayed response! :( This just crept to the back of my inbox unfortunately. Answer inline...

On 09/14/2016 07:24 PM, Bashmakov, Alexander wrote:
>> Glance and Keystone do not participate in a rolling upgrade, because 
>> Keystone and Glance do not have a distributed component architecture. 
>> Online data migrations will reduce total downtime experienced during 
>> an *overall upgrade procedure* for an OpenStack cloud, but Nova, 
>> Neutron and Cinder are the only parts of OpenStack that are going to 
>> participate in a rolling upgrade because they are the services that 
>> are distributed across all the many compute nodes.
>
> Hi Jay, I'd like to better understand why your definition of rolling 
> upgrades excludes Glance and Keystone? Granted they don't run multiple 
> disparate components over distributed systems, however, they can still 
> run the same service on multiple distributed nodes. So a rolling 
> upgrade can still be applied on a large cloud that has, for instance 
> 50 Glance nodes.

If you've seen a cloud with 50 Glance nodes, I would be astonished :) That said, the number 50 doesn't really have to do with my definition of rolling... lemme explain.

The primary thing that, to me at least, differentiates rolling upgrades of distributed software is that different nodes can contain multiple versions of the software and continue to communicate with other nodes in the system without issue.

In the case of Glance, you cannot have different versions of the Glance service running simultaneously within an environment, because those Glance services each directly interface with the Glance database and therefore expect the Glance DB schema to look a particular way for a specific version of the Glance service software.

In contrast, Nova's distributed service nodes -- the nova-compute services and (mostly) the nova-api services do *not* talk directly to the Nova database. If those services need to get or set data in the database, they communicate with the nova-conductor services which are responsible for translating (called back-versioning) the most updated object model schema that matches the Nova database to the schema that the calling node understands. This means that Nova deployers can update the Nova database schema and not have to at the same time update the software on the distributed compute nodes. In this way deployers can "roll out" an upgrade of the Nova software across many hundreds of compute nodes over an extended period of time without needing to restart/upgrade services all at once.

Hope this clarifies things.

Best,
-jay

p.s. I see various information on the web referring to "rolling updates" 
or "rolling releases" as simply the process of continuously applying new versions of software to a deployment. This is decidedly *not* what I refer to as a "rolling upgrade". Perhaps we should invent a different term from "rolling upgrade" to refer to the attributes involved in being able to run multiple versions of distributed software with no impact on the control plane? Is that what folks call a "partial upgrade"? Not sure...

  > In this case different versions of the
> same service will run on different nodes simultaneously. Regards, Alex



__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev