[openstack-dev] [cinder] Testing Cinder upgrades - c-bak upgrade

Michał Dulko michal.dulko at intel.com
Sat Jan 30 00:36:30 UTC 2016

On 01/20/2016 09:11 PM, Li, Xiaoyan wrote:
> @ DuncanT and @dule: 
> I noticed from IRC log that you are discussing about c-bak upgrade, and I am working on this, please see following message. Hope I don't miss anything.
> As you know, currently c-bak and c-vol are in same nodes. c-bak depends on c-vol service. 
> But it is not necessary that all c-vols needs to be upgraded before c-backs.
> The sequences can be random. As described in the patch https://review.openstack.org/#/c/269412/,
> when c-api decides which c-bak service to create/restore, it checks the version of c-vol. If c-vol is new version, find a c-bak in new version.
> If c-vol is in old version, find a c-bak in old version.
> Let's us think a real case. Customers upgrade c-api, c-sch, and start to upgrade c-vol and c-bak.
> There are two c-vol services c-vol1 and c-vol2, and two c-bak services c-bak1 and c-bak2.
> There are four typical upgrade sequences as following. 
> Meanwhile, please notice that c-vol and c-bak are in same nodes in Liberty. So during upgrades, if c-vol went down to upgrade, c-bak is also down. 

It's not exactly like that. You may upgrade services on a single node
one-by-one if you're for example running them in containers.

> 1. c-vol1->c-bak1->c-vol2->c-bak2
> The only insufficiency is when c-vol1 upgrades, and other c-bak services haven't upgraded, the request to create/restore backups from/to volumes in c-vol1 will fail with reason "no valid c-bak service
> found". It is acceptable, as it is similar to scenario in Liberty: c-vol active and c-bak fails.
> 2. c-vol1->c-vol2->c-bak1->c-bak2
> Before c-bak1 upgrades, no back request can be completed as no active c-bak services. This is reasonable.
> 3. c-bak1->c-vol1->c-bak2->c-vol2:
> The issue is when c-bak2 upgrades, the request to create/restore backups from/to volumes in c-vol2 will fail with reason c-vol not active. This is consistent with behaviors in Liberty.
> 4. c-bak1->c-bak2->c-vol1->c-vol2: 
> Before c-vol1 upgrades, no back request can be completed as c-vol services not active This is reasonable.

Resolution on this matter from the Cinder mid-cycle is that we're fine
as long as we safely fail in case of upgrade conducted in an improper
order. And it seems we can implement that in a simple way by raising an
exception from volume.rpcapi when c-vol is pinned to a version too old.
This means that scalable backup patches aren't blocked by this issue.

More information about the OpenStack-dev mailing list