[openstack-dev] Mixed service version CI testing

Matt Riedemann mriedemos at gmail.com
Tue Dec 19 15:58:34 UTC 2017

During discussion in the TC channel today [1], we got talking about how 
there is a perception that you must upgrade all of the services together 
for anything to work, at least the 'core' services like 
keystone/nova/cinder/neutron/glance - although maybe that's really just 

Anyway, I posit that the services are not as tightly coupled as some 
people assume they are, at least not since kilo era when microversions 
started happening in nova.

However, with the way we do CI testing, and release everything together, 
the perception is there that all things must go together to work.

In our current upgrade job, we upgrade everything to N except the 
nova-compute service, that remains at N-1 to test rolling upgrades of 
your computes and to make sure guests are unaffected by the upgrade of 
the control plane.

I asked if it would be valuable to our users (mostly ops for this 
right?) if we had an upgrade job where everything *except* nova were 
upgraded. If that's how the majority of people are doing upgrades anyway 
it seems we should make sure that works.

I figure leaving nova at N-1 makes more sense because nova depends on 
the other services (keystone/glance/cinder/neutron) and is likely the 
harder / slower upgrade if you're going to do rolling upgrades of your 
compute nodes.

This type of job would not run on nova changes on the master branch, 
since those changes would not be exercised in this type of environment. 
So we'd run this on master branch changes to 

Does that make sense? Would this be valuable at all? Or should the 
opposite be tested where we upgrade nova to N and leave all of the 
dependent services at N-1?

Really looking for operator community feedback here.





More information about the OpenStack-dev mailing list