[Openstack-operators] Mixed service version CI testing

Sam Morrison sorrison at gmail.com
Tue Jan 2 07:30:13 UTC 2018

We usually upgrade nova last so would be helpful. 

Nectar has been running a mix of versions for a couple of years now and we treat each project as it’s own thing and upgrade everything separately.

You can see what versions we run currently at https://trello.com/b/9fkuT1eU/nectar-openstack-versions <https://trello.com/b/9fkuT1eU/nectar-openstack-versions>


> On 29 Dec 2017, at 4:28 am, Clint Byrum <clint at fewbar.com> wrote:
> Excerpts from Matt Riedemann's message of 2017-12-19 09:58:34 -0600:
>> During discussion in the TC channel today [1], we got talking about how 
>> there is a perception that you must upgrade all of the services together 
>> for anything to work, at least the 'core' services like 
>> keystone/nova/cinder/neutron/glance - although maybe that's really just 
>> nova/cinder/neutron?
>> Anyway, I posit that the services are not as tightly coupled as some 
>> people assume they are, at least not since kilo era when microversions 
>> started happening in nova.
>> However, with the way we do CI testing, and release everything together, 
>> the perception is there that all things must go together to work.
>> In our current upgrade job, we upgrade everything to N except the 
>> nova-compute service, that remains at N-1 to test rolling upgrades of 
>> your computes and to make sure guests are unaffected by the upgrade of 
>> the control plane.
>> I asked if it would be valuable to our users (mostly ops for this 
>> right?) if we had an upgrade job where everything *except* nova were 
>> upgraded. If that's how the majority of people are doing upgrades anyway 
>> it seems we should make sure that works.
>> I figure leaving nova at N-1 makes more sense because nova depends on 
>> the other services (keystone/glance/cinder/neutron) and is likely the 
>> harder / slower upgrade if you're going to do rolling upgrades of your 
>> compute nodes.
>> This type of job would not run on nova changes on the master branch, 
>> since those changes would not be exercised in this type of environment. 
>> So we'd run this on master branch changes to 
>> keystone/cinder/glance/neutron/trove/designate/etc.
>> Does that make sense? Would this be valuable at all? Or should the 
>> opposite be tested where we upgrade nova to N and leave all of the 
>> dependent services at N-1?
> It makes sense completely. What would really be awesome would be to test
> the matrix of single upgrades:
> upgrade only keystone
> upgrade only glance
> upgrade only neutron
> upgrade only cinder
> upgrade only nova
> That would have a good chance at catching any co-dependencies that crop
> up.
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org <mailto:OpenStack-operators at lists.openstack.org>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20180102/ed898cd2/attachment.html>

More information about the OpenStack-operators mailing list