Open Stack

Hi all,

One of our priority goals for Liberty was the adoption of
oslo.versionedobjects in order for Cinder to achieve ability to do
rolling upgrades. We weren't successful with that in L, and work got
postponed to Mitaka. I want to highlight remaining work in that topic as
well as other pieces that are still missing in order for Cinder to
support no-downtime-upgrades.

Basically in order to be able to perform such upgrade we need make sure
that our services are compatible between versions. There is a set of
problems that needs to be solved:
* Non-compatible DB migrations (e.g. dropping or altering DB columns).
* Non-compatible RPC API changes (e.g. rename of an argument of a RPC
method).
* Non-compatible changes inside objects/dicts sent over RPC (e.g.
removal of a key there).

Good explanation of how Nova solves these can be found in a series of
posts by Dan Smith - [1][2][3]. I'll walk through all of these.

DB migrations
-------------
Since Juno no non-compatible DB migration was merged. We may stick to
this approach and allow only additive migrations to be performed (we
probably may allow dropping columns in further release - require that
only two subsequent releases are compatible). This is easy to prevent
using an unit test [4]. Another solution would be to implement online
schema migrations. This was implemented in Nova [5], but is considered
to be unstable and experimental.

RPC API compatibility
---------------------
We're already versioning our RPC layer, but we aren't actually
benefiting from it - we don't support RPC API pinning and don't pay
attention to merge only changes that are backward compatible. This
requires cultural change in reviewing and I think we should discuss the
approach at the Design Summit sprint.

Versioned Objects
-----------------
Right now there's a few outstanding DB-based objects:
* CGSnapshot (in review).
* Volume (partly in review).

You can find patches in [5].

Apart from that I think we need to convert dictionaries sent over RPC to
versioned objects. This would include:
* request_spec (scheduler.rpcapi)
* filter_properites (scheduler.rpcapi)
* capabilities (scheduler.rpcapi) - I'm not sure on this one…

Changing this is required for us to be able to remove or rename fields
in these dictionaries and still be able to provide interoperability of
services working in different versions.

I would love to get some feedback on these thoughts and possibly start a
pre-summit discussion on the whole topic.

[1] http://www.danplanet.com/blog/2015/10/05/upgrades-in-nova-rpc-apis/
[2] http://www.danplanet.com/blog/2015/10/06/upgrades-in-nova-objects/
[3] http://www.danplanet.com/blog/2015/10/07/upgrades-in-nova-database-migrations/
[4] https://github.com/openstack/nova/blob/master/nova/tests/unit/db/test_migrations.py#L186-L227
[5] http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/online-schema-changes.html
[6] https://review.openstack.org/#/q/status:open+project:openstack/cinder+branch:master+topic:bp/cinder-objects,n,z