Open Stack

On Thu, Aug 25, 2016 at 1:13 PM, Steve Martinelli <s.martinelli at gmail.com>
wrote:

> The keystone team is pursuing a trigger-based approach to support rolling,
> zero-downtime upgrades. The proposed operator experience is documented here:
>
>   http://docs.openstack.org/developer/keystone/upgrading.html
>
> This differs from Nova and Neutron's approaches to solve for rolling
> upgrades (which use oslo.versionedobjects), however Keystone is one of the
> few services that doesn't need to manage communication between multiple
> releases of multiple service components talking over the message bus (which
> is the original use case for oslo.versionedobjects, and for which it is
> aptly suited). Keystone simply scales horizontally and every node talks
> directly to the database.
>
> Database triggers are obviously a new challenge for developers to write,
> honestly challenging to debug (being side effects), and are made even more
> difficult by having to hand write triggers for MySQL, PostgreSQL, and
> SQLite independently (SQLAlchemy offers no assistance in this case), as
> seen in this patch:
>
>   https://review.openstack.org/#/c/355618/
>
> However, implementing an application-layer solution with
> oslo.versionedobjects is not an easy task either; refer to Neutron's
> implementation:
>
>   https://review.openstack.org/#/q/topic:bp/adopt-oslo-
> versioned-objects-for-db
>
> Our primary concern at this point are how to effectively test the triggers
> we write against our supported database systems, and their various
> deployment variations. We might be able to easily drop SQLite support (as
> it's only supported for our own test suite), but should we expect variation
> in support and/or actual behavior of triggers across the MySQLs, MariaDBs,
> Perconas, etc, of the world that would make it necessary to test each of
> them independently? If you have operational experience working with
> triggers at scale: are there landmines that we need to be aware of? What is
> it going to take for us to say we support *zero* dowtime upgrades with
> confidence?
>
> Steve & Dolph
>
>

No experience to add for triggers, but I'm happy to help test this on a
MySQL Galera cluster. I'd also like to add thanks for looking into this. A
keystone outage is a cloud outage and being able to eliminate them from
upgrades will be beneficial to everyone.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160825/891aa4ec/attachment.html>