Open Stack

On Mon, Nov 09, 2015 at 04:05:51PM -0500, Sean Dague wrote:

:Can you be more specific about "upgrade process is hell!"? We continue
:to work on improvements in upgrade testing to block patches that will
:make life hell for upgrading. Getting a bunch of specifics on bugs that
:triggered during upgrade by anyone doing it would go a long way in
:helping us figure out what's the next soft spot to tackle there.

For myself it's the unknows no any particular bug so not sure there's
a patern to capture.

I rarely hit an upgrade bug first and typically do upgrades around mid
cycle (went to kilo in late August will go to Libery in January and
have ben on this rotation since essex/folsom update), but that usually
means I hit the bug go searching for it and find it's either in review
or merged (but not yet in my distro packages)

Of course hopefully I catch this in testing.  I've moved to using a
recent dump of my live produciton DB in final testing because many
bugs that hit me seem to relate to instances that we started long ago
(ie have already gone through at least one round of DB migrations).
But 'at scale' bugs are hard to test for until you're at scale (unless
50% of your hardware is dedicated to test)

I can take the halfday of full day API maintenance window so less of
an issue for my site, but Nova is the only service with a story for
really live update, I imagine that could wualify as 'Hell' for some.

On though on making the unkowns more know, perhaps if bugs were tagged
'upgrade' and then there was a single view that surfaced a cross
project view of that to operators things would be a bit less
exciting...


-Jon


:But without that data coming back in specifics it's hard to close
:whatever gap is here, real or perceived.
:
:	-Sean
:
:-- 
:Sean Dague
:http://dague.net
:
:_______________________________________________
:OpenStack-operators mailing list
:OpenStack-operators at lists.openstack.org
:http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

--