Open Stack

Back when Nova first wanted to test partial upgrade, we did a bunch of
slightly odd conditionals inside of grenade and devstack to make it so
that if you were very careful, you could just not stop some of the old
services on a single node, upgrade everything else, and as long as the
old services didn't stop, they'd be running cached code in memory, and
it would look a bit like a 2 node worker not upgraded model. It worked,
but it was weird.

There has been some interest by the Nova team to expand what's not being
touched, as well as the Neutron team to add partial upgrade testing
support. Both are great initiatives, but I think going about it the old
way is going to add a lot of complexity in weird places, and not be as
good of a test as we really want.

Nodepool now supports allocating multiple nodes. We have a multinode job
in Nova regularly testing live migration using this.

If we slice this problem differently, I think we get a better
architecture, a much easier way to add new configs, and a much more
realistic end test.

Conceptually, use devstack-gate multinode support to set up 2 nodes, an
all in one, and a worker. Let grenade upgrade the all in one, leave the
worker alone.

I think the only complexity here is the fact that grenade.sh implicitly
drives stack.sh. Which means one of:

1) devstack-gate could build the worker first, then run grenade.sh

2) we make it so grenade.sh can execute in parts more easily, so it can
hand something else running stack.sh for it.'

3) we make grenade understand the subnode for partial upgrade, so it
will run the stack phase on the subnode itself (given credentials).

This kind of approach means deciding which services you don't want to
upgrade doesn't require devstack changes, it's just a change of the
services on the worker.

We need a volunteer for taking this on, but I think all the follow on
partial upgrade support will be much much easier to do after we have
this kind of mechanism in place.

	-Sean

-- 
Sean Dague
http://dague.net