Open Stack

On 11/12/2015 02:41 PM, Korzeniewski, Artur wrote:
> Hi Sean,
> 
> I’m interested in introducing to Neutron the multinode partial upgrade
> job in Grenade.
> 
>  
> 
> Can you explain how multinode is currently working in Grenade and how
> Nova is doing the partial upgrade?

We're hopefully a couple of days out from making this voting. Here is
how it works conceptually:

Grenade itself knows how to upgrade 1 node. For simplicity sake we've
left it that way. Putting a real orchestration layer into Grenade would
be... potentially challenging. However Grenade implicitly runs stack.sh
today to make developer's life easier.

Devstack-gate knows how to setup a 2 nodes, and has the rest of the
multinode logic. Under a grenade multinode environment we:

* Allocate 2 nodes

* Setup all the source trees and configs correctly on all the nodes
(which includes less services on the workers) -
https://github.com/openstack-infra/devstack-gate/blob/92d130938406e4b42cdb1fe3e6fa62f3a2466024/devstack-vm-gate.sh#L206-L222

* Then we give Grenade a post-stack.sh script which includes the logic
to run stack.sh on all the subnodes at the right time -
https://github.com/openstack-infra/devstack-gate/blob/92d130938406e4b42cdb1fe3e6fa62f3a2466024/devstack-vm-gate.sh#L587-L597

* We run grenade

* It runs stack.sh on the main node, runs stack.sh on the subnode
because of post-stack.sh, then proceeds to run:

   - tempest smoke
   - creating of long running resources
   - shutsdown the controller / upgrades / restarts
   - verifies the long running resources are still there
   - runs tempest smoke
   - success

It completely ignores the subnode after post-stack.sh, which means that
will continue soldiering on as the stable version.

That means support for a new partial upgrade scenario is really only 2
things:

1. generic multinode support for the collection of services you want
(i.e. a definition of what's in the subnode).
2. support in grenade (or via a plugin) for upgrading the entire
controller node.

Fortunately, I believe Neutron already has #1 and #2 because of existing
jobs, so getting a multinode grenade job should just be a matter of a
project-config stanza. No additional code needs to be written. I'm sure
there might be some bugs (there always are), but getting rolling
shouldn't be too bad.

	-Sean

-- 
Sean Dague
http://dague.net