Open Stack

Some TripleO folks are currently working hard on making automation for
upgrading TripleO between major OpenStack releases.
One of the biggest challenges is how to deal with Puppet who usually
notify Services when configuration change and Pacemaker who actually
manage the services.

Currently, we do this horrible thing that is disabling service
management when Pacemaker manages the service.
It's terrible because Puppet wants to manage the service with
'systemd' but we disable it to let Pacemaker deal with that.
The main limitation is that most of Puppet modules (specially in
OpenStack) know how to do orchestration for deployments & upgrades,
but we don't let it do because we disable service management.

A few months ago, we initiated a discussion about writing a Puppet
Resource Service Provider in openstack/puppet-pacemaker that will deal
with service management.
What will it change?
* we will enable service management again in THT.
* Puppet won't use systemd to deal with services, but use 'pcs' tool.
* We'll take the benefit of puppet-* modules orchestration. Example:
this configuration change will restart this service (something we had
hard time to do before).

We are currently working on it:
https://review.openstack.org/#/c/286124/

But this is work in progress:
* we need to implement the restart
* we need to create a special Puppet fact, that will make sure we only
run pcs on a single node (master by preference)
* investigate how to find who is master and stop hardcoding it in THT
(good for a bootstrap, but not good for upgrades).

Here is the kind of scenario we expect at the end:

Deployment of Neutron Server service
* openstack/puppet-neutron will configure neutron.conf
* openstack/puppet-neutron will notify neutron-server service
* THT is overriding the neutron-server service resource to use 'pcs'
provider, implemented in openstack/puppet-pacemaker
* once neutron.conf is configured on controllers, pcs will start
neutron-server, only from one node.

Upgrade of Neutron Server service
* we change RabbitMQ parameters in neutron.conf with THT
* openstack/puppet-neutron will update neutron.conf and notify
neutron-server service
* 'pcs' provider will restart neutron-server from one node, after the
change in the neutron.conf

That will be, in my opinion the most elegant way to make Puppet &
Pacemaker working together, any comment / feedback is highly welcome,
it will be a big change and we want to make it during Newton cycle.
Note: by designing this change, I found out we will also be able to
reduce the number of steps and also drop some bash code in the upgrade
scripts, which are both good news at making TripleO more consistent
and fast to deploy.

Thanks,
-- 
Emilien Macchi