Open Stack

Hey folks,

Based on my previous email, I promised to send out a note this week
about this topic.  My original email also included testing out the
development version of the upstream puppet modules. I was able to get
an environment working with the master branches of the OpenStack
Puppet modules but ran into further issues.  To simplify the
discussions, I've decided to split the topics into using upstream
packages and using upstream modules.  Below is the preliminary results
for the upstream package topic.  I plan on publicly posting (somewhere
TBD) both sets of results as well as additional instructions. I look
forward to comments.

Using alternative OpenStack packages with Fuel
----------------------------------------------

Fuel[0] is an open source deployment and management tool for
OpenStack.  Current versions of Fuel provide the ability to deploy and
configure OpenStack clouds utilizing Debian based packages on Ubuntu
based systems.  But if you wanted to use a different set of packages
for your cloud?  Well Fuel can be configured to deploy those packages
instead of the packages currently used.

The Plugin
----------

Fuel provides a plugin framework than can be used to inject additional
steps in the deployment process.  In addition to deployment tasks,
plugins can be used to create additional roles that can be assigned to
nodes.  With this framework, we were able to create a proof of concept
plugin[1] that can be used to configure specific nodes to use the
Ubuntu Cloud Archive[2] packages as their preferred OpenStack package
repository.

The plugin creates a new role that can be applied to specific nodes to
indicate that the UCA packages should be used.  The plugin sets a
puppet fact called os_package_type[3] which is will be consumed by the
upstream OpenStack Puppet modules to indicate which packages to
install and configure.  The Debian based packages which are the
default for Fuel have some minor differences from the UCA packages.
Our plugin overrides the default os_package_type to specify that we
want to use 'ubuntu' packages.

In addition to specifying the os_package_type, the plugin also pins a
specific set of packages that must be used with Fuel. Currently, the
haproxy package that is used with Fuel differs from a version provided
by Ubuntu based systems. At time of writing, this package needs to be
used so the plugin ensures this package set gets installed.  The last
configuration this plugin performs is to setup the UCA package
repository on the host system and sets the priority of this repository
to be higher than the default repository so that the UCA packages are
installed.

The plugin creates a role[4] that must be assigned the system(s) that
the user would like to use the UCA package set with.  Since a role is
provided, this would allow the end user fine grained control over
which packages are used for the node.  With this flexibility, a user
could use Debian based packages for Controller nodes and UCA based
packages for Compute nodes or vice versa.

Fuel Flexibility
----------------

With this proof of concept, Fuel shows its potential as a deployment
and management system for OpenStack. With the plugin, we are also able
to add additional configurations around what version of the packages
we would like to use. We are able to specify the Kilo or Liberty
packages[5] (and future Mikita, etc).  Since Fuel is leveraging
upstream OpenStack Puppet manifests, it should be able to support two
different packages versions for a given version of Fuel.  With the
plugin for a given environment, we would be able to test a future
version of the packages with the same version of fuel-library.

In addition to providing Ubuntu package installation support, as we
work on additional RedHat family support we would able to provide a
similar mechanism for RDO packages or another distribution's packages.
Because of the options provided by a plugin, we could even support
CentOS packages for Controllers and RDO packages for Compute nodes.

Issues
------

In order to get the plugin and alternate package set, a few items
needed to be addressed in fuel-library. A patch[6] was created to
address these issues.

 * The Debian based packages assume that python-mysqldb should be used
for the mysql driver. As such, we add an additional parameter to our
database connection string called read_timeout for our HA
configuration. Unfortunately the Ubuntu based packages assume that
python-pymysql should be used for the mysql drive.  In order to allow
for the Ubuntu based packages, we prepared a patch to remove the
read_timeout from the mysql database connection strings used.
 * The Debian based packages provide a heat-docker package which
provides docker files for Heat. Since there appears to be no such
package with the Ubuntu package set, we only install it when the
Debian based packages are installed.
 * During testing we found due to Bug 1499620[7] that there is a
version of the Ubuntu based packages have an issues with syslog so the
patch currently disables it for Ubuntu based packages.  Once the
packages get fixed, this workaround can be removed.
 * The Debian based packages have different service names for libvirt.
The patch provides a temporary fix until the change[8] has been
included in the upstream puppet-nova module.

With the patch and the Fuel development[9] tools, we created a custom
ISO to deploy a Fuel environment capable of deploying nodes using the
UCA packages.  Since the changes were limited to just fuel-library, we
could also get away with packaging an updated fuel-library RPM to be
used on with any current development ISO.

Conclusion
----------

With our custom ISO and plugin, we were able to spin up a new Fuel
deployed environment that was able to deploy a Controller node and a
Compute (with Cinder) node.  This environment passed all of the
expected OSTF tests.  The plugin provided us with the ability to
specify a third party UCA style repository and configure Fuel to
leverage it.  The next step after this effort would be to improve the
upstream OpenStack puppet modules with additional os_package_type
functionality, continue to address Fuel package specific logic within
fuel-library, and expand Fuel flexibility to continue alignment closer
to upstream OpenStack.

References
----------

[0] https://wiki.openstack.org/wiki/Fuel
[1] https://github.com/mwhahaha/fuel-plugin-upstream
[2] https://wiki.ubuntu.com/ServerTeam/CloudArchive
[3] https://review.openstack.org/#/c/241615/
[4] http://i.imgur.com/nQrrsxv.png
[5] http://i.imgur.com/WsgZGyP.png
[6] https://review.openstack.org/#/c/244959/
[7] https://bugs.launchpad.net/oslo.log/+bug/1499620
[8] https://review.openstack.org/#/c/243912/
[9] https://docs.fuel-infra.org/fuel-dev/develop/env.html

Thanks,
-Alex