Open Stack

TL;DR We're doing a PoC to split out common testing plays/vars into a
generic repository, for the purposes of making in-role testing more uniform
and generic. Looking for feedback, comments, informal reviews, ideas etc!
https://github.com/andymcc/openstack-ansible-testing-core (Includes a
simple README)

We now have a lot of duplication after moving to a single role per
repository structure with specific testing per role. For example, almost
all repositories require Galera/Rabbit/Keystone in order to deploy testing
successfully. This has led to a scenario where each repository essentially
carries the same deployment code.


Aims:
- The primary aim of extracting the testing infrastructure into a single
repository is to reduce the cases where a simple change is needed, which
dominoes, causing a patch to each of 15 repositories. Instead, a change to
the uniform testing repository would resolve the issue for all other
repository's testing.
- In the future, we are looking to deploy scenario testing. For example, we
may want to test glance with a swift backend, or keystone with memcache. If
the testing play to deploy swift is already in a uniform repository, the
changes required to get glance testing enabled for that use case would be
minimal.
- This allows new projects to consume existing structure/playbooks to
deploy common components and vars, which should simplify the manner in
which new openstack-ansible projects set up their testing.


Steps taken so far:
- The deployment plays for each project have been split out into the
separate testing role.
- Each role only carries a specific "Test project" play.
- The test playbooks have been made generic so it uses the inventory
specified per repository (defining what hosts/roles there are).
- The test-vars have been put in the testing-repository and moved out of
the generic role.
- An override file has been created for each project and included using
"-e" (the highest precedence) but at present, of the 4 projects converted
the maximum number of overrides used is 2, so these overrides are minimal.
- Adjustments to tox.ini and var files references have been made to use the
new repository.


Further work to look into:
*- *It may be worth looking into making the tox.ini more generic, if we
were to make a sweeping change that impacts on tox.ini we would still need
to do changes to each repository. (I am not sure on how feasible this is
though)


Raised concerns:
- This creates a situation where a change may require me to make a change
to the central testing repository before changing the role repository. For
example, in order to get the generic testing for a keystone change I would
have to change the testing repository in advance, and then change the
keystone repository. Or override the var, adjust the testing repo and then
remove the keystone override.
- Changes to the testing-repo can cause all other repo tests (aside from
the overarching openstack-ansible repository) from breaking.


Where to find the PoC, what next?

The repository can be found here:
https://github.com/andymcc/openstack-ansible-testing-core

This is a proof of concept so in no way is it considered complete or
perfect, we are asking for more eyes on this; test it, let us know what you
like/do not like/want changed/additional ideas to improve.

Thanks!

Andy
irc: andymccr
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160603/08671a06/attachment.html>