Open Stack

Nice work Andy. I'm strongly in favor of getting the central test repo into Gerrit for broader reviews and once done, getting one or two patches up with roles making use of it.

- Travis Truman

From: Andy McCrae <andy.mccrae at gmail.com<mailto:andy.mccrae at gmail.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Friday, June 3, 2016 at 11:41 AM
To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: [openstack-dev] [OpenStack-Ansible] Splitting out generic testing components

TL;DR We're doing a PoC to split out common testing plays/vars into a generic repository, for the purposes of making in-role testing more uniform and generic. Looking for feedback, comments, informal reviews, ideas etc! https://github.com/andymcc/openstack-ansible-testing-core (Includes a simple README)


We now have a lot of duplication after moving to a single role per repository structure with specific testing per role. For example, almost all repositories require Galera/Rabbit/Keystone in order to deploy testing successfully. This has led to a scenario where each repository essentially carries the same deployment code.


Aims:
- The primary aim of extracting the testing infrastructure into a single repository is to reduce the cases where a simple change is needed, which dominoes, causing a patch to each of 15 repositories. Instead, a change to the uniform testing repository would resolve the issue for all other repository's testing.
- In the future, we are looking to deploy scenario testing. For example, we may want to test glance with a swift backend, or keystone with memcache. If the testing play to deploy swift is already in a uniform repository, the changes required to get glance testing enabled for that use case would be minimal.
- This allows new projects to consume existing structure/playbooks to deploy common components and vars, which should simplify the manner in which new openstack-ansible projects set up their testing.


Steps taken so far:
- The deployment plays for each project have been split out into the separate testing role.
- Each role only carries a specific "Test project" play.
- The test playbooks have been made generic so it uses the inventory specified per repository (defining what hosts/roles there are).
- The test-vars have been put in the testing-repository and moved out of the generic role.
- An override file has been created for each project and included using "-e" (the highest precedence) but at present, of the 4 projects converted the maximum number of overrides used is 2, so these overrides are minimal.
- Adjustments to tox.ini and var files references have been made to use the new repository.


Further work to look into:
- It may be worth looking into making the tox.ini more generic, if we were to make a sweeping change that impacts on tox.ini we would still need to do changes to each repository. (I am not sure on how feasible this is though)


Raised concerns:
- This creates a situation where a change may require me to make a change to the central testing repository before changing the role repository. For example, in order to get the generic testing for a keystone change I would have to change the testing repository in advance, and then change the keystone repository. Or override the var, adjust the testing repo and then remove the keystone override.
- Changes to the testing-repo can cause all other repo tests (aside from the overarching openstack-ansible repository) from breaking.


Where to find the PoC, what next?

The repository can be found here: https://github.com/andymcc/openstack-ansible-testing-core

This is a proof of concept so in no way is it considered complete or perfect, we are asking for more eyes on this; test it, let us know what you like/do not like/want changed/additional ideas to improve.

Thanks!

Andy
irc: andymccr
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160708/a2b6debf/attachment.html>