Open Stack

Hi folks!

I'd like to propose a plan on how to simultaneously extend the coverage of our 
jobs and reduce their number.

Currently, we're running one instance per job. This was reasonable when the 
coreos-based IPA image was the default, but now with tinyipa we can run up to 7 
instances (and actually do it in the grenade job). I suggest we use 6 fake bm 
nodes to make a single CI job cover many scenarios.

The jobs will be grouped based on driver (pxe_ipmitool and agent_ipmitool) to be 
more in sync with how 3rd party CI does it. A special configuration option will 
be used to enable multi-instance testing to avoid breaking 3rd party CI systems 
that are not ready for it.

To ensure coverage, we'll only leave a required number of nodes "available", and 
deploy all instances in parallel.

In the end, we'll have these jobs on ironic:
gate-tempest-ironic-pxe_ipmitool-tinyipa
gate-tempest-ironic-agent_ipmitool-tinyipa

Each job will cover the following scenarious:
* partition images:
** with local boot:
** 1. msdos partition table and BIOS boot
** 2. GPT partition table and BIOS boot
** 3. GPT partition table and UEFI boot  <*>
** with netboot:
** 4. msdos partition table and BIOS boot <**>
* whole disk images:
* 5. with msdos partition table embedded and BIOS boot
* 6. with GPT partition table embedded and UEFI boot  <*>

  <*> - in the future, when we figure our UEFI testing
  <**> - we're moving away from defaulting to netboot, hence only one scenario

I suggest creating the jobs for Newton and Ocata, and starting with Xenial right 
away.

Any comments, ideas and suggestions are welcome.