[openstack-dev] [QA][Tempest]Run only multinode tests in multinode jobs
cboylan at sapwetik.org
Tue Feb 16 19:24:34 UTC 2016
On Tue, Feb 16, 2016, at 10:42 AM, Assaf Muller wrote:
> On Tue, Feb 16, 2016 at 12:26 PM, Clark Boylan <cboylan at sapwetik.org>
> > On Tue, Feb 16, 2016, at 01:07 AM, Jordan Pittier wrote:
> >> Hi list,
> >> I understood we need to limit the number of tests and jobs that are run
> >> for
> >> each Tempest patch because our resources are not unlimited.
> >> In Tempest, we have 5 multinode experimental jobs:
> >> experimental-tempest-dsvm-multinode-full-dibtest
> >> gate-tempest-dsvm-multinode-full
> >> gate-tempest-dsvm-multinode-live-migration
> >> gate-tempest-dsvm-neutron-multinode-full
> >> gate-tempest-dsvm-neutron-dvr-multinode-full
> >> These jobs largely overlap with the non-multinode jobs. What about
> >> tagging
> >> (with a python decorator) each test that really requires multiple nodes
> >> and
> >> only run those tests as part of the multinode jobs ?
> > One of the goals I had was to hopefully replace the single node jobs
> > with the multinode jobs because as you point out there is a lot of
> > redundancy and 2 VMs < 3 VMs. One of the prerequisites for this to
> > happen is to have an easy way to reproduce the multinode test envs using
> > something like vagrant. I have been meaning to work on that this cycle
> > but adding new cloud resources (and keeping existing resources happy)
> > have taken priority.
> These are not conflicting efforts, are they? We could attack it on
> both fronts: Send a patch that tags a dozen (?) or so tests with
> 'multinode' run only those in the multinode jobs. You could accomplish
> that almost immediately. In parallel work on replacing the single node
> jobs with multinode (And then change their tests regex from
> 'multinode' back to full).
If you do this then you no longer know if multinode works completely so
you can regress while you only run some subset of tests. If that happens
(and history has shown the likelyhood is high) then you have to debug
multinode all over again to get it working.
More information about the OpenStack-dev