Open Stack

On Mon, Apr 17, 2017 at 7:58 PM Ihar Hrachyshka <ihrachys at redhat.com> wrote:

> On Mon, Apr 17, 2017 at 9:35 AM, Jordan Pittier
> <jordan.pittier at scality.com> wrote:
> > We don"t run slow tests because the QA team think that they don't bring
> > enough value to be executed, every time and everywhere. The idea is that
> if
> > some specific slow tests are of some interest to some specific openstack
> > projects, those projects can change the config of their jobs to enable
> these
> > tests.
>
>
> But since it's not executed anywhere in tempest gate, even as
> non-voting (?), it's effectively dead code that may be long broken
> without anyone knowing. Of course there are consumers of the tests
> downstream, but for those consumers it's a tough call to start
> depending on the tests if they are not sanity checked by tempest
> itself. Wouldn't it make sense to have some job in tempest gate that
> would execute those tests (maybe just them to speed up such a job?
> maybe non-voting?


We reduced the number of scenario tests we run in the main gate because
some of those tests are not so
critical for the integrated gate, and also because we had an unstable gate
we had to deal with to allow for
Pike development to happen.

The SUT was overloaded in the gate, due to a combination of effects: too
many services running, many
consuming more memory that it did in the past, as well as too many "heavy"
tests running in parallel.

As soon as removed scenario tests from the main gate we introduced a new
non-voting job against
Tempest, which runs *all* scenario tests against a multimode environment,
see for instance [0], so
we can ensure the code is not dead as long as it lives in Tempest.

I'm planning to add more "heavy/long" tests to that jobs, e.g. compute
migrate ones [1].

andreaf

[0]
http://logs.openstack.org/69/451769/2/check/gate-tempest-dsvm-neutron-scenario-multinode-ubuntu-xenial-nv/1f71416/

[1] https://review.openstack.org/#/c/451769/


> maybe even as periodic? but there should be
> something that keeps it green in long run).
>
> Ihar
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170417/e741f258/attachment.html>