Open Stack

Hi Daniel,

Thanks for pointing this up.

2015-11-25 1:40 GMT+09:00 Daniel Mellado <daniel.mellado.es at ieee.org>:
> Hi All,
>
> As you might already know, within Red Hat's tempest fork, we do have one
> tempest configuration script which was built in the past by David Kranz [1]
> and that's been actively used in our CI system. Regarding this topic, I'm
> aware that quite some effort has been done in the past [2] and I would like
> to complete the implementation of this blueprint/spec.
>
> My plan would be to have this script under the /tempest/cmd or
> /tempest/tools folder from tempest so it can be used to configure not the
> tempest gate but any cloud we'd like to run tempest against.
>
> Adding the configuration script was discussed briefly at the Mitaka summit
> in the QA Priorities meting [3]. I propose we use the existing etherpad to
> continue the discussion around and tracking of implementing "tempest
> config-create" using the downstream config script as a starting point. [4]
>
> If you have any questions, comments or opinion, please let me know.

This topic have happened several times, and I also felt this kind of
tool was very useful for Tempest users, because Tempest contains 296
options($ grep cfg * -R | grep Opt | wc -l) now and it is difficult to
set the configuration up.
However, there is a big concern:
If the script contain a bug and creates the configuration which makes
most tests skipped, we cannot do enough tests on the gate.
Tempest contains 1432 tests and difficult to detect which tests are
skipped as unexpected.
Actually we faced unexpected skipped tests on the gate before due to
some bug, then the problem has been fixed.
But I can imagine this kind of problem happens after implementing this
kind of script.

So now I am feeling Tempest users need to know what cloud they want to
test with Tempest, and need to know what tests run with Tempest.
Testers need to know what test target/items they are testing basically.

Thanks
Ken Ohmichi

---

> ---
> [1]
> https://github.com/redhat-openstack/tempest/blob/master/tools/config_tempest.py
> [2] https://blueprints.launchpad.net/tempest/+spec/tempest-config-generator
> [3] https://etherpad.openstack.org/p/mitaka-qa-priorities
> [4] https://etherpad.openstack.org/p/tempest-cli-improvements
>
> https://github.com/openstack/qa-specs/blob/master/specs/tempest/tempest-cli-improvements.rst
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>