Open Stack

Hi QA-team,

Thanks for joining QA sessions on OpenStack Summit Barcelona.
They were interesting and good to get directions to move forward in
this development cycle.
This is a summary of these sessions for next steps and hope this helps
our works.

* (Tempest) Add an option for stopping cleanup when test failure happens
  Main assignee / organizer: dpaterson, mkopec
  Milestone: O-3
  Description:
    As the design principle, Tempest should clean all created resource
up when finishing.
    However the cleanup makes difficult to debug problems sometimes
because the failure situation is deleted.
    Nice to add an option into tempest.conf or 'tempest run' command
for disabling the cleanup.

* (Tempest) Add an option for the number of target VMs to test live-migration
  Main assignee / organizer: gmann
  Milestone: O-2
  Description:
    On production clouds, it is common to migrate multiple virtual
machines to the other host.
    Current Tempest migrates a single virtual machine in each test.
    Nice to add an option to control the number of target machines.

* (Tempest) Add and decorator into bug-reproducing tests to know
actual bug number from test failure
  Main assignee / organizer: oomichi, dmellado
  Milestone: O-2
  Description:
    When fixing a bug on each project, it is nice to propose Tempest
test to reproduce the bug on the gate.
    Such Tempest tests can help detecting latent bugs on production
clouds which are deployed with older OpenStack versions.
    By knowing the LP bug number from the test, testers can know which
patch is necessary to be applied to thier own clouds according to the
LP report.
    Now they can know it from Tempest git history(Related-Bug tag),
but that is a little hard.
    A new test decorator will help to know that easily.

* (Tempest) Reduce deep test class inheritance for easy debugging
  Main assignee / organizer: ekhugen, dmellado, jhakimra andreaf
  Milestone: O-3
  Description:
    We are still seeing deep backtrace when some Tempest tests fail.
    That makes hard to debug problems because testers need to read
many test modules.
    At first, we need to know how deep on current test inheritances
and define the target depth for reducing.
    So some tool is necessary to know current test inheritances as first step.

* (Tempest) Bug Triage
  Main assignee / organizer: masayukig, gmann, jhakimra, luzC,
ababich, dmellado
  Milestone: End of Ocata
  Description:
    The bug report number of Tempest continues increasing and we need
bug triage.
    In this Ocata cycle, many people raise hands for this bug triage.
Thanks so much.
    We will do that in weekly rotation and report the progress in
weekly meetings.
    https://etherpad.openstack.org/p/ocata-qa-bug-triage is for
managing assignees.

* OpenStack Health
  Main assignee / organizer: masayukig
  Milestone: O-2
  Description:
    Submit ideas to launchpad from the session feedback and prioritize them.
    The feedback was
    - Unit test coverage of each project (Nova, Cinder, etc)
    - Test failure ratio ranking by test

* Destructive testing
  Main assignee / organizer: Timur Nurlygayanov
  Milestone: O-2 (qa-spec at least)
  Description:
    To clarify the scope, user story and test scenario, qa-spec is
necessary to be proposed.
    On the implementation side, it is better to avoid separated repos
of os-faults and stepler for its maintenance.

* Policy testing
  Main assignee / organizer:
  Milestone: O-2 (qa-spec at least)
  Description:
    This test will be implemented with tempest-plugin from separated
repo which is different from Tempest.
    The qa-spec is already proposed as https://review.openstack.org/#/c/382672/

If having questions, please send mails to me or "Main assignee / organizer".
Thanks for your help.

Reference:
* Ocata Priorities: https://etherpad.openstack.org/p/ocata-qa-priorities
* Etherpads of QA:
https://wiki.openstack.org/wiki/Design_Summit/Ocata/Etherpads#QA_.28Quality_Assurance.29

Thanks
Ken Omichi