[openstack-qa] Tempest Coverage Analysis and Gaps in Compute Tests

Matthew Treinish treinish at linux.vnet.ibm.com
Wed Jan 30 16:57:03 UTC 2013


On Wed, Jan 30, 2013 at 09:21:01AM +0200, Yaniv Kaul wrote:
> On 30/01/13 00:54, Matthew Treinish wrote:
> >Hi Everyone,
> >
> >I've started working on an analysis of the coverage reports generated by the
> >periodic Jenkins jobs:
> >
> >https://jenkins.openstack.org/job/periodic-tempest-devstack-coverage-vm-full/
> >
> >I wanted to see how good a job tempest is doing for covering the Nova API code.
> >Right now we've got some gaps in what is being tested in a tempest full gate
> >run. I've started putting up a list of what functionality is missing from the
> >tempest runs here:
> >
> >https://etherpad.openstack.org/coverage-analysis
> >
> >The periodic jobs uses the same configuration as our current gate jobs, so
> >there are couple cases where we having testing but they depend on things not
> >set in tempest.conf (Resize action for example)
> >
> >Then based on the results of the coverage report I put together a list of
> >proposed additional tests here:
> >
> >https://etherpad.openstack.org/MissingTempestTests
> 
> Will you be converting some of the bigger and/or contained tasks to
> blueprints for Tempest?
> Y.
I think that for right now I'm just going to keep it in the etherpad. Since
it's going to be a pretty dynamic list that will be growing and getting multiple
contributors, an etherpad is really just more conducive to that kind of list.
I think that once the list becomes more static I'll push whatever work is left
over into a bp.

Thanks,

Matt Treinish

> 
> >
> >Keep in mind that both of these etherpads are still WIP (and quite incomplete)
> >and I'll be continually updating them over the next few days until I go through
> >the rest of the coverage reports. I just wanted to share what I'm working on
> >and some of the initial findings. Feel free to ping me on IRC for specifics or
> >questions about any of this.
> >
> >Thanks,
> >
> >Matt Treinish
> >
> >
> >_______________________________________________
> >openstack-qa mailing list
> >openstack-qa at lists.openstack.org
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
> 




More information about the openstack-qa mailing list