Open Stack

I like your idea of adding metadata to test cases that we could generate 
this later.

One of the reasons I was writing all the "Tempest Guide to .... " files 
as README.rst was I'm planning on creating a docs job once we are done 
that will rebuild qa.openstack.org to be a collection of those READMEs. 
Going from rst -> html should be pretty easy, and will mean that changes 
in the code will be the website.

I think something equivalent like this could really be useful as well 
and part of that publishing stream.

Now it just needs a champion to dive in and make it happen! :)

	-Sean

On 05/17/2013 07:01 AM, Attila Fazekas wrote:
> I like the ACC model, but it looks like too high level overview for starting,
>   but it can be a good continuation.
>
> A simple model for mainly API testing can be good for now.
>
> Initial considerations:
> * We have services or components and and sub-part of this at API level.
>    (3D matrix mapped to 2D)
> * We have operations to test
> * We have things to verify
> * I would like to know is something missing or not covered properly
>
> I have constructed a matrix, than I simplified it.
>
> Something like below as a colored HTML table seams OK.
>
> ------------------------------------------------------------------------
> | component    | CRUD | list | associations | quota | security |  misc |
> ------------------------------------------------------------------------
> | nova-server  |      |      |              |       |          |       |
> ------------------------------------------------------------------------
> | swift-object |      |      |              |       |          |       |
> ------------------------------------------------------------------------
> | keystone-user|      |      |              |       |          |       |
> ------------------------------------------------------------------------
> | misc         |      |      |              |       |          |       |
> ------------------------------------------------------------------------
>
> CRUD: create, read, update, get
> list: list, list detailed, filtering, pagination
> misc: action , bulk-action
> security: tenant-isolates, admin-denied
> association: member, attach, assign
>
>
> * In every intersection we should have multiple links to the source
>    (github, automatically generated link).
> * One test case can be in multiple intersection
> * every line and row should have a short description
> * every intersection should have a short description
> * based on the description anyone can color the intersections
>    1. Black: Does not makes sense in this context
>    2. Red: completely missing
>    3. yellow: partially completed
>    4. light green: good progress
>    5. green: OK
>    6. white: unknown
>
> We could have tables for different API versions or CLI ..
>
> In theory if we add enough meta-data (attribute) on test cases to be map able
>   into this kind of table. The only paper work will be:
>
> * create short description about what we need to verify
>   (likely the columns will tell it alone)
> * Color the table
>
>
> Possible addition:
> * Links to additional resources like blueprints, bugs..

-- 
Sean Dague
http://dague.net