Open Stack

Yes, we have Tempest and unit tests and Functional tests.  But still we 
need test plans....

Keystone often plays a role deep in others work flows. Often we have 
complicated features that span multiple services;  WebSSO, Trusts and 
HEAT,  EC2 Credentials...

Before we can automate the tests, we need to know what to test.  And 
that is a conversation best started by the developers implementing the 
feature, and then handed off to the engineer that needs to make sure the 
feature actually works, positive, negative, and edge.

I'm sure that we have many organizations with internal QA that can 
benefit from test plans that show how to exercise and test these 
features.  As part of getting comfortable working up stream, my team has 
written the start of a few test plans.  I'd like to share them with the 
larger Keystone consuming community;


https://etherpad.openstack.org/p/ldap-backend-non-default-domain
https://etherpad.openstack.org/p/hierarchical-projects
https://etherpad.openstack.org/p/keystone-token-scoping

As you can guess, my goal is not purely altruistic:  I'd like to get 
other people to both input into these test cases and to write some for 
other aspects of Keystone.  Other ones we've identified:

CADF Notifications Everywhere
Allow Redelegation via Trusts
mapping to existing user in federated workflow

I think Etherpad is the right medium for this.  I was originally 
thinking the Spec process, but approving test plans seems to 
centralized:  we want multiple engineers contributing to the test 
plans.  They should be collaborative documents as people add suggestions 
of what to test.

Eventually, these should be automated, but before we can automate, we 
need to identify what to test.  Getting just that down is incredibly 
valuable. We can be more rigorous on the review process when we automate.

I am actively seeking suggestions.  How should we best organize this effort?