[openstack-dev] [qa] moratorium on new negative tests in Tempest

Clint Byrum clint at fewbar.com
Wed Nov 13 02:40:56 UTC 2013

Excerpts from Clint Byrum's message of 2013-11-12 10:36:31 -0800:
> Excerpts from Sean Dague's message of 2013-11-12 10:01:06 -0800:
> > During the freeze phase of Havana we got a ton of new contributors
> > coming on board to Tempest, which was super cool. However it meant we
> > had this new influx of negative tests (i.e. tests which push invalid
> > parameters looking for error codes) which made us realize that human
> > creation and review of negative tests really doesn't scale. David Kranz
> > is working on a generative model for this now.
> > 
> Are there some notes or other source material we can follow to understand
> this line of thinking? I don't agree or disagree with it, as I don't
> really understand, so it would be helpful to have the problems enumerated
> and the solution hypothesis stated. Thanks!

Thanks to all for explaining the motivations fully. I understand now, and
agree with all of the points made (including how incredibly frustrating
it is to come up with all of the conceptual ways to get errors back).

More information about the OpenStack-dev mailing list