Open Stack

On 01/12/2015 03:18 PM, Boris Pavlovic wrote:
> Chris, 
> 
> The Idea is brilliant. I may steal it! =)
> 
> But there are some issues that will be faced: 
> 
> 1) Using as a base unittest: 
> 
>     python -m subunit.run discover -f gabbi | subunit2pyunit
> 
> 
> So rally team won't be able to reuse it for load testing (if we directly
> integrate it) because we will have huge overhead (of discover stuff)
> 
> 2) Load testing. 
> 
> Using unittest for functional testing adds a lot of troubles: 
> 2.1) It makes things complicated: 
> Like reusing fixtures via input YAML will be painfull
> 2.2) It adds a lot of functionality that is not required 
> 2.3) It makes it hardly integratabtle with other tools. Like Rally.. 
> 
> 3) Usage by Operators is hard in case of N projects. 
> 
> So you should have some kind of 
> 
> Operators would like to have 1 button that will say (does cloud work or
> not). And they don't want to combine all gabbi files from all projects
> and run test. 
> 
> From other side there should be a way to write such code
> in-projects-tree (so new features are directly tested) and then moved to
> some common place that is run on every patch (without breaking gates) 
> 
> 4) Using subunit format is not good for functional testing.
>  
> It doesn't allow you to collect detailed information about execution of
> test. Like for benchmarking it will be quite interesting to collect
> durations of every API call. 

I'm not sure how subunit causes an issue here either way. You can either
put content into one of the existing subunit attachments, or could
modify it to have a new one.

	-Sean

-- 
Sean Dague
http://dague.net