Open Stack

On 01/12/2015 05:00 PM, Boris Pavlovic wrote:
> Hi Chris, 
> 
>     If there's sufficient motivation and time it might make sense to
>     separate the part of gabbi that builds TestCases from the part that
>     runs (and evaluates) HTTP requests and responses. If that happens then
>     integration with tools like Rally and runners is probably possible.
> 
> 
> 
> Having separated engine seems like a good idea. It will really simplify
> stuff 
> 
> 
>     So, while this is an interesting idea, it's not something that gabbi
>     intends to be. It doesn't validate existing clouds. It validates code
>     that is used to run clouds.
>     Such a thing is probably possible (especially given the fact that you
>     can give a "real" host to gabbi tests) but that's not the primary
>     goal.
> 
> 
> 
> This seems like a huge duplication of efforts. I mean operators will
> write own 
> tools developers own... Why not just resolve more common problem: 
> "Does it work or not?"

I think it's important to look at this in the narrower context, we're
not testing full environments here, this is custom crafting HTTP req /
resp in a limited context to make sure components are completing a contract.

"Does it work or not?" is so broad a statement as to be meaningless most
of the time. It's important to be able to looking at these lower level
response flows and make sure they both function, and that when they
break, they do so in a debuggable way.

So I'd say let's focus on that problem right now, and get some traction
on this as part of functional test suites in OpenStack. Genericizing it
too much just turns this back into a version of every other full stack
testing tool, which we know isn't sufficient for having quality
components in OpenStack.

	-Sean

-- 
Sean Dague
http://dague.net