Open Stack

On 10/30/2014 10:47 AM, Eoghan Glynn wrote:
> 
>> Matthew wrote:
>>
>> This would have the advantage of making tempest slimmer for every project
>> and begin the process of getting projects to take responsibility for their
>> functional testing rather than relying on tempest.
> 
> [much snipping]
> 
>> Sean wrote:
>>
>> Ok, so part of this remains to be seen about what the biggest bang for the
>> buck is. The class of bugs I feel like we need to nail in Nova right now are
>> going to require tests that bring up pieces of the wsgi stack, but are
>> probably not runable on a real deploy. Again, this is about debugability.
> 
> So this notion of the biggest bang for our buck is an aspect of the drive
> for in-tree functional tests, that's not entirely clear to me as yet.
> 
> i.e. whether individual projects should be prioritizing within this effort:
> 
> (a) the creation of net-new coverage for scenarios (especially known or
>     suspected bugs) that were not previously tested, in a non-unit sense
> 
> (b) the relocation of existing integration test coverage from Tempest to
>     the project trees, in order to make the management of Tempest more
>     tractable
> 
> It feels like there may be a tension between (a) and (b) in terms of the
> pay-off for this effort. I'd interested in hearing other opinions on this,
> on what aspect projects are expecting (and expected) to concentrate on
> initially.

For what it's worth I have a bunch of early targets listed for Nova for
our summit session -
https://etherpad.openstack.org/p/kilo-nova-functional-testing

My focus in kilo is going to be first about A), as that provides value
out of the gate (pun intended). Then peel off some stuff from B as makes
sense.

	-Sean

-- 
Sean Dague
http://dague.net