Open Stack

On Tue, 28 Aug 2018, Matt Riedemann wrote:

>> Are people okay with that and willing to commit to being okay with
>> that answer in reviews? To some extent we need to have some faith on
>> the end result: the tests work. If people are not okay with that, we
>> need the people who are not to determine and prove the alternate
>> strategy. I've had this one work and work well.
>
> Seems reasonable to me. But to be clear, if there are 70 failed tests, are 
> you going to have 70 separate patches? Or this is just one of those things 
> where you start with 70, fix something, get down to 50 failed tests, and 
> iterate until you're down to all passing. If so, I'm OK with that. It's hard 
> to say without knowing how many patches get from 70 failures to 0 and what 
> the size/complexity of those changes is, but without knowing I'd default to 
> the incremental approach for ease of review.

It's lumpy. But at least at the begining it will be something like:
0 passing, stil 0 passing; still 0 passing; still 0 passing; 150
passing, 700 failing; 295 passing, X failing, etc. Because in the
early stages, test discovery and listing doesn't work at all, for
quite a few different reasons. Based on the discussion here,
resolving those "different reasons" is things people want to see in
different commits.

One way to optimize this (if people preferred) would be to not use
stestr as called by tox, with its built in test discovery, but
instead run testtools or subunit in a non-parallel and failfast
where not all tests need to be discovered first. That would provide a
more visible sense of "it's getting better" to someone who is running
the tests locally using that alternate method, but would not do much
for the jobs run by zuul, so probably not all that useful.

Thanks for the other info on the devstack and grenade stuff. If I
read you right, from your perspective it's a case of "we'll see" and
"we'll figure it out", which sounds good to me.

-- 
Chris Dent                       ٩◔̯◔۶           https://anticdent.org/
freenode: cdent                                         tw: @anticdent