Open Stack

On 08/09/2016 11:25 AM, Matthew Thode wrote:
> On 08/09/2016 10:22 AM, Ian Cordasco wrote:
>> -----Original Message-----
>> From: Matthew Thode <prometheanfire at gentoo.org>
>> Reply: prometheanfire at gentoo.org <prometheanfire at gentoo.org>, OpenStack Development Mailing List (not for usage questions) <openstack-dev at lists.openstack.org>
>> Date: August 9, 2016 at 09:53:53
>> To: openstack-dev at lists.openstack.org <openstack-dev at lists.openstack.org>
>> Subject:  Re: [openstack-dev] [requirements] History lesson please
>>
>>> One of the things on our todo list is to test the 'lower-constraints' to
>>> make sure they still work with the head of branch.
>>
>> That's not sufficient. You need to find versions in between the lowest tested version and the current version to also make sure you don't end up with breakage. You might have somepackage that has a lower version of 2.0.1 and a current constraint of 2.12.3. You might even have a blacklist of versions in between those two versions, but you still need other versions to ensure that things in between those continue to work.
>>
>> THe tiniest of accidental incompatibilities can cause some of the most bizarre bugs.
>>
>> --  
>> Ian Cordasco
>>
> 
> I'm aware of this, but this would be a good start.

And, more importantly, assuming that testing is only valid if it covers
every scenario, sets the bar at entirely the wrong place.

A lower bound test would eliminate some of the worst fiction we've got.
Testing is never 100%. With a complex system like OpenStack, it's
probably not even 1% (of configs matrix for sure). But picking some
interesting representative scenarios and seeing that it's not completely
busted is worth while.

	-Sean

-- 
Sean Dague
http://dague.net