[openstack-dev] [requirements] History lesson please
sigmavirus24 at gmail.com
Tue Aug 9 16:24:26 UTC 2016
From: Sean Dague <sean at dague.net>
Reply: OpenStack Development Mailing List (not for usage questions) <openstack-dev at lists.openstack.org>
Date: August 9, 2016 at 11:21:47
To: openstack-dev at lists.openstack.org <openstack-dev at lists.openstack.org>
Subject: Re: [openstack-dev] [requirements] History lesson please
> On 08/09/2016 11:25 AM, Matthew Thode wrote:
> > On 08/09/2016 10:22 AM, Ian Cordasco wrote:
> >> -----Original Message-----
> >> From: Matthew Thode
> >> Reply: prometheanfire at gentoo.org , OpenStack Development
> Mailing List (not for usage questions)
> >> Date: August 9, 2016 at 09:53:53
> >> To: openstack-dev at lists.openstack.org
> >> Subject: Re: [openstack-dev] [requirements] History lesson please
> >>> One of the things on our todo list is to test the 'lower-constraints' to
> >>> make sure they still work with the head of branch.
> >> That's not sufficient. You need to find versions in between the lowest tested version
> and the current version to also make sure you don't end up with breakage. You might have
> somepackage that has a lower version of 2.0.1 and a current constraint of 2.12.3. You
> might even have a blacklist of versions in between those two versions, but you still need
> other versions to ensure that things in between those continue to work.
> >> THe tiniest of accidental incompatibilities can cause some of the most bizarre bugs.
> >> --
> >> Ian Cordasco
> > I'm aware of this, but this would be a good start.
> And, more importantly, assuming that testing is only valid if it covers
> every scenario, sets the bar at entirely the wrong place.
> A lower bound test would eliminate some of the worst fiction we've got.
> Testing is never 100%. With a complex system like OpenStack, it's
> probably not even 1% (of configs matrix for sure). But picking some
> interesting representative scenarios and seeing that it's not completely
> busted is worth while.
Right. I'm not advocating for testing every released version of a dependency. In general, it's good to test versions that have *triggered* changes though. If upgrading from 2.3.0 to 2.4.1 caused you to need to fix something, test something earlier than 2.4.1, and 2.4.1, and then something later. That's what I'm advocating for.
More information about the OpenStack-dev