On Fri, 2020-12-11 at 10:20 +0100, Bernard Cafarelli wrote:
Hi,
now that master branches have recovered from pip new resolver use, I started looking at stable branches status. tl;dr for those with open pending backports, all branches are broken at the moment so please do not recheck.
Thinking about fixing gates for these branches, older EM branches may be fine once the bandit 1.6.3 issue [1] is sorted out, but most need a fix against the new pip resolver.
pip has a flag to switch back to old resolver, but this is a temporary one that will only be there for a few weeks [2]
From a quick IRC chat, the general guidance for us was always to leave pip uncapped, and the new resolver issues are actually broken requirements.
But looking at master fixes, these indicate large and complicated changes on requirements and lower-contraints. Neutron fix [3] required a few major linter bumps and major version bumps in l-c. I guess it may be doable as victoria backport, but this will be messy for previous branches.
To make this effort slightly simpler, is there any reason we couldn't drag linters out of 'test-requirements.txt' across the board? Those seem to be the most problematic from what I've seen and they're generally not required to use the project nor to run tests. The exception to this rule is projects that have custom hacking plugins and tests for same, in which case I'm not yet sure what to do.
ovn-octavia-provider is a scarier example [4], from stable point of view the change by itself does not look good for backport, even just for victoria.
Also, in master, some fixes were possible by bumping versions on dependencies, but how to fix them if the max possible versions have broken deps themselves?
So, how do we proceed to fix stable gates? Ideas and feedback will be most welcome
This isn't a proper answer, but are there any circumstances where it would be possible to get a functioning deployment using the supposedly incorrect dependencies in lower-constraints.txt right now? For example, considering [4], would the deployment actually work with 'amqp==2.1.1' rather than 'amqp==5.0.2'? In fact, would pip < 20.3, in all its apparent brokenness, truly constrain amqp like this? I'm going to guess that in many cases it wouldn't be an issue, since these minimum dependencies were most likely selected arbitrarily, however, I also suspect there are cases where this would be an issue and we simply hadn't noticed. Assuming this to be the case, I think the question is more do we want to continue to rely on this known broken feature (by sticking to pip < 20.3) because it's "good enough" for these older branches, or do we want to spend our valuable time going through the dull but necessary work of fixing the dependencies? All of this assumes we find a way to work around dependencies that have broken dependencies themselves. That might well force our hand. Cheers, Stephen
[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019292.... [2] http://pyfound.blogspot.com/2020/11/pip-20-3-new-resolver.html [3] https://review.opendev.org/c/openstack/neutron/+/766000 [4] https://review.opendev.org/c/openstack/ovn-octavia-provider/+/765872/32/lowe...