Open Stack

On Wed, Sep 22, 2021, at 6:15 AM, Jeremy Stanley wrote:
> On 2021-09-22 14:56:43 +0200 (+0200), Előd Illés wrote:
> [...]
>> Option 3: it is really just a temporary solution (maybe worse than
>> Option 1), though it is probably acceptable to unblock gate on
>> stable branches until the appropriate solution will be chosen.
>> Also worth to emphasize that this problem doesn't only affect
>> lower-constraints tests, but also could affect branches where we
>> have old packages (versions) in upper-constraints.txt!
> [...]
>
> Do also note that all solutions in this space are effectively
> temporary. When we drafted the stable branch and extended
> maintenance policies, we accepted that there comes a time for any
> job where circumstances may prevent us from being able to
> effectively continue running it. If the complexity of the solution
> outweighs the *temporary* benefits of being able to run the job for
> longer, it may not be worth the effort.
>
> While we almost certainly could come up with a non-default
> replacement for Zuul's standard tox job which allows us to use
> specific versions of tox, virtualenv, pip, setuptools, et cetera,
> that added complexity does not come for free. It has a maintenance
> cost which extends far into the future. Also think back on the times
> when we've reached for the "easy" solution of pinning such tools in
> other frameworks like DevStack: everyone's in a rush to stop the
> bleeding and get back to testing, but nobody cares about fixing the
> underlying issue and so such things end up pinned for years until
> that pinning breaks something else because we're blind to all the
> bitrot around us.
>
> And then there's the fact that while we can "fix" things this way in
> CI jobs, these are ultimately tools for our developers which we just
> happen to also be able to run in the CI system, not the other way
> around. Pinning setuptools in the tox jobs does not solve this for
> developers running tox locally on their own systems. Are we going to
> start providing them with a list of specific versions of these tools
> which we're only able to test with, substantially increasing the
> complexity of setting up local dev environments and making it even
> harder for newcomers to contribute to the project?

Related to this is the trap that we can often see passing CI as the goal. In reality the goal should be building quality reliable software that functions properly. The testing and CI gives us a measurement against that goal. We do pre merge testing so that we can avoid merging code that does not function. Occasionally external factors break functionality and in those cases the CI and testing have done their job. We are alerted to non functioning software that needs to be fixed.

In this case we need to be careful to avoid fixing this only in a CI context because we see passing CI as the goal. Instead we should try to fix the software so that it is functional for the CI system, developers on their local machines, and the individuals/orgs that ultimately deploy the software into production.

>
> The stable branch policy is a set of guidelines and suggestions, not
> an unbreakable covenant. It's there to help us make sound choices
> under normal circumstances, but we should not be afraid to come to a
> collective decision about making policy exceptions in unusual
> situations such as this one.
> -- 
> Jeremy Stanley