On Fri, Oct 4, 2024, at 9:30 AM, Takashi Kajinami wrote:
Hello,
As you may know, current master runs unit tests with only Python 3.9 and Python 3.12. My understanding about this set up is that we test the minimum boundary and the maximum boundary and assume the change may work with all the versions between these two.
However in some of my recent work I had to introduce the switch in the middle of these two versions. For example https://review.opendev.org/c/openstack/neutron/+/931271 introduces the logic to distinguish Python < 3.10 from Python >= 3.10 , and we don't specifically test that boundary. I'm afraid that this may potentially cause uncaught problems.
One approach that wasn't discussed in the change is to continue to use the existing code (pkg_resources) until python3.12 then only change the behavior for 3.12. That approach would generally work if we are only testing the boundaries since we're flipping the behavior on the boundary.
So my question here is ... Can we test all supported python versions, not only min/max, at least in check jobs ? This would helps us catch any regressions caused by such implementation.
We can. The main limitation is that we may not have a test platform for every version of python depending on how the distro releases align with python releases. Currently I think we have a platform for every release though (centos/rocky 3.9, jammy 3.10, bookworm 3.11, noble 3.12). If we end up without a platform for a version of python jobs may need to be updated to install python from source or some other method.
I know there could be tradeoffs between test coverage and infra resources, but IMO this is the test coverage we should fill.
Yes, the main concern was simply to avoid unnecessary effort. In the case of unittest jobs the vast majority of those are much cheaper in terms of resources than tempest jobs. I think if we stuck to this primarily for unittests the impact would be minimal. If we tried to expand the tempest matrix that would become unwieldy.
Thank you,
-- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit