Re: [openstack-dev] [python3] Enabling py37 unit tests
So, at this point, is it OK to have projects running against both py35 and py37 and considering py36 covered as being included in the interval?
Also about the lowest supported version, I think that is the one that should be stated in the envlist of tox.ini to fail fast during development.
Em ter, 13 de nov de 2018 às 19:32, Corey Bryant corey.bryant@canonical.com escreveu:
On Wed, Nov 7, 2018 at 11:12 AM Clark Boylan cboylan@sapwetik.org wrote:
On Wed, Nov 7, 2018, at 4:47 AM, Mohammed Naser wrote:
On Wed, Nov 7, 2018 at 1:37 PM Doug Hellmann doug@doughellmann.com
wrote:
Corey Bryant corey.bryant@canonical.com writes:
On Wed, Oct 10, 2018 at 8:45 AM Corey Bryant <
corey.bryant@canonical.com>
wrote:
I'd like to start moving forward with enabling py37 unit tests for
a subset
of projects. Rather than putting too much load on infra by enabling
3 x py3
unit tests for every project, this would just focus on enablement
of py37
unit tests for a subset of projects in the Stein cycle. And just to
be
clear, I would not be disabling any unit tests (such as py35). I'd
just be
enabling py37 unit tests.
As some background, this ML thread originally led to updating the python3-first governance goal (
https://review.openstack.org/#/c/610708/)
but has now led back to this ML thread for a +1 rather than
updating the
governance goal.
I'd like to get an official +1 here on the ML from parties such as
the TC
and infra in particular but anyone else's input would be welcomed
too.
Obviously individual projects would have the right to reject
proposed
changes that enable py37 unit tests. Hopefully they wouldn't, of
course,
but they could individually vote that way.
Thanks, Corey
This seems like a good way to start. It lets us make incremental progress while we take the time to think about the python version management question more broadly. We can come back to the other
projects
to add 3.7 jobs and remove 3.5 jobs when we have that plan worked out.
What's the impact on the number of consumption in upstream CI node
usage?
For period from 2018-10-25 15:16:32,079 to 2018-11-07 15:59:04,994, openstack-tox-py35 jobs in aggregate represent 0.73% of our total capacity usage.
I don't expect py37 to significantly deviate from that. Again the major resource consumption is dominated by a small number of projects/repos/jobs. Generally testing outside of that bubble doesn't represent a significant resource cost.
I see no problem with adding python 3.7 unit testing from an infrastructure perspective.
Clark
Thanks all for the input on this. It seems like we have no objections to moving forward so I'll plan on getting started soon.
Thanks, Corey __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
On Tue, Feb 26, 2019 at 05:28:19PM +0100, Moises Guimaraes de Medeiros wrote:
So, at this point, is it OK to have projects running against both py35 and py37 and considering py36 covered as being included in the interval?
Also about the lowest supported version, I think that is the one that should be stated in the envlist of tox.ini to fail fast during development.
In my opinion, the py35 jobs should all be dropped.
The official runtime for Stein is py36, and the upcoming runtime is py37, so it doesn't add much value to be running py35 tests at this point.
Sean
On 26/02/19 11:37 AM, Sean McGinnis wrote:
On Tue, Feb 26, 2019 at 05:28:19PM +0100, Moises Guimaraes de Medeiros wrote:
So, at this point, is it OK to have projects running against both py35 and py37 and considering py36 covered as being included in the interval?
According to the resolution we should unit test "[e]ach Python 3 version that is the default in any of the Linux distributions specifically identified in the Project Testing Interface at the beginning of the development cycle."
https://governance.openstack.org/tc/resolutions/20181024-python-update-proce...
Python 3.6 is the default in Ubuntu Bionic and openSUSE Leap 15.0, so we should keep py36 jobs running.
(Unit test jobs are not particularly expensive, so we shouldn't worry too much about running 4 of them.)
Also about the lowest supported version, I think that is the one that should be stated in the envlist of tox.ini to fail fast during development.
In my opinion, the py35 jobs should all be dropped.
I'm all for getting rid of py35 ASAP, but a prerequisite for that would be that we move all integration test jobs to Bionic (from Xenial). Apparently gmann has a plan for that to happen in Stein, but we haven't set it as a goal or anything. If that ended up working out we could drop it in Train though (the resolution says we should unit test "[e]ach Python 3 version that was still used in any integration tests at the beginning of the development cycle").
The official runtime for Stein is py36, and the upcoming runtime is py37, so it doesn't add much value to be running py35 tests at this point.
It prevents a project from breaking another project's Xenial-based integration tests by committing some trivial non-py35-compatible code (e.g. f-strings).
cheers, Zane.
On 2019-02-27 11:23:19 -0500 (-0500), Zane Bitter wrote: [...]
the resolution says we should unit test "[e]ach Python 3 version that was still used in any integration tests at the beginning of the development cycle"
[...]
Now I'm getting worried that the phrasing we settled on is leading to misinterpretation. The entire point, I thought, was that we decide at the beginning of the development cycle on which platforms we're testing, and so choose the most recent releases of those LTS distros. If some projects were still running jobs on an earlier platform at the start of the cycle, I don't think we need to be stuck maintaining that testing. The beginning of the cycle is the point at which it's safe for them to switch to the agreed-upon current platform for our upcoming release under development.
On 2/27/19 10:36 AM, Jeremy Stanley wrote:
On 2019-02-27 11:23:19 -0500 (-0500), Zane Bitter wrote: [...]
the resolution says we should unit test "[e]ach Python 3 version that was still used in any integration tests at the beginning of the development cycle"
[...]
Now I'm getting worried that the phrasing we settled on is leading to misinterpretation. The entire point, I thought, was that we decide at the beginning of the development cycle on which platforms we're testing, and so choose the most recent releases of those LTS distros. If some projects were still running jobs on an earlier platform at the start of the cycle, I don't think we need to be stuck maintaining that testing. The beginning of the cycle is the point at which it's safe for them to switch to the agreed-upon current platform for our upcoming release under development.
Part of the problem is that this didn't actually happen at the beginning of the cycle. The current plan is not to finish the legacy job migration until Apr. 1[0], which means until then projects may have a dependency on py35.
I'm not sure whether this necessarily indicates that the resolution is flawed though - we only update the distro for functional tests once every couple of years, and in this case we didn't adopt the resolution until partway through the cycle so we got a late start. On the other hand, it sounds like the distro migration is going to take around 4 months total (started in Dec., ending early April). Maybe expecting everyone to be on the current distro release right at the start of the cycle is overly ambitious?
On 2019-02-27 11:07:44 -0600 (-0600), Ben Nemec wrote:
On 2/27/19 10:36 AM, Jeremy Stanley wrote:
On 2019-02-27 11:23:19 -0500 (-0500), Zane Bitter wrote: [...]
the resolution says we should unit test "[e]ach Python 3 version that was still used in any integration tests at the beginning of the development cycle"
[...]
Now I'm getting worried that the phrasing we settled on is leading to misinterpretation. The entire point, I thought, was that we decide at the beginning of the development cycle on which platforms we're testing, and so choose the most recent releases of those LTS distros. If some projects were still running jobs on an earlier platform at the start of the cycle, I don't think we need to be stuck maintaining that testing. The beginning of the cycle is the point at which it's safe for them to switch to the agreed-upon current platform for our upcoming release under development.
Part of the problem is that this didn't actually happen at the beginning of the cycle. The current plan is not to finish the legacy job migration until Apr. 1[0], which means until then projects may have a dependency on py35.
[...]
In the past, the way it worked was at the beginning of a new cycle the Infra team said "there's a new Ubuntu LTS release and we have working images for it, you all need to move your jobs to it before the end of the cycle." (Or we just set a date when we were going to switch everyone's jobs over and it was up to them to fix them if they didn't do so before the deadline.) This time the Infra team left it up to the TC to decide on the plan and messaging, so as committee design tradition dictates we spent half the cycle just coming to the conclusion we'd do pretty much what we've done in past cycles. This of course has left teams with far less time to actually implement a transition, and as a result it's overlapping with their attempts to prepare the release.
The key is determination of the platform occurs as early in the cycle as possible so teams have an opportunity to get it all working before doing so impacts finalizing their release. Hopefully now that the plan is baked, we can avoid similar delays for instating future testing platform transitions. I don't think that the delay this time should necessarily inform our policy for future iterations.
---- On Thu, 28 Feb 2019 01:36:29 +0900 Jeremy Stanley fungi@yuggoth.org wrote ----
On 2019-02-27 11:23:19 -0500 (-0500), Zane Bitter wrote: [...]
the resolution says we should unit test "[e]ach Python 3 version that was still used in any integration tests at the beginning of the development cycle"
[...]
Now I'm getting worried that the phrasing we settled on is leading to misinterpretation. The entire point, I thought, was that we decide at the beginning of the development cycle on which platforms we're testing, and so choose the most recent releases of those LTS distros. If some projects were still running jobs on an earlier platform at the start of the cycle, I don't think we need to be stuck maintaining that testing. The beginning of the cycle is the point at which it's safe for them to switch to the agreed-upon current platform for our upcoming release under development.
Main point is, it's all happening together in stein :), migration of LTS distro, mixed of legacy and zuulv3 jobs. Maybe from next cycle (or when we will have new dtiro), it will be easy to migrate the latest distro when all jobs are zuul v3 native and counting the all python 3 versions we need to support as per resolution.
But yes, let's keep the py35 testing until we move all integration jobs to bionic.
-gmann
-- Jeremy Stanley
On 2019-02-28 02:08:37 +0900 (+0900), Ghanshyam Mann wrote: [...]
Main point is, it's all happening together in stein :), migration of LTS distro, mixed of legacy and zuulv3 jobs. Maybe from next cycle (or when we will have new dtiro), it will be easy to migrate the latest distro when all jobs are zuul v3 native and counting the all python 3 versions we need to support as per resolution.
Well, to be fair Zuul v3 job migration has been happening over the course of the past several cycles. I'd love to be able to say that 1.5 years is enough warning and any teams that don't have enough understanding of the new system and their existing jobs to have rewritten them in that time should probably just stop running those jobs instead because they're more of a liability than a benefit. I'm sure that's not a popular view for many, however, so I doubt it's what's actually going to happen.
But yes, let's keep the py35 testing until we move all integration jobs to bionic.
This is less of a question of Python3.5 testing in a vacuum and more a question of platform-specific system interfaces and dependencies. Retaining py35 unit tests does little to ensure that you won't break DevStack on platforms which have Python3.5 as their default Python3 interpreter. If that's the driving concern, then projects need to continue running DevStack jobs on both platforms until all projects have added equivalent jobs on the newer platform. As we saw last time (in the Trusty to Xenial transition), projects who were still gating on integration tests for the old platform got wedged by projects who had switched to only gating their integration tests on the new platform even though they were both using Python2.7 (so it had nothing to do with dropping support for older Python interpreters).
On 27/02/19 11:36 AM, Jeremy Stanley wrote:
On 2019-02-27 11:23:19 -0500 (-0500), Zane Bitter wrote: [...]
the resolution says we should unit test "[e]ach Python 3 version that was still used in any integration tests at the beginning of the development cycle"
[...]
Now I'm getting worried that the phrasing we settled on is leading to misinterpretation. The entire point, I thought, was that we decide at the beginning of the development cycle on which platforms we're testing, and so choose the most recent releases of those LTS distros.
That's a separate bullet point - the first one I quoted. There's two other bullet points, one of which is the one above.
If some projects were still running jobs on an earlier platform at the start of the cycle,
Note that when the platform changes (as it has from Rocky->Stein), it's inevitable that *all* projects will still be running jobs on an earlier platform at the start of the cycle.
I don't think we need to be stuck maintaining that testing. The beginning of the cycle is the point at which it's safe for them to switch to the agreed-upon current platform for our upcoming release under development.
Right, but until they do it's not safe for other projects to drop their unit tests for the old platforms that are still being tested:
https://review.openstack.org/#/c/613145/2..3/resolutions/20181024-python-upd...
In the final version we did say that "Support for these versions can be dropped once all integration tests have migrated", so if everything moved to Bionic before the end of Stein then we could drop py35 unit tests then, and not keep running them on stable/stein.
You commented that you'd like to require that to happen within one release cycle: https://review.openstack.org/#/c/613145/4/resolutions/20181024-python-update... but we decided we'd leave it up to the goal champions to decide case-by-case.
Of course right now we're trying to retrospectively apply guidelines that we set up for Train and beyond to Stein, where we didn't set a goal, we don't have a goal champion, and the configs aren't managed centrally so different projects could theoretically make different choices.
cheers, Zane.
On 2019-02-26 17:28:19 +0100 (+0100), Moises Guimaraes de Medeiros wrote:
So, at this point, is it OK to have projects running against both py35 and py37 and considering py36 covered as being included in the interval?
For Stein (current master development) "...all Python-based projects must target and test against, at a minimum: Python 2.7, Python 3.6."
https://governance.openstack.org/tc/reference/runtimes/stein.html#python-run...
Also keeping 3.5 around and/or adding 3.7 is of course fine, but we set the expectation that 2.7 and 3.6 are actually tested directly, not just indirectly inferred to work.
Also about the lowest supported version, I think that is the one that should be stated in the envlist of tox.ini to fail fast during development.
[...]
Any versions you're testing against in the CI system should also be listed in your tox.ini envlist, ideally. Developers can choose to run as many or as few of those locally as they desire of course.
And be careful with the word "supported" as people often think it means something it doesn't. We do our best to avoid the term in our guidelines, though we did also publish a Technical Committee resolution clarifying what it means in cases where it does show up:
https://governance.openstack.org/tc/resolutions/20170620-volunteer-support.h...
participants (6)
-
Ben Nemec
-
Ghanshyam Mann
-
Jeremy Stanley
-
Moises Guimaraes de Medeiros
-
Sean McGinnis
-
Zane Bitter