Open Stack

There seems to be agreement that this is broadly a good direction to 
pursue, so I proposed a TC resolution. Let's shift discussion to the review:

https://review.openstack.org/613145

cheers,
Zane.

On 19/10/18 11:17 AM, Zane Bitter wrote:
> There hasn't been a Python 2 release in 8 years, and during that time 
> we've gotten used to the idea that that's the way things go. However, 
> with the switch to Python 3 looming (we will drop support for Python 2 
> in the U release[1]), history is no longer a good guide: Python 3 
> releases drop as often as every year. We are already feeling the pain 
> from this, as Linux distros have largely already completed the shift to 
> Python 3, and those that have are on versions newer than the py35 we 
> currently have in gate jobs.
> 
> We have traditionally held to the principle that we want each release to 
> support the latest release of CentOS and the latest LTS release of 
> Ubuntu, as they existed at the beginning of the release cycle.[2] 
> Currently this means in practice one version of py2 and one of py3, but 
> in the future it will mean two, usually different, versions of py3.
> 
> There are two separate issues that we need to address: unit tests (we'll 
> define this as code tested in isolation, within or spawned from within 
> the testing process), and integration tests (we'll define this as code 
> running in its own process, tested from the outside). I have two 
> separate but related proposal for how to handle those.
> 
> I'd like to avoid discussion which versions of things we think should be 
> supported in Stein in this thread. Let's come up with a process that we 
> think is a good one to take into T and beyond, and then retroactively 
> apply it to Stein. Competing proposals are of course welcome, in 
> addition to feedback on this one.
> 
> Unit Tests
> ----------
> 
> For unit tests, the most important thing is to test on the versions of 
> Python we target. It's less important to be using the exact distro that 
> we want to target, because unit tests generally won't interact with 
> stuff outside of Python.
> 
> I'd like to propose that we handle this by setting up a unit test 
> template in openstack-zuul-jobs for each release. So for Stein we'd have 
> openstack-python3-stein-jobs. This template would contain:
> 
> * A voting gate job for the highest minor version of py3 we want to 
> support in that release.
> * A voting gate job for the lowest minor version of py3 we want to 
> support in that release.
> * A periodic job for any interim minor releases.
> * (Starting late in the cycle) a non-voting check job for the highest 
> minor version of py3 we want to support in the *next* release (if 
> different), on the master branch only.
> 
> So, for example, (and this is still under active debate) for Stein we 
> might have gating jobs for py35 and py37, with a periodic job for py36. 
> The T jobs might only have voting py36 and py37 jobs, but late in the T 
> cycle we might add a non-voting py38 job on master so that people who 
> haven't switched to the U template yet can see what, if anything, 
> they'll need to fix.
> 
> We'll run the unit tests on any distro we can find that supports the 
> version of Python we want. It could be a non-LTS Ubuntu, Fedora, Debian 
> unstable, whatever it takes. We won't wait for an LTS Ubuntu to have a 
> particular Python version before trying to test it.
> 
> Before the start of each cycle, the TC would determine which range of 
> versions we want to support, on the basis of the latest one we can find 
> in any distro and the earliest one we're likely to need in one of the 
> supported Linux distros. There will be a project-wide goal to switch the 
> testing template from e.g. openstack-python3-stein-jobs to 
> openstack-python3-treasure-jobs for every repo before the end of the 
> cycle. We'll have goal champions as usual following up and helping teams 
> with the process. We'll know where the problem areas are because we'll 
> have added non-voting jobs for any new Python versions to the previous 
> release's template.
> 
> Integration Tests
> -----------------
> 
> Integration tests do test, amongst other things, integration with 
> non-openstack-supplied things in the distro, so it's important that we 
> test on the actual distros we have identified as popular.[2] It's also 
> important that every project be testing on the same distro at the end of 
> a release, so we can be sure they all work together for users.
> 
> When a new release of CentOS or a new LTS release of Ubuntu comes out, 
> the TC will create a project-wide goal for the *next* release cycle to 
> switch all integration tests over to that distro. It's up to individual 
> projects to make the switch for the tests that they own (e.g. it'd be 
> the QA team for Tempest, but other individual projects for their own 
> jobs). Again, there'll be a goal champion to monitor and follow up.
> 
> 
> [1] 
> https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation-timeline.html 
> 
> [2] 
> https://governance.openstack.org/tc/reference/project-testing-interface.html#linux-distributions 
>