Open Stack

On 22/10/18 10:33 AM, Thomas Goirand wrote:
> On 10/19/18 5:17 PM, Zane Bitter wrote:
>> We have traditionally held to the principle that we want each release to
>> support the latest release of CentOS and the latest LTS release of
>> Ubuntu, as they existed at the beginning of the release cycle.[2]
>> Currently this means in practice one version of py2 and one of py3, but
>> in the future it will mean two, usually different, versions of py3.
> 
> That's not very nice to forget about the Debian case, which usually
> closely precedes Ubuntu. If you want to support Ubuntu better, then
> supporting Debian better helps. I usually get the issue before everyone,
> as Sid is the distro which is updated the most often. Therefore, please
> make sure to include Debian in your proposal.

This is something that needs to be addressed separately I think. It has 
been our long-standing, documented testing policy. If you want to change 
it, make a proposal. For the purposes of this discussion though, the 
main point to take away from the paragraph you quoted is that once 
Python2 is EOL there will rarely be a _single_ version of Python3 that 
is sufficient to support even 2 distros, let alone more.

I haven't forgotten about you, and in fact one of the goals of this 
process is to ensure that we stay up-to-date and not get into situations 
like you had in Rocky where we were two releases behind. Debian will 
definitely benefit from that.

>> For unit tests, the most important thing is to test on the versions of
>> Python we target. It's less important to be using the exact distro that
>> we want to target, because unit tests generally won't interact with
>> stuff outside of Python.
> 
> One of the reoccurring problem that I'm facing in Debian is that not
> only Python 3 version is lagging behind, but OpenStack dependencies are
> also lagging behind the distro. Often, the answer is "we don't support
> this or that version of X", which of course is very frustrating. One
> thing which would be super nice, would be a non-voting gate job that
> test with the latest version of every Python dependencies as well, so we
> get to see breakage early. We've stopped seeing them since we decided it
> breaks too often and we would hide problems behind the
> global-requirement thing.

I'll leave this to the requirements team, who are more qualified to comment.

> And sometimes, we have weird interactions. For example, taskflow was
> broken in Python 3.7 before this patch:
> https://salsa.debian.org/openstack-team/libs/python-taskflow/commit/6a10261a8a147d901c07a6e7272dc75b9f4d0988
> 
> which broke multiple packages using it. Funny thing, it looks like it
> wouldn't have happen if we didn't have a pre-version of Python 3.7.1 in
> Sid, apparently. Anyway, this can happen again.
> 
>> So, for example, (and this is still under active debate) for Stein we
>> might have gating jobs for py35 and py37, with a periodic job for py36.
>> The T jobs might only have voting py36 and py37 jobs, but late in the T
>> cycle we might add a non-voting py38 job on master so that people who
>> haven't switched to the U template yet can see what, if anything,
>> they'll need to fix.
> 
> This can only happen if we have supporting distribution packages for it.
> IMO, this is a call for using Debian Testing or even Sid in the gate.

It depends on which versions we choose to support, but if necessary yes.

>> We'll run the unit tests on any distro we can find that supports the
>> version of Python we want. It could be a non-LTS Ubuntu, Fedora, Debian
>> unstable, whatever it takes. We won't wait for an LTS Ubuntu to have a
>> particular Python version before trying to test it.
> 
> I very much agree with that.
> 
>> Before the start of each cycle, the TC would determine which range of
>> versions we want to support, on the basis of the latest one we can find
>> in any distro and the earliest one we're likely to need in one of the
>> supported Linux distros.
> 
> Release of Python aren't aligned with OpenStack cycles. Python 3.7
> appeared late in the Rocky cycle. Therefore, unfortunately, doing what
> you propose above doesn't address the issue.

This is valuable feedback; it's important to know where there are 
real-world cases that we're not addressing.

Python 3.7 was released 3 weeks after rocky-2 and only 4 weeks before 
rocky-3. TBH I find it hard to imagine any process that would have led 
us to attempt to get every OpenStack project supporting 3.7 in Rocky 
without a radical change in our conception of how OpenStack is distributed.

On the bright side, under this process we would have had 3.6 support in 
Ocata and we could have automatically added a non-voting (or periodic) 
3.7 job during Rocky development as soon as a distro was available for 
testing, which would at least have made it easier to locate problems 
earlier even if we didn't get full 3.7 support until the Stein release.

>> Integration Tests
>> -----------------
>>
>> Integration tests do test, amongst other things, integration with
>> non-openstack-supplied things in the distro, so it's important that we
>> test on the actual distros we have identified as popular.[2] It's also
>> important that every project be testing on the same distro at the end of
>> a release, so we can be sure they all work together for users.
> 
> I find very disturbing to see the project only leaning toward these only
> 2 distributions. Why not SuSE & Debian?

The bottom line is it's because targeting those two catches 88% of our 
users. (For once I did not make this statistic up.)

Also note that in practice I believe almost everything is actually 
tested on Ubuntu LTS, and only TripleO is testing on CentOS. It's 
difficult to imagine how to slot another distro into the mix without 
doubling up on jobs.

cheers,
Zane.