[kolla] Let's switch to Python 3

Mark Goddard mark at stackhpc.com
Fri Mar 1 09:43:05 UTC 2019

Thanks for getting this conversation going.

On Thu, 28 Feb 2019 at 19:46, Marcin Juszkiewicz <
marcin.juszkiewicz at linaro.org> wrote:

> Python 2 is going bye bye. Everyone knows that, lot of us ignores that.
> But we need to move on.

*pulls head out of sand*

> Kolla has few patches to make use of Python3 based binary packages. One
> for Fedora [1], other for Ubuntu and Debian [2].
> 1. https://review.openstack.org/624838
> 2. https://review.openstack.org/625298

There is also the python3 on RHEL8 patch:

> But those patches are just tip of iceberg...
> # Types of images
> There are two types of images in Kolla:
> - binary
> - source
> ## Binaries state in distributions
> Binary ones are built from distribution packages. For
> RHEL/CentOS/OracleLinux is means RDO packages, for Ubuntu it is UCA and
> for Debian it is whatever is in distribution repositories.
> RDO ones are Python 2 based because there is no other version of Python
> in RHEL7 (I do not count EPEL or SCL).

Although RHEL8 is python 3 only.

> For Ubuntu we have a mix of Python 2 and Python 3 (from what I heard).
> Debian needs update to Buster (current 'testing', currently in soft
> freeze) and then we will get Python 3 packages (currently for Rocky).
> ### Pip issue
> On binary distributions we tend to use 'pip' from distribution packages.
> Which means 8.1.2 for CentOS:7, 9.0.1 for Ubuntu:bionic and 18.1 for
> Debian:buster.
For binary images we don't typically use pip, right - so this is just for
edge cases where a package isn't available?

> ## Source images
> In my opinion this type of images is most common during development cycle.
> People use them in production too - better for CI/CD, or if you have
downstream patches.

> Here we install Python and binaries from distribution repositories and
> then bootstrap latest 'pip' and use it to install all Python modules we
> need.
> For now those images are Python 2 based.
> # WIP plans for T cycle
> For T cycle we should move whatever possible to Python 3. Which means
> Debian:buster, Ubuntu:bionic and source images. No idea yet does it
> involves CentOS:7 source images as well as this would require us to use
> EPEL repository for Python 3.
As Jeremy suggests, I think we may need to support both in some cases. That
may not be possible for binary images - we can only use what is offered by
distros - but for source images there's no reason (AFAIK) why we can't have
a config option for the version of python to use. We now [1] have a
distro_python3 config flag that does this. In hindsight, 'distro' might not
be the most appropriate name for source images, but hey.

## Always bootstrap pip
> Due to 'pip' issue we may need to bootstrap latest version in each type
> of images. This will give us simple 'pip' command in each image
> nevermind which version of Python is used. That saves us from dealing
> with "is it 'pip' or 'pip3' we need to use" thing.
Although that question is fairly trivial if there is a config option for
the python version.

I think Martin Andre would have an issue with this, given his recent work
on trying to evict non-distro stuff from binary images.

> ## Create new variables
> There will be few different versions of Python in use: 2.7 (RHEL7
> family), 3.6 (Ubuntu:bionic and RHEL7/EPEL), 3.7 (Debian:buster). As we
> tend to modify some files or need to know where Python modules are
> present. We may use values from 'sys.path' list for it or create
> 'python_path' (or similar) variable and set it per distribution/image_type.
> Also, RHEL8/CentOS8 will be 3.6, Fedora 28 is 3.7?

> # Some kind of work items list
> 1. Move Debian to 'buster' release. https://review.openstack.org/612681
> 2. Bootstrap 'pip' in all image types.
> 3. Move source images to Python 3 (Debian, Ubuntu for sure, no idea yet
>    about CentOS).

s/Move/Add support for/

> 4. Move binary images to Python 3 (Debian, Ubuntu, Fedora/RHEL8)
> Each step will require several build/test/fix cycles. We also need to
> identify which images still require Python 2.
Time for a big table.

> # Summary
> I know that this mail looks a bit chaotic but I think that we need to
> discuss and plan what we need to do and how. And cooperate with external
> teams which provide binary packages (that's why Thomas and Haïkel are in
> Cc:).
> What do you think?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20190301/a797e294/attachment-0001.html>

More information about the openstack-discuss mailing list