Thanks for getting this conversation going. On Thu, 28 Feb 2019 at 19:46, Marcin Juszkiewicz < marcin.juszkiewicz@linaro.org> wrote:
Python 2 is going bye bye. Everyone knows that, lot of us ignores that. But we need to move on.
*pulls head out of sand*
Kolla has few patches to make use of Python3 based binary packages. One for Fedora [1], other for Ubuntu and Debian [2].
1. https://review.openstack.org/624838 2. https://review.openstack.org/625298
There is also the python3 on RHEL8 patch: https://review.openstack.org/#/c/632156/19.
But those patches are just tip of iceberg...
# Types of images
There are two types of images in Kolla:
- binary - source
## Binaries state in distributions
Binary ones are built from distribution packages. For RHEL/CentOS/OracleLinux is means RDO packages, for Ubuntu it is UCA and for Debian it is whatever is in distribution repositories.
RDO ones are Python 2 based because there is no other version of Python in RHEL7 (I do not count EPEL or SCL).
Although RHEL8 is python 3 only.
For Ubuntu we have a mix of Python 2 and Python 3 (from what I heard).
Debian needs update to Buster (current 'testing', currently in soft freeze) and then we will get Python 3 packages (currently for Rocky).
### Pip issue
On binary distributions we tend to use 'pip' from distribution packages. Which means 8.1.2 for CentOS:7, 9.0.1 for Ubuntu:bionic and 18.1 for Debian:buster.
For binary images we don't typically use pip, right - so this is just for edge cases where a package isn't available?
## Source images
In my opinion this type of images is most common during development cycle.
People use them in production too - better for CI/CD, or if you have
downstream patches.
Here we install Python and binaries from distribution repositories and then bootstrap latest 'pip' and use it to install all Python modules we need.
For now those images are Python 2 based.
# WIP plans for T cycle
For T cycle we should move whatever possible to Python 3. Which means Debian:buster, Ubuntu:bionic and source images. No idea yet does it involves CentOS:7 source images as well as this would require us to use EPEL repository for Python 3.
As Jeremy suggests, I think we may need to support both in some cases. That may not be possible for binary images - we can only use what is offered by distros - but for source images there's no reason (AFAIK) why we can't have a config option for the version of python to use. We now [1] have a distro_python3 config flag that does this. In hindsight, 'distro' might not be the most appropriate name for source images, but hey. ## Always bootstrap pip
Due to 'pip' issue we may need to bootstrap latest version in each type of images. This will give us simple 'pip' command in each image nevermind which version of Python is used. That saves us from dealing with "is it 'pip' or 'pip3' we need to use" thing.
Although that question is fairly trivial if there is a config option for the python version. I think Martin Andre would have an issue with this, given his recent work on trying to evict non-distro stuff from binary images.
## Create new variables
There will be few different versions of Python in use: 2.7 (RHEL7 family), 3.6 (Ubuntu:bionic and RHEL7/EPEL), 3.7 (Debian:buster). As we tend to modify some files or need to know where Python modules are present. We may use values from 'sys.path' list for it or create 'python_path' (or similar) variable and set it per distribution/image_type.
Also, RHEL8/CentOS8 will be 3.6, Fedora 28 is 3.7?
# Some kind of work items list
1. Move Debian to 'buster' release. https://review.openstack.org/612681 2. Bootstrap 'pip' in all image types. 3. Move source images to Python 3 (Debian, Ubuntu for sure, no idea yet about CentOS).
s/Move/Add support for/
4. Move binary images to Python 3 (Debian, Ubuntu, Fedora/RHEL8)
Each step will require several build/test/fix cycles. We also need to identify which images still require Python 2.
Time for a big table.
# Summary
I know that this mail looks a bit chaotic but I think that we need to discuss and plan what we need to do and how. And cooperate with external teams which provide binary packages (that's why Thomas and Haïkel are in Cc:).
What do you think?