Thanks for getting this conversation going.

On Thu, 28 Feb 2019 at 19:46, Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org> wrote:
Python 2 is going bye bye. Everyone knows that, lot of us ignores that.
But we need to move on.

*pulls head out of sand*

Kolla has few patches to make use of Python3 based binary packages. One
for Fedora [1], other for Ubuntu and Debian [2].

1. https://review.openstack.org/624838
2. https://review.openstack.org/625298

There is also the python3 on RHEL8 patch: https://review.openstack.org/#/c/632156/19.


But those patches are just tip of iceberg...

# Types of images

There are two types of images in Kolla:

- binary
- source

## Binaries state in distributions

Binary ones are built from distribution packages. For
RHEL/CentOS/OracleLinux is means RDO packages, for Ubuntu it is UCA and
for Debian it is whatever is in distribution repositories.

RDO ones are Python 2 based because there is no other version of Python
in RHEL7 (I do not count EPEL or SCL).

Although RHEL8 is python 3 only. 


For Ubuntu we have a mix of Python 2 and Python 3 (from what I heard).

Debian needs update to Buster (current 'testing', currently in soft
freeze) and then we will get Python 3 packages (currently for Rocky).

### Pip issue

On binary distributions we tend to use 'pip' from distribution packages.
Which means 8.1.2 for CentOS:7, 9.0.1 for Ubuntu:bionic and 18.1 for
Debian:buster.


For binary images we don't typically use pip, right - so this is just for edge cases where a package isn't available?
 

## Source images

In my opinion this type of images is most common during development cycle.

People use them in production too - better for CI/CD, or if you have downstream patches.
 
Here we install Python and binaries from distribution repositories and
then bootstrap latest 'pip' and use it to install all Python modules we
need.

For now those images are Python 2 based.


# WIP plans for T cycle

For T cycle we should move whatever possible to Python 3. Which means
Debian:buster, Ubuntu:bionic and source images. No idea yet does it
involves CentOS:7 source images as well as this would require us to use
EPEL repository for Python 3.

 
As Jeremy suggests, I think we may need to support both in some cases. That may not be possible for binary images - we can only use what is offered by distros - but for source images there's no reason (AFAIK) why we can't have a config option for the version of python to use. We now [1] have a distro_python3 config flag that does this. In hindsight, 'distro' might not be the most appropriate name for source images, but hey.

## Always bootstrap pip

Due to 'pip' issue we may need to bootstrap latest version in each type
of images. This will give us simple 'pip' command in each image
nevermind which version of Python is used. That saves us from dealing
with "is it 'pip' or 'pip3' we need to use" thing.


Although that question is fairly trivial if there is a config option for the python version.

I think Martin Andre would have an issue with this, given his recent work on trying to evict non-distro stuff from binary images.
## Create new variables

There will be few different versions of Python in use: 2.7 (RHEL7
family), 3.6 (Ubuntu:bionic and RHEL7/EPEL), 3.7 (Debian:buster). As we
tend to modify some files or need to know where Python modules are
present. We may use values from 'sys.path' list for it or create
'python_path' (or similar) variable and set it per distribution/image_type.

Also, RHEL8/CentOS8 will be 3.6, Fedora 28 is 3.7?


# Some kind of work items list

1. Move Debian to 'buster' release. https://review.openstack.org/612681
2. Bootstrap 'pip' in all image types.
3. Move source images to Python 3 (Debian, Ubuntu for sure, no idea yet
   about CentOS).

s/Move/Add support for/
 
4. Move binary images to Python 3 (Debian, Ubuntu, Fedora/RHEL8)

Each step will require several build/test/fix cycles. We also need to
identify which images still require Python 2.

 
Time for a big table.


# Summary

I know that this mail looks a bit chaotic but I think that we need to
discuss and plan what we need to do and how. And cooperate with external
teams which provide binary packages (that's why Thomas and Haïkel are in
Cc:).

What do you think?