Open Stack

On Fri, Mar 1, 2019 at 2:47 AM Mark Goddard <mark at stackhpc.com> wrote:
>
> Thanks for getting this conversation going.
>
> On Thu, 28 Feb 2019 at 19:46, Marcin Juszkiewicz <marcin.juszkiewicz at linaro.org> wrote:
>>
>> Python 2 is going bye bye. Everyone knows that, lot of us ignores that.
>> But we need to move on.
>
>
> *pulls head out of sand*
>>
>>
>> Kolla has few patches to make use of Python3 based binary packages. One
>> for Fedora [1], other for Ubuntu and Debian [2].
>>
>> 1. https://review.openstack.org/624838
>> 2. https://review.openstack.org/625298
>
>
> There is also the python3 on RHEL8 patch: https://review.openstack.org/#/c/632156/19.

Just to raise this up again, but would it be possible to try and land
https://review.openstack.org/#/c/632156/ and
https://review.openstack.org/#/c/639219/. We'd really like to get a
head start on the python3 testing as we've got some python3 packages
for fedora & stein.

>>
>>
>>
>> But those patches are just tip of iceberg...
>>
>> # Types of images
>>
>> There are two types of images in Kolla:
>>
>> - binary
>> - source
>>
>> ## Binaries state in distributions
>>
>> Binary ones are built from distribution packages. For
>> RHEL/CentOS/OracleLinux is means RDO packages, for Ubuntu it is UCA and
>> for Debian it is whatever is in distribution repositories.
>>
>> RDO ones are Python 2 based because there is no other version of Python
>> in RHEL7 (I do not count EPEL or SCL).
>
>
> Although RHEL8 is python 3 only.
>
>>
>> For Ubuntu we have a mix of Python 2 and Python 3 (from what I heard).
>>
>> Debian needs update to Buster (current 'testing', currently in soft
>> freeze) and then we will get Python 3 packages (currently for Rocky).
>>
>> ### Pip issue
>>
>> On binary distributions we tend to use 'pip' from distribution packages.
>> Which means 8.1.2 for CentOS:7, 9.0.1 for Ubuntu:bionic and 18.1 for
>> Debian:buster.
>>
>
> For binary images we don't typically use pip, right - so this is just for edge cases where a package isn't available?
>
>>
>>
>> ## Source images
>>
>> In my opinion this type of images is most common during development cycle.
>>
> People use them in production too - better for CI/CD, or if you have downstream patches.
>
>>
>> Here we install Python and binaries from distribution repositories and
>> then bootstrap latest 'pip' and use it to install all Python modules we
>> need.
>>
>> For now those images are Python 2 based.
>>
>>
>> # WIP plans for T cycle
>>
>> For T cycle we should move whatever possible to Python 3. Which means
>> Debian:buster, Ubuntu:bionic and source images. No idea yet does it
>> involves CentOS:7 source images as well as this would require us to use
>> EPEL repository for Python 3.
>>
>
> As Jeremy suggests, I think we may need to support both in some cases. That may not be possible for binary images - we can only use what is offered by distros - but for source images there's no reason (AFAIK) why we can't have a config option for the version of python to use. We now [1] have a distro_python3 config flag that does this. In hindsight, 'distro' might not be the most appropriate name for source images, but hey.
>
>> ## Always bootstrap pip
>>
>> Due to 'pip' issue we may need to bootstrap latest version in each type
>> of images. This will give us simple 'pip' command in each image
>> nevermind which version of Python is used. That saves us from dealing
>> with "is it 'pip' or 'pip3' we need to use" thing.
>>
>
> Although that question is fairly trivial if there is a config option for the python version.
>
> I think Martin Andre would have an issue with this, given his recent work on trying to evict non-distro stuff from binary images.
>>
>> ## Create new variables
>>
>> There will be few different versions of Python in use: 2.7 (RHEL7
>> family), 3.6 (Ubuntu:bionic and RHEL7/EPEL), 3.7 (Debian:buster). As we
>> tend to modify some files or need to know where Python modules are
>> present. We may use values from 'sys.path' list for it or create
>> 'python_path' (or similar) variable and set it per distribution/image_type.
>>
> Also, RHEL8/CentOS8 will be 3.6, Fedora 28 is 3.7?
>
>>
>> # Some kind of work items list
>>
>> 1. Move Debian to 'buster' release. https://review.openstack.org/612681
>> 2. Bootstrap 'pip' in all image types.
>> 3. Move source images to Python 3 (Debian, Ubuntu for sure, no idea yet
>>    about CentOS).
>
>
> s/Move/Add support for/
>
>>
>> 4. Move binary images to Python 3 (Debian, Ubuntu, Fedora/RHEL8)
>>
>> Each step will require several build/test/fix cycles. We also need to
>> identify which images still require Python 2.
>>
>
> Time for a big table.
>
>>
>> # Summary
>>
>> I know that this mail looks a bit chaotic but I think that we need to
>> discuss and plan what we need to do and how. And cooperate with external
>> teams which provide binary packages (that's why Thomas and Haïkel are in
>> Cc:).
>>
>> What do you think?