Open Stack

On Wed, 2019-10-16 at 10:03 +1100, Ian Wienand wrote:
> Hello,
> 
> I'm happy to say that CentOS 8 images are now live in OpenDev infra.
> Using a node label of "centos-8" will get you started.
> 
> ---
> 
> The python environment setup on these images is different to our other
> images.  Firstly, some background: currently during image build we go
> to some effort to:
> 
> a) install the latest pip/virtualenv/setuptools
> b) ensure standard behaviour:
>     /usr/bin/python     -> python2
python on centos8 should be a link to python3
infact ideally python 2 should not be installed at all.
>     /usr/bin/pip        -> python2 install
>     /usr/bin/pip3       -> python3 install
>     /usr/bin/virtualenv -> creates python2 environment by default;
>                            python3 virtualenv package in sync
> 
> This means a number of things; hosts always have python2,
why would we want this. ideally we should try to ensure that
there is non python 2 on the system so that we can ensure we are not using
it bay mistake and can do an entirly python3 only install on centos8
----
later: i realised read later that you are descibing how thigns work on other images
here.
>  and because
> we overwrite the system pip/virtualenv/setuptools we put these
> packages on "hold" (depending on the package manager) so jobs don't
> re-install them and create (even more of) a mess.
> 
> This made sense in the past, when we had versions of pip/setuptools in
> distributions that couldn't understand syntax in requirements files
> (and other bugs) and didn't have the current fantastic job inheritance
> and modularity that Zuul (v3) provides.  However, it also introduces a
> range of problems for various users, and has a high maintenance
> overhead.
> 
> Thus these new images are, by default, python3 only, and have upstream
> pip and virtualenv packages installed.  You will have a default
> situation:
> 
>  /usr/bin/python -> not provided
>  /usr/bin/pip    -> not provided, use /usr/bin/pip3 or "python3 -m pip"
>  /usr/bin/virtualenv -> create python3 environment;
>                         provided by upstream python3-virtualenv package
>  /usr/bin/pyvenv -> not provided (is provided by Ubuntu python3-venv),
>                     use /usr/bin/pyvenv-3 or "python3 -m venv".
> 
oh i see you were descirbinbg the standard behavior of our other envs before
this is closer to what i was expecting alther i would personally prefer to have
/usr/bin/python -> /user/bin/python3

linux distros  seem to be a bit split on this
i belive arch and maybe debian (i saw on of the other majory disto families adopt the same apparch) link 
python to python3

the redhat family of operating systems do not provide python any more an leave it to the user to either use
only the versions specific pythons or user update-alternitives to set there default python
> Ergo, the "standard behaviour" is not so standard any more.
> 
> I would suggest if you wish to write somewhat portable jobs/roles
> etc., you do the following:
> 
>  * in general don't rely on "unversioned" calls of tools at all
>    (python/pip/virtualenv) -- they can all mean different things on
>    different platforms.
>  * scripts should always be #!/usr/bin/python3
   see i think that is an anti pattern they could be but i think 
   /usr/bin/python should map to /usr/bin/python3 and you should assume
   that it now python3. if you dont do that hen
every script that has ever been
   writtne or packaged needs to be updated to reference python3 explictly.
   there were much fewer user of python1 when that tansition happened but
   python became a link to the default systme python and eventully pointed to python2
   i think we should continue to do that and after a decase of deprecating python2 we
   should reclaim the python symlink and point it to python3
>  * use "python3 -m venv" for virtual environments (if you really need
>    "virtualenv" because of one of the features it provides, use "-m
>    virtualenv")
>  * use "python3 -m pip" to install global pip packages; but try not
>    too -- mixing packages and pip installs never works that well.
well from a devstack point of view we almost exclucive install form
pip so installing python packages form the disto is the anti pattern
not installing form pip. that said we shoudl consider moving devstack to use
--user at somepoint.
>  * if you need python2 for some reason, use a bindep file+role
>    to install it (don't assume it is there)
+1 also dont assmue python will be python
> 
> ---
> 
> For any Zuul admins, note that to use python3-only images similar to
> what we make, you'll need to set "python-path" to python3 in nodepool
> so that Ansible calls the correct remote binary.  Keep an eye on [1]
> which will automate this for Ansible >=2.8 after things are merged and
> released.
> 
> ---
> 
> Most of the job setup has been tested (network configs, setting
> mirrors, adding swap etc.) but there's always a chance of issues with
> a new platform.  Please bring up any issues in #openstack-infra and
> we'll be sure to get them fixed.
> 
> ---
> 
> If you're interested in the images, they are exported at
> 
>  https://nb01.openstack.org/images/
> 
> although they are rather large, because we pre-cache a lot.  If you'd
> like to build your own, [2] might help with:
> 
>  DISTRO=centos-minimal
>  DIB_RELEASE=8
> 
> ---
> 
> Thanks,
thanks for al the work on this.
> 
> -i
> 
> [1] https://review.opendev.org/#/c/682797/
> [2] https://opendev.org/openstack/project-config/src/branch/master/tools/build-image.sh
> 
>