Open Stack

On 2017-06-01 12:53:27 -0400 (-0400), Paul Belanger wrote:
> Last night, I hastily approved 469559[1] which ended up doing some
> damage to our production servers.  The symlink logic was not
> correct and what ended up happening was python3 pip was
> downloaded, and installed, followed by our symlink command.  EG:
> 
>   1 - We ran get-pip.py under python3
>   2 - This create pip, pip3, pip3.x for python3
>   3 - pip2 was symlinked to pip (making it python3 also)
> 
> This meant, any existing pip installs that were python2 based were
> incorrectly made python3.
> 
>   pip(python3), pip2(symlink python3), pip3 (new python3)
> 
> We posted 469851[2] this morning to undo the symlinking and
> correctly reinstall pip as python2.  However, during that time,
> any puppet task that used pip could have attempted to install the
> package using python3.
> 
> It is recommend you audit your servers, specifically 3rd party CI,
> to see if there was any issues during this time period.  We
> created an etherpad[3] for openstack-infra to track the failures,
> is has some example commands on how you can help to audit your own
> server.
> 
> Apologies for the troubles today, I should have been more careful
> in reading the initial patch.
> 
> [1] https://review.openstack.org/#/c/469559/
> [2] https://review.openstack.org/#/c/469851/
> [3] https://etherpad.openstack.org/p/infra-pip-symlink-failure

Sorry to follow up to a 6-week-old thread, but we're still running
into situations we missed manually cleaning up. In particular, be
aware that your systems may have ended up with tox and/or virtualenv
reinstalled defaulting to python3, which will in turn create venvs
with that as their default python.

On servers where this has been encountered, it's been sufficient to
perform the following arcane incantation:

    sudo python2.7 -m pip install -U --force pip setuptools tox virtualenv

So just a tip, in case you run into any yourself where virtualenvs
are still getting created with python3 even after the pip symlink
correctly points at pip2.
-- 
Jeremy Stanley
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 949 bytes
Desc: Digital signature
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20170719/7b85b41a/attachment.sig>