Open Stack

On Fri, Oct 20, 2017 at 4:32 PM, Doug Hellmann <doug at doughellmann.com>
wrote:

> Excerpts from Clark Boylan's message of 2017-10-20 13:14:13 -0700:
> > On Fri, Oct 20, 2017, at 11:17 AM, Clark Boylan wrote:
> > > On Fri, Oct 20, 2017, at 07:23 AM, Doug Hellmann wrote:
> > > > It sounds like the PyPI/PyPA folks are planning some major changes to
> > > > pip internals, soon.
> > > >
> > > > I know pbr uses setuptools, and I don't think it uses pip, but if
> > > > someone has time to verify that it would be helpful.
> > > >
> > > > We'll also want to watch out for breakage in normal use of pip 10. If
> > > > they're making changes this big, they may miss something in their own
> > > > test coverage that affects our jobs.
> > > >
> > >
> > > After a quick skim of PBR I don't think we use pip internals anywhere.
> > > Its all executed via the command itself. That said we should test this
> > > so I've put up https://review.openstack.org/513825 (others should feel
> > > free to iterate on it if it doesn't work) to install latest pip master
> > > in a devstack run.
> >
> > The current issue this change is facing can be seen at
> > http://logs.openstack.org/25/513825/4/check/legacy-tempest-
> dsvm-py35/c31deb2/logs/devstacklog.txt.gz#_2017-10-20_20_07_54_838.
> > The tl;dr is that for distutils installed packages (basically all the
> > distro installed python packges) pip refuses to uninstall them in order
> > to perform upgrades because it can't reliably determine where all the
> > files are. I think this is a new pip 10 behavior.
> >
> > In the general case I think this means we can not rely on global pip
> > installs anymore. This may be a good thing to bring up with upstream
> > PyPA as I expect it will break a lot of people in a lot of places (it
> > will break infra for example too).
> >
> > Clark
> >
>
> Yes, it would be good for someone who understands the cases where we're
> doing these sorts of global package installations using pip to post on
> distutils-sig explaining the breakage and asking for some time to let us
> work around the issue.
>
> We have a couple of ways to mitigate it, depending on the situation.
>
> 1. Pin the affected packages to the system package versions in our
>    constraints list.
>
> 2. Install into a virtualenv that ignores system packages, avoiding the
>    need to upgrade at all.
>
> 3. Remove the system package before installing from pip (why is it there
>    in the first place?).
>
> It's not clear to me which is the best approach.  I've added
> [devstack][qa] to the subject line to draw some more attention to
> this thread from folks familiar with these jobs.
>
> Doug
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

I have used option 2 with great success to avoid the issue of system
package conflicts for a couple of years now, back when pip would still
uninstall system packages for upgrades.

Option 3 is difficult to make work since so many packages have dependancies
on python-* packages, I abandoned that approach fairly quickly.

Thanks,
SamYaple
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20171020/6907c2f9/attachment.html>