Open Stack

Hi, I'm one of the networking-odl core devs.

On Wed, Apr 18, 2018 at 5:48 AM, Jeffrey Zhang <zhang.lei.fly at gmail.com>
wrote:

>
> Recently, one of networking-odl package breaks kolla's gate[0]. The direct
> issue is ceilometer is added in networking-odl's requirements.txt file[1]
>

This is an issue that concerns me too. First off let me start with a simple
solution, which is to install ceilometer from git before requiring
networking-odl. Also, if networking-odl is installed through devstack's
enable_plugin this issue wouldn't arise (as the plugin.sh takes care of
installing ceilometer before installing networking-odl).

Still, I see this as a problem, I just didn't find a way to solve it in
general, except ceilometer being published to PyPI. What happened then is I
got caught up in other priorities that took bandwidth away from it and
kinda forgot about it.


>
> Then when install network-odl with upper-contraints.txt file, it will
> raise error like
>
> $ pip install -c https://git.openstack.org/cgit/openstack/requirements/
> plain/upper-constraints.txt ./networking-odl
> ...
> collecting networking-bgpvpn>=8.0.0 (from networking-odl==12.0.1.dev54)
>   Downloading http://pypi.doubanio.com/packages/5a/e5/
> 995be0d53d472f739a7a0bb6c9d9fecbc4936148651aaf56d39f3b65b1f1
> /networking_bgpvpn-8.0.0-py2-none-any.whl (172kB)
>     100% |████████████████████████████████| 174kB 12.0MB/s
> Collecting ceilometer (from networking-odl==12.0.1.dev54)
>   Could not find a version that satisfies the requirement ceilometer (from
> networking-odl==12.0.1.dev54) (from versions: )
> No matching distribution found for ceilometer (from
> networking-odl==12.0.1.dev54)
>
>
> But if you just install the networking-odl's requirements.txt file, it
> works
>
>
> $ pip install -c https://git.openstack.org/cgit/openstack/requirements/
> plain/upper-constraints.txt -r ./networking-odl/requirements.txt
> ...
> Obtaining ceilometer from git+https://git.openstack.org/
> openstack/ceilometer at master#egg=ceilometer (from -r
> networking-odl/requirements.txt (line 21))
>   Cloning https://git.openstack.org/openstack/ceilometer (to revision
> master) to /home/jeffrey/.dotfiles/virtualenvs/test/src/ceilometer
> ...
>
>
> Is this expected? and how could we fix this?
>

This is an interesting case of how pip works differently when installing
from a requirements file or from a folder (as it would happen with -e or
the first command you issued). While in the former it knows how to solve
the dependencies correctly, in the second it actually relies in the
setup.py file to install. That means it goes into pbr's realm and does not
use the requirements at all. So let's analyse what happens in pbr.

Internally in PBR what is doing is reading the requirements.txt, finding
the -e line, reading it's comment that says #egg=ceilometer and adding that
as a requirement [1]. What is failing to do though, is to instruct pip to
fetch it from the git repository (as the requirements file would do).
Sadly, this is not only a problem of pbr but it's also a limitation of the
current state of pip and the corresponding PEPs, which apparently is
already addressed for the long term with new PEPs and upcoming changes to
pip.

How can we fix this? There are several ways I can think of the top of my
head:


   1. When encountered with edge cases like this one, first install that
   dependency with a manual pip run [2]
   2. Modify pbr to handle these situations by handling the installation of
   those depenencies differently with a workaround to the current
   functionality of pip
   3. Leverage on the work of corvus [3] to not only do what that patch is
   doing, but also including the checked out path of the dependency in
   PIP_FIND_LINKS, that way pip knows how to solve the issue.

All these solutions have different set of pros and cons, but I favor #3 as
the long term solution, #1 as short term and I think #2 requires further
analysis by the pbr team.


Hope my contribution helped to clarify this issue.


[1]:
https://github.com/openstack-dev/pbr/blob/7767c44ab1289ed7d1cc4f9e12986bef07865d5c/pbr/packaging.py#L168

[2]:
https://github.com/openstack/networking-odl/blob/aa3acb23a5736f128fee0a514a588b9035551d88/devstack/entry_points#L259

[3]: https://review.openstack.org/549252/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180418/3c0c964c/attachment.html>