<div dir="ltr">Hi, I'm one of the networking-odl core devs.<br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Apr 18, 2018 at 5:48 AM, Jeffrey Zhang <span dir="ltr"><<a href="mailto:zhang.lei.fly@gmail.com" target="_blank">zhang.lei.fly@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:arial,helvetica,sans-serif;font-size:small"><br clear="all"></div><div><div style="font-family:arial,helvetica,sans-serif;font-size:small">Recently, one of networking-odl package breaks kolla's gate[0]. The direct issue is ceilometer is added in networking-odl's requirements.txt file[1]</div></div></div></blockquote><div><br>This is an issue that concerns me too. First off let me start with a simple solution, which is to install ceilometer from git before requiring networking-odl. Also, if networking-odl is installed through devstack's enable_plugin this issue wouldn't arise (as the plugin.sh takes care of installing ceilometer before installing networking-odl).<br><br></div><div>Still, I see this as a problem, I just didn't find a way to solve it in general, except ceilometer being published to PyPI. What happened then is I got caught up in other priorities that took bandwidth away from it and kinda forgot about it.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div style="font-family:arial,helvetica,sans-serif;font-size:small"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small">Then when install network-odl with upper-contraints.txt file, it will raise error like </div><div style="font-family:arial,helvetica,sans-serif;font-size:small"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small">$ pip install -c <a href="https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt" target="_blank">https://git.openstack.org/<wbr>cgit/openstack/requirements/<wbr>plain/upper-constraints.txt</a> ./networking-odl</div><div style="font-family:arial,helvetica,sans-serif;font-size:small">...</div><div><div style="font-family:arial,helvetica,sans-serif;font-size:small">collecting networking-bgpvpn>=8.0.0 (from networking-odl==12.0.1.dev54)</div><div style="font-family:arial,helvetica,sans-serif;font-size:small">  Downloading <a href="http://pypi.doubanio.com/packages/5a/e5/995be0d53d472f739a7a0bb6c9d9fecbc4936148651aaf56d39f3b65b1f1/networking_bgpvpn-8.0.0-py2-none-any.whl" target="_blank">http://pypi.doubanio.com/<wbr>packages/5a/e5/<wbr>995be0d53d472f739a7a0bb6c9d9fe<wbr>cbc4936148651aaf56d39f3b65b1f1<wbr>/networking_bgpvpn-8.0.0-py2-<wbr>none-any.whl</a> (172kB)</div><div style="font-family:arial,helvetica,sans-serif;font-size:small">    100% |█████████████████████████████<wbr>███| 174kB 12.0MB/s </div><div style="font-family:arial,helvetica,sans-serif;font-size:small">Collecting ceilometer (from networking-odl==12.0.1.dev54)</div><div style="font-family:arial,helvetica,sans-serif;font-size:small">  Could not find a version that satisfies the requirement ceilometer (from networking-odl==12.0.1.dev54) (from versions: )</div><div style="font-family:arial,helvetica,sans-serif;font-size:small">No matching distribution found for ceilometer (from networking-odl==12.0.1.dev54)</div><div style="font-family:arial,helvetica,sans-serif;font-size:small"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small">But if you just install the networking-odl's requirements.<wbr>txt file, it works</div><div style="font-family:arial,helvetica,sans-serif;font-size:small"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small">$ <span style="color:rgb(34,34,34);font-family:arial,helvetica,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">pip install -c <a href="https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt" target="_blank">https://git.openstack.org/<wbr>cgit/openstack/requirements/<wbr>plain/upper-constraints.txt</a> -r ./networking-odl/requirements.<wbr>txt</span></div><div style="font-family:arial,helvetica,sans-serif;font-size:small">...</div><div><div><font face="arial, helvetica, sans-serif">Obtaining ceilometer from git+<a href="https://git.openstack.org/openstack/ceilometer@master#egg=ceilometer" target="_blank">https://git.openstack.org/<wbr>openstack/ceilometer@master#<wbr>egg=ceilometer</a> (from -r networking-odl/requirements.<wbr>txt (line 21))</font></div><div><font face="arial, helvetica, sans-serif">  Cloning <a href="https://git.openstack.org/openstack/ceilometer" target="_blank">https://git.openstack.org/<wbr>openstack/ceilometer</a> (to revision master) to /home/jeffrey/.dotfiles/<wbr>virtualenvs/test/src/<wbr>ceilometer</font></div></div><div style="font-family:arial,helvetica,sans-serif;font-size:small">...</div></div><div style="font-family:arial,helvetica,sans-serif;font-size:small"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small"><br></div><div style="font-family:arial,helvetica,sans-serif;font-size:small">Is this expected? and how could we fix this?</div></div></div></blockquote><div><br></div><div>This is an interesting case of how pip works differently when installing from a requirements file or from a folder (as it would happen with -e or the first command you issued). While in the former it knows how to solve the dependencies correctly, in the second it actually relies in the setup.py file to install. That means it goes into pbr's realm and does not use the requirements at all. So let's analyse what happens in pbr.<br><br></div><div>Internally in PBR what is doing is reading the requirements.txt, finding the -e line, reading it's comment that says #egg=ceilometer and adding that as a requirement [1]. What is failing to do though, is to instruct pip to fetch it from the git repository (as the requirements file would do). Sadly, this is not only a problem of pbr but it's also a limitation of the current state of pip and the corresponding PEPs, which apparently is already addressed for the long term with new PEPs and upcoming changes to pip.<br><br></div><div>How can we fix this? There are several ways I can think of the top of my head:<br><br><ol><li>When encountered with edge cases like this one, first install that dependency with a manual pip run [2]</li><li>Modify pbr to handle these situations by handling the installation of those depenencies differently with a workaround to the current functionality of pip</li><li>Leverage on the work of corvus [3] to not only do what that patch is doing, but also including the checked out path of the dependency in PIP_FIND_LINKS, that way pip knows how to solve the issue.</li></ol><p>All these solutions have different set of pros and cons, but I favor #3 as the long term solution, #1 as short term and I think #2 requires further analysis by the pbr team.</p><p><br></p><p>Hope my contribution helped to clarify this issue. <br></p><p><br></p><p>[1]: <a href="https://github.com/openstack-dev/pbr/blob/7767c44ab1289ed7d1cc4f9e12986bef07865d5c/pbr/packaging.py#L168">https://github.com/openstack-dev/pbr/blob/7767c44ab1289ed7d1cc4f9e12986bef07865d5c/pbr/packaging.py#L168</a><br></p><p>[2]: <a href="https://github.com/openstack/networking-odl/blob/aa3acb23a5736f128fee0a514a588b9035551d88/devstack/entry_points#L259">https://github.com/openstack/networking-odl/blob/aa3acb23a5736f128fee0a514a588b9035551d88/devstack/entry_points#L259</a></p><p>[3]: <a href="https://review.openstack.org/549252/">https://review.openstack.org/549252/</a><br></p></div></div></div></div>