Open Stack

On 2014-01-08 10:50, Doug Hellmann wrote: 

> On Wed, Jan 8, 2014 at 11:31 AM, Ben Nemec <openstack at nemebean.com> wrote:
> 
> On 2014-01-08 08:24, Doug Hellmann wrote: 
> 
> On Tue, Jan 7, 2014 at 12:32 PM, Ben Nemec <openstack at nemebean.com> wrote:
> 
> On 2014-01-07 07:16, Doug Hellmann wrote: 
> 
> On Tue, Jan 7, 2014 at 6:24 AM, Michael Kerrin <michael.kerrin at hp.com> wrote:
> 
> I have been seeing this problem also. 
> 
> My problem is actually with oslo.sphinx. I ran sudo pip install -r test-requirements.txt in cinder so that I could run the tests there, which installed oslo.sphinx. 
> 
> Strange thing is that the oslo.sphinx installed a directory called oslo in /usr/local/lib/python2.7/dist-packages with no __init__.py file. With this package installed like so I get the same error you get with oslo.config. 
> 
> The oslo libraries use python namespace packages, which manifest themselves as a directory in site-packages (or dist-packages) with sub-packages but no __init__.py(c). That way oslo.sphinx and oslo.config can be packaged separately, but still installed under the "oslo" directory and imported as oslo.sphinx and oslo.config. 
> 
> My guess is that installing oslo.sphinx globally (with sudo), set up 2 copies of the namespace package (one in the global dist-packages and presumably one in the virtualenv being used for the tests).

Actually I think it may be the opposite problem, at least where I'm
currently running into this. oslo.sphinx is only installed in the venv
and it creates a namespace package there. Then if you try to load
oslo.config in the venv it looks in the namespace package, doesn't find
it, and bails with a missing module error. 

I'm personally running into this in tempest - I can't even run pep8 out
of the box because the sample config check fails due to missing
oslo.config. Here's what I'm seeing: 

In the tox venv: 
(pep8)[fedora at devstack site-packages]$ ls oslo*
oslo.sphinx-1.1-py2.7-nspkg.pth

oslo:
sphinx

oslo.sphinx-1.1-py2.7.egg-info:
dependency_links.txt namespace_packages.txt PKG-INFO top_level.txt
 installed-files.txt not-zip-safe SOURCES.txt 

And in the system site-packages: 
[fedora at devstack site-packages]$ ls oslo*
oslo.config.egg-link oslo.messaging.egg-link 

Since I don't actually care about oslo.sphinx in this case, I also found
that deleting it from the venv fixes the problem, but obviously that's
just a hacky workaround. My initial thought is to install oslo.sphinx in
devstack the same way as oslo.config and oslo.messaging, but I assume
there's a reason we didn't do it that way in the first place so I'm not
sure if that will work. 

So I don't know what the proper fix is, but I thought I'd share what
I've found so far. Also, I'm not sure if this even relates to the
ceilometer issue since I wouldn't expect that to be running in a venv,
but it may have a similar issue. 

I wonder if the issue is actually that we're using "pip install -e" for
oslo.config and oslo.messaging (as evidenced by the .egg-link files). Do
things work properly if those packages are installed to the global
site-packages from PyPI instead? We don't want to change the way
devstack installs them, but it would give us another data point. 

Another solution is to have a list of dependencies needed for building
documentation, separate from the tests, since oslo.sphinx isn't needed
for the tests. 

It does work if I remove the pip install -e version of oslo.config and
reinstall from the pypi package, so this appears to be an issue with the
egg-links. 

You had already tested installing oslo.sphinx with pip install -e,
right? That's probably the least-wrong answer. Either that or move
oslo.sphinx to a different top level package to avoid conflicting with
runtime code. 

Right. This https://review.openstack.org/#/c/65336/ also fixed the
problem for me, but according to Sean that's not something we should be
doing in devstack either. 

-Ben 
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140108/2591b91a/attachment.html>