Open Stack

On Sep 25, 2014, at 6:14 PM, Robert Collins <robertc at robertcollins.net> wrote:

> On 24 September 2014 11:03, Robert Collins <robertc at robertcollins.net> wrote:
> 
>> So... FWIW I think I've got a cleaner implementation of namespaces
>> *for our context* - it takes inspiration from the PEP-420 discussion
>> and final design. It all started when Mike reported issues with testr
>> to me.
>> 
>> https://bugs.launchpad.net/oslo.db/+bug/1372250
>> 
>> tl;dr: we should stop using pkg_resources style namespace packages and
>> instead have an effectively empty oslo package that sets up the
>> namespace, which all namespaced libraries would depend on. With a stub
>> __init__ in local source trees that adds the site-packages path to
>> itself automatically, and excluding that file in sdist, it should be
>> entirely transparent to developers and packagers, with no file
>> conflicts etc.
>> 
>> This works with the existing pkg_resources namespace packages, lets us
>> migrate away from the pkg_resources implementation one package at a
>> time, and we don't need to rename any of the packages, and it works
>> fine with uninstalled and install -e installed source trees.
>> 
>> We need:
>> - a new olso package to introduce a common oslo/__init__.py
>> (recommended in the pre-PEP420 world)
>> - a tiny pbr bugfix: https://review.openstack.org/123597
>> - and a patch like so to each project: https://review.openstack.org/123604
>> 
>> I have such an olso package https://github.com/rbtcollins/oslo, if
>> this sounds reasonable I will push up an infra patch to create it.
> 
> Doug raised on IRC a concern about system-site-packages.
> 
> I have tested this, and I can make it work, but I'm not sure its
> needed: it is totally broken today:
> ----
> # Put oslo.config in the system site and oslo.18n not yet installed
> sudo apt-get install oslo.config
> sudo apt-get remove oslo.i18n
> # make a virtualenv with system site packages
> mkvirtualenv --system-site-packages test-system-site
> # install oslo.18n
> pip install oslo.i18n
> # now when I tested oslo.i18n doesn't depent on oslo.config, but lets be sure:
> python -c 'import oslo.config; print oslo.config.__file__'
> Traceback (most recent call last):
> File "<string>", line 1, in <module>
> ImportError: No module named config
> 
> # Oh look! can't import oslo.config.
> pip install oslo.db
> ...
> python -c 'import oslo.config; print oslo.config.__file__'
> /home/robertc/.virtualenvs/test-system-site/local/lib/python2.7/site-packages/oslo/config/__init__.pyc
> 
> # Now we need it, it got pulled in.
> ----
> 
> Now, as I say, I can fix this quite easily with a virtualenv aware pth
> file, but since its broken today and AFAIK there isn't a bug open
> about this, I think it will be fine.

The breaking case was having a package installed editable in the global site-packages and then installing another sub-package non-editable in the virtualenv. We stopped doing that by making devstack install oslo libraries non-editable, but I think, based on what you’re saying, if the test environment has access to the global site-packages and only some of the oslo libs are installed then building the test environment will fail if it needs another lib because it won’t install the packages it can see globally. Is that right? Has anyone encountered that situation?


> 
> When you install e.g. oslo.db which *does* depend on oslo.config,
> oslo.config is being installed within the venv. I'm not sure if thats
> strictly due to version constraints, or if its systemic.
> 
> So - I'd like to say that its a separate preexisting issue and we can
> loop back and tackle it should it show up as a problem.
> 
> -Rob
> 
> -- 
> Robert Collins <rbtcollins at hp.com>
> Distinguished Technologist
> HP Converged Cloud
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev