Open Stack

Donald linked to a pip bug later in this thread, so we might be able
to help by working on a fix. I haven't investigated that, but I assume
if it was easy the pypa team would have already fixed it.

When you saw the problem, were you running a current version of
devstack with Sean's change to install oslo libraries via "pip"
instead of "pip -e"?

Doug


On Mon, Apr 7, 2014 at 4:34 PM, Vishvananda Ishaya
<vishvananda at gmail.com> wrote:
> I dealt with this myself the other day and it was a huge pain. That said,
> changing all the packages seems like a nuclear option. Is there any way
> we could change python that would make it smarter about searching multiple
> locations for namespace packages?
>
> Vish
>
> On Apr 7, 2014, at 12:24 PM, Doug Hellmann <doug.hellmann at dreamhost.com> wrote:
>
>> Some of the production Oslo libraries are currently being installed
>> into the "oslo" namespace package (oslo.config, oslo.messaging,
>> oslo.vmware, oslo.rootwrap, and oslo.version). Over the course of the
>> last 2 release cycles, we have seen an increase in the number of
>> developers who end up with broken systems, where an oslo library (most
>> often oslo.config) cannot be imported. This is usually caused by
>> having one copy of a library installed normally (via a system package
>> or via pip) and another version in "development" (a.k.a., "editable")
>> mode as installed by devstack. The symptom is most often an error
>> about importing oslo.config, although that is almost never the library
>> causing the problem.
>>
>> We have already worked around this issue with the non-production
>> libraries by installing them into their own packages, without using
>> the namespace (oslotest, oslosphinx, etc.). We have also changed the
>> way packages are installed in nova's tox.ini, to force installation of
>> packages into the virtualenv (since exposing the global site-packages
>> was a common source of the problem). And very recently, Sean Dague
>> changed devstack to install the oslo libraries not in editable mode,
>> so that installing from source should replace any existing installed
>> version of the same library.
>>
>> However, the problems seem to persist, and so I think it's time to
>> revisit our decision to use a namespace package.
>>
>> After experimenting with non-namespace packages, I wasn't able to
>> reproduce the same import issues. I did find one case that may cause
>> us some trouble, though. Installing a package and then installing an
>> editable version from source leaves both installed and the editable
>> version appears first in the import path. That might cause surprising
>> issues if the source is older than the package, which happens when a
>> devstack system isn't updated regularly and a new library is released.
>> However, surprise due to having an old version of code should occur
>> less frequently than, and have less of an impact than, having a
>> completely broken set of oslo libraries.
>>
>> We can avoid adding to the problem by putting each new library in its
>> own package. We still want the Oslo name attached for libraries that
>> are really only meant to be used by OpenStack projects, and so we need
>> a naming convention. I'm not entirely happy with the "crammed
>> together" approach for oslotest and oslosphinx. At one point Dims and
>> I talked about using a prefix "oslo_" instead of just "oslo", so we
>> would have "oslo_db", "oslo_i18n", etc. That's also a bit ugly,
>> though. Opinions?
>>
>> Given the number of problems we have now (I help about 1 dev per week
>> unbreak their system), I think we should also consider renaming the
>> existing libraries to not use the namespace package. That isn't a
>> trivial change, since it will mean updating every consumer as well as
>> the packaging done by distros. If we do decide to move them, I will
>> need someone to help put together a migration plan. Does anyone want
>> to volunteer to work on that?
>>
>> Before we make any changes, it would be good to know how bad this
>> problem still is. Do developers still see issues on clean systems, or
>> are all of the problems related to updating devstack boxes? Are people
>> figuring out how to fix or work around the situation on their own? Can
>> we make devstack more aggressive about deleting oslo libraries before
>> re-installing them? Are there other changes we can make that would be
>> less invasive?
>>
>> Doug
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>