Open Stack

On 06/29/2015 12:01 PM, Victor Stinner wrote:
> Hi,
> 
> Le 29/06/2015 11:03, Thomas Goirand a écrit :
>> cliff-tablib is used for the unit tests of things like
>> python-neutronclient. The annoying bit is that cliff-tablib depends on
>> tablib, which itself is a huge mess. It has loads of 3rd party embedded
>> packages and most of them aren't Python 3.x compatible.
> 
> tablib includes copies of various dependencies in its tablib/packages/
> directory. Some of them are for Python 2, others are for Python 3.
> 
> It would be better to use dependencies (requirements in setup.py), not
> copies.

Yes!

> Do you try to contact tablib authors to ask them to remove
> completly tablib/packages/?

I haven't yet. Though some of the dependencies are simply not Py3
compatible at all, full stop. So switching to them as independent
package wouldn't help me much: we'd still have to do the work of porting
them to Py3.

> setup.py uses a different list of packages on Python 2 and Python 3.
> 
> I tried "python3 setup.py install": the bytecode compilation of
> markup.py fails with an obvious SyntaxError, the code is for Python 2.
> But there is also markup3.py which is compiled successfully.
> 
> Even if the compilation of the markup.py fails, "python setup.py
> install" succeed with the exit code 0. What is your problem?

As I wrote to you on IRC, if the compilation of the code fails, then
this breaks the world when installing a Python package (the .py are
built as .pyc at install time using "pycompile -p <package-name>" in the
postinst of all python packages). So having "python setup.py install"
that works isn't enough for Debian/Ubuntu.

> setup.py should be fixed to skip markup.py on Python 3, and skip
> markup3.py on Python 2. A workaround is to remove manually the file
> depending on the Python major version.

Then will it magically know which one to use? Also, markup{3,}.py isn't
the only issue. I gave you a list of files which have compile issues.
Digging more into it, it's hell...

> Note: "pip install tablib" works on Python 3 (pip uses the binary wheel
> package).

This doesn't help me, unfortunately. It's like if you were saying "it
worked on devstack"...

> cliff-tablib is used in tests. If you remove the cliff-tablib
> dependency, tests will obviously fail.

Yes, that's the issue.

> What do you propose? Modify tests to reimplement cliff-tablib? Remove
> tests?

Some of the above, yes. Or at least, in the short term, make the tests
skipped if cliff-tablib isn't installed, so that I can completely ignore
these tests and remove cliff-tablib from Debian. Long term: get rid of
cliff-tablib and tablib completely if we can.

On 06/29/2015 01:39 PM, Doug Hellmann wrote:
> tablib is managed by Kenneth Reitz, and as with his requests library
> he feels vendoring is the best way to distribute dependencies. For
> a while I've had a to-do on my list to rewrite those formatters to
> not use tablib, it just hasn't been a high priority.

Oh! I didn't notice it was form him. Here, we see how much vendoring is
hurting in a *very* bad way. And give the answers about de-vendoring
requests, we can already guess that asking any kinds of fixing of the
situation will be pure time loss.

I don't know much about tablib internals, I just see how badly it is
"packaged" on PyPi. If you could get a replacement for it, it'd be awesome.

Proposal: get a list of stubborn upstream having potentially dangerous
behavior, and blacklist them completely (or, worst case, fork their
project if we can afford it). I have also Gabriel Falcao as a potential
candidate for such blacklist (author of sure, httpretty, steadymark, and
more...), because he has the dangerous habit to do API breakage which
has already broke the gate multiple times.

Cheers,

Thomas Goirand (zigo)