Open Stack

On 11/14/2014 06:40 AM, Donald Stufft wrote:
>>>> Sure! That's how I do most of my Python modules these days. I don't just
>>>> create them from scratch, I use my own "debpypi" script, which generates
>>>> a template for packaging. But it can't be fully automated. I could
>>>> almost do it in a fully automated manner for PEAR packages for PHP (see
>>>> "debpear" in the Debian archive), but it's harder with Python and pip/PyPi.
>>>
>>> I would be interested to know what makes Python harder in this regard, I
>>> would like to fix it.
>>
>> The fact that the standard from PyPi is very fuzzy is one of the issue.
>> There's nothing in the format (for example in the DOAP.xml record) that
>> tells if a module supports Python3 for example. Then the short and long
>> descriptions aren't respected, often, you get some changelog entries
>> there. Then there's no real convention for the location of the sphinx
>> doc. There's also the fact that dependencies for Python have to be
>> written by hand on a Debian package. See for example, dependencies on
>> arparse, distribute, ordereddict, which I never put in a Debian package
>> as it's available in Python 2.7. Or the fact that there's no real unique
>> place where dependencies are written on a PyPi "package" (is it hidden
>> somewhere in setup.py, or is it explicitly written in
>> requirements.txt?). Etc. On the PHP world, everything is much cleaner,
>> in the package.xml, which is very easily parse-able.
> 
> (This is fairly off topic, so if you want to reply to this in private that’s
> fine):

Let's just change the subject line, so that those not interested in the
discussion can skip the topic entirely.

> Nothing that says if it supports py3:
>     Yea, this is a problem, you can somewhat estimate it using the Python 3
>     classifier though.

The issue is that this is a not-mandatory tag. And often, it isn't set.

> Short and Long descriptions aren’t respected:
>     I’m not sure what you mean by isn’t respected?

On my templating script, I grab what's supposed to be the short and long
description. But this leads to importing some RST format long
description that do include unrelated things. In fact, I'm not even sure
there's such things as long and short desc in the proper way, so that it
could just be included in debian/control without manual work.

> Have to write dependencies by hand:
>     Not sure what you mean by not depending on argparse, distribute, ordereddict,
>     etc? argparse and order edict are often depended on because of Python 2.6,

Right. I think this is an issue in Debian: we should have had a
Provides: in python 2.7, so that it wouldn't have mater. I just hope
this specific issue will just fade away as Python 2.6 gets older and
less used.

>     setuptools/distribute should only be dependended on if the project is using
>     entry points or something similar.

If only everyone was using PBR... :)

> No unique place where dependencies are written:
>     If the project is using setuptools (or is usable from pip) then dependencies
>     should be inside of the install_requires field in the setup.py. I can send
>     some code for getting this information. Sadly it’s not in a static form yet
>     so it requires executing the setup.py.

Executing blindly setup.py before I can inspect it would be an issue.
However, yes please, I'm curious on how to extract the information, so
please do send the code!

>> No, that's for arch independent *things*. Like for example, javascript.
>> In Debian, these are going in /usr/share/javascript. Python code used to
>> live within /usr/share/pyshared too (but we stopped the symlink forest
>> during the Jessie cycle).
> 
> Why does the FHS webpage say differently?
> 
> From [1]:
> 
>     The /usr/share hierarchy is for all read-only architecture independent data files.

Which is exactly what I wrote. Oh, maybe it's the "data files" that
bothers you? Well, in some ways, javascript can be considered as data
files. But let's take another example. PHP, java and perl library files
are all stored into /usr/share as well (though surprisingly, ruby is in
/usr/lib... but maybe because it also integrates compiled-in .so files).

>>> I believe it also states that
>>> /usr/lib is for object files, libraries, and internal binaries.
>>
>> It's for arch dependent things.
> 
> Why does the FHS webpage say differently?
> 
> From [2]:
> 
>     /usr/lib includes object files, libraries, and internal binaries that are not
>     intended to be executed directly by users or shell scripts.

That's nothing that goes against what I wrote. "object files, libraries,
and internal binaries" are all arch-dependent things if you know how to
read between the lines, especially if you know that /usr/share is for
"architecture independent" stuff.

Thomas