Open Stack

On 11/14/2014 02:11 AM, Donald Stufft wrote:
>> On Nov 13, 2014, at 12:38 PM, Thomas Goirand <zigo at debian.org> wrote:
>> On 11/13/2014 10:56 PM, Martin Geisler wrote:
>>> However, the whole JavaScript ecosystem seems to be centered around the
>>> idea of doing local installations. That means that you no longer need
>>> the package manager to install the software -- you only need a package
>>> manager to install the base system (NodeJs and npm for JavaScript).
>>
>> Yeah... Just like for Java, PHP, Perl, Python, you-name-it...
>>
>> In what way Javascript will be any different from all of these languages?
> 
> Node.js, and npm in particular tends to solve the dependency hell problem
> by making it possible to install multiple versions of a particular thing
> and use them all within the same process. As far as I know the OS tooling
> doesn’t really handle SxS installs of the same thing in multiple versions
> very well, I think the closest that you can do is multiple separate packages
> with version numbers in the package name?

Yeah, and for a very good reason: having multiple version of the same
thing is just really horrible, and should be avoided at all costs.

>> Also, does your $language-specific-package--manager has enough checks so
>> that there's no man in the middle attack possible? Is it secured enough?
>> Can a replay attack be done on it? Does it supports any kind of
>> cryptography checks like yum or apt does? I'm almost sure that's not the
>> case. pip is really horrible in this regard. I haven't checked, but I'm
>> almost sure what we're proposing (eg: npm and such) have the same
>> weakness. And here, I'm only scratching security concerns. There's other
>> concerns, like how good is the dependency solver and such (remember: it
>> took *years* for apt to be as good as it is right now, and it still has
>> some defects).
> 
> As far as I’m aware npm supports TLS the same as pip does. That secures the
> transport between the end users and the repository so you can be assured
> that there is no man in the middle. Security wise npm (and pip) are about
> ~95% (mad up numbers, but you can get the gist) of the effectiveness as the
> OS package managers.

I don't agree at all with this view. Using TLS is *far* from being
enough IMO. But that's not the point. Using anything else than the
distribution package manager is a hack (or unfinished work).

>> On 11/14/2014 12:59 AM, Martin Geisler wrote:
>>> It seems to me that it should be possible translate the node module
>>> into system level packages in a mechanical fashion, assuming that
>>> you're willing to have a system package for each version of the node
>>> module
>>
>> Sure! That's how I do most of my Python modules these days. I don't just
>> create them from scratch, I use my own "debpypi" script, which generates
>> a template for packaging. But it can't be fully automated. I could
>> almost do it in a fully automated manner for PEAR packages for PHP (see
>> "debpear" in the Debian archive), but it's harder with Python and pip/PyPi.
> 
> I would be interested to know what makes Python harder in this regard, I
> would like to fix it.

The fact that the standard from PyPi is very fuzzy is one of the issue.
There's nothing in the format (for example in the DOAP.xml record) that
tells if a module supports Python3 for example. Then the short and long
descriptions aren't respected, often, you get some changelog entries
there. Then there's no real convention for the location of the sphinx
doc. There's also the fact that dependencies for Python have to be
written by hand on a Debian package. See for example, dependencies on
arparse, distribute, ordereddict, which I never put in a Debian package
as it's available in Python 2.7. Or the fact that there's no real unique
place where dependencies are written on a PyPi "package" (is it hidden
somewhere in setup.py, or is it explicitly written in
requirements.txt?). Etc. On the PHP world, everything is much cleaner,
in the package.xml, which is very easily parse-able.

>> On 11/14/2014 12:59 AM, Martin Geisler wrote:
>>> The guys behind npm has written a little about how that could work
>>> here:
>>>
>>> http://nodejs.org/api/modules.html#modules_addenda_package_manager_tips
>>
>> It's fun to read, but very naive. First thing that is very shocking is
>> that arch independent things gets installed into /usr/lib, where they
>> belong in /usr/share. If that is what the NPM upstream produces, that's
>> scary: he doesn't even know how the FSHS (FileSystem Hierarchy Standard)
>> works.
> 
> I may be wrong, but doesn’t the FHS state that /usr/share is for arch
> independent *data* that is read only?

No, that's for arch independent *things*. Like for example, javascript.
In Debian, these are going in /usr/share/javascript. Python code used to
live within /usr/share/pyshared too (but we stopped the symlink forest
during the Jessie cycle).

> I believe it also states that
> /usr/lib is for object files, libraries, and internal binaries.

It's for arch dependent things.

> As far as
> I’m aware the things that npm installs are libraries the same as what
> pip installs and should go under /usr/lib yea?

The difference is that Python code gets compiled into .pyc, so they
should go into /usr/lib, and potentially also use multiarch if it's
compiled C code.

Thomas