Open Stack

Thomas Goirand <zigo at debian.org> writes:

> On 11/13/2014 10:56 PM, Martin Geisler wrote:
>> Maybe a silly question, but why insist on this? Why would you insist on
>> installing a JavaScript based application using your package manager?
>> 
>> I'm a huge fan of package managers and typically refuse to install
>> anything globally if it doesn't come as a package.
>> 
>> However, the whole JavaScript ecosystem seems to be centered around the
>> idea of doing local installations. That means that you no longer need
>> the package manager to install the software -- you only need a package
>> manager to install the base system (NodeJs and npm for JavaScript).
>
> Yeah... Just like for Java, PHP, Perl, Python, you-name-it...
>
> In what way Javascript will be any different from all of these
> languages?

Let me again say that I'm fairly new to this modern JavaScript world. I
knew almost nothing about node, npm, bower, and grunt six months ago.

One answer may be that there isn't an expectation in the JavaScript
community that you'll be reusing system libraries the same way that
there is in at least the Python and Perl communities.

It's my impression that the JavaScript world is used to move *very*
quickly. People release versions very rapidly and are happy to break
APIs. I think they're okay with it because of semver -- the idea that as
long as you increment the right digit in your version number, you don't
have to care (much) about the work you put on your users when you ask
them to upgrade.

I hope that's too harsh a description, but the way the node module
system is explicitly designed to allow a single running program to use
multiple versions of the *same* module hints that this chaotic situation
is both expected and considered normal.

When reading about the module system, I came across blog posts that
called it superior compared to, say, Python, exactly because of this
flexibility. Reasonable people are sure to disagree, but that seems to
be the current situation.

>> Notice that Python has been moving rapidly in the same direction for
>> years: you only need Python and pip to bootstrap yourself. After getting
>> used to virtualenv, I've mostly stopped installing Python modules
>> globally and that is how the JavaScript world expects you to work too.
>
> Fine for development. Not for deployments. Not for distributions. Or you
> just get a huge mess of every library installed 10 times, with 10
> different versions, and then a security issue needs to be fixed...

While I agree that it's chaotic, I also think you make the problem worse
than it really is. First, remember that the user who installs Horizon
won't need to use the JavaScript based *developer* tools such as npm,
bower, etc.

That is, I think Horizon developers will use these tools to produce a
release -- a tarball -- and that tarball will be something you unpack on
your webserver and then you're done. I base this on what I've seen in
the project I've been working. The release tarball you download here
don't mention npm, bower, or any of the other tools:

  https://github.com/zerovm/swift-browser/releases

The tools were used to produce the tarball and were used to test it, but
they're not part of the released product. Somewhat similar to how GCC
isn't included in the tarball if you download a pre-compiled binary.

>> So maybe the Horizon package should be an installer package like the
>> ones that download fonts or Adobe?
>
> This is a horrible design which will *never* make it to distributions.
> Please think again. What is it that makes Horizon so special? Answer:
> nothing. It's "just a web app", so it doesn't need any special care.
> It should be packaged, just like the rest of everything, with
> .deb/.rpm and so on.

Maybe a difference is that you don't (yet) install a web application
like you install a system application. Instead you *deploy* it: you
unpack files on a webserver, you configure permissions, you setup cache
rules, you configure a database, etc.

I find this to be quite different from, say, installing Emacs. Emacs is
something you install once on a system and this single installation can
be done in a "right way" so that it's useable for several users on the
system.

A web app is something a single user installs on a system (www-data or a
similar user) and then this user configures the system web server to
serve this web app.

I agree that it would be cool to have web apps be as robust and general
purpose as system apps. However, I think that day is a ways off.

>> That package would get the right version of node and which then runs
>> the npm and bower commands to download the rest plus (importantly and
>> much appreciated) puts the files in a sensible location and gives
>> them good permissions.
>
> Fine for your development environment. But that's it.
>
> Also, does your $language-specific-package--manager has enough checks
> so that there's no man in the middle attack possible? Is it secured
> enough? Can a replay attack be done on it? Does it supports any kind
> of cryptography checks like yum or apt does? I'm almost sure that's
> not the case. pip is really horrible in this regard. I haven't
> checked, but I'm almost sure what we're proposing (eg: npm and such)
> have the same weakness. And here, I'm only scratching security
> concerns.

You're probably right that this infrastructure isn't there.

> There's other concerns, like how good is the dependency solver and
> such (remember: it took *years* for apt to be as good as it is right
> now, and it still has some defects).

The dependency solver is as good as the community needs it to be. Or put
differently, if the JavaScript community is able to produce working
software with npm, then they obviously produce it within the bounds of
the capabilities of its dependency solver.

I'm happy to believe that apt has a top-notch and highly tuned
dependency solver. That doesn't really matter since it would be solving
problems we don't have.

> On 11/14/2014 12:59 AM, Martin Geisler wrote:
>> It seems to me that it should be possible translate the node module
>> into system level packages in a mechanical fashion, assuming that
>> you're willing to have a system package for each version of the node
>> module
>
> Sure! That's how I do most of my Python modules these days. I don't
> just create them from scratch, I use my own "debpypi" script, which
> generates a template for packaging. But it can't be fully automated. I
> could almost do it in a fully automated manner for PEAR packages for
> PHP (see "debpear" in the Debian archive), but it's harder with Python
> and pip/PyPi.

It's my impression that node modules are very self-contained. They are
unpacked into node_modules and that's it -- perhaps a symlink is added
to node_modules/.bin.

I haven't looked closely at this, though, but that's the behavior I've
seen from the packages I've installed so far.

> Stuff like debian/copyright files have to be processed by hand, and
> each package is different (How to run unit tests? nose, testr, pytest?
> Does it support python3? Is there a sphinx doc? How good is upstream
> short and long description?). I guess it's going to be the same for
> Javascript packages: it will be possible to do automation for some
> parts, but manual work will always be needed.

In my view, you're taking on way too much work by going into those
details. I don't think I need or want you do to anything more than
repack the tarball that npm retrieves -- I don't think you should run
tests or generate documentation.

As a user or sysadmin, I would be happy to add a deb line to my
sources.list and get Debian packages that wrap the node modules. I'll
lookup the documentation online as needed.

The packages would also have to add dependencies on other wrapped
packages -- these dependencies would be read from the package.json file.
If done recursively, you'll end up with wrapped packages for all of the
node modules needed.

-- 
Martin Geisler

http://google.com/+MartinGeisler
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141113/3ed1abd3/attachment.pgp>