Open Stack

On Wed, Jun 24, 2020 at 02:07:06PM +1200, Adrian Turjak wrote:
> On 24/06/20 1:05 am, Thomas Goirand wrote:
> > Anyone dismissing how huge of a problem this is, isn't doing serious
> > programming, for serious production use. That person would just be doing
> > script kiddy work in the playground. Yes, it works, yes, it's shiny and
> > all. The upstream code may even be super nice, well written and all. But
> > it's *NOT* serious to put such JS bundling approach in production.
> And yet people are running huge projects in production like this just
> fine. So clearly people are finding sane practices around it that give
> them enough security to feel safe that don't involve packaging each npm
> requirement as an OS package. How exactly are all the huge powerhouses
> doing it then when most of the internet's front end is giant js bundles
> built from npm dependencies? How does gitlab do it for their omnibus?
> From a cursory glance it did seem like they did use npm, and had a rake
> job to compile the js. Gitlab most definitely isn't "script kiddy work".
> 
> I'm mostly a python dev, so I don't deal with npm often. When it comes
> to python though, other than underlying OS packages for python/pip
> itself, I use pip for installing my versions (in a venv or container).
> I've had too many painful cases of weird OS package versions, and I
> dislike the idea of relying on the OS when there is a perfectly valid
> and working package management system for my application requirements. I
> can audit the versions installed against known CVEs, and because I
> control the requirements, I can ensure I'm never using out of date
> libraries.
> 
> Javascript and npm is only different because the sheer number of
> dependencies. Which is terrifying, don't get me wrong, but you can lock
> versions, you can audit them against CVEs, you can be warned if they are
> out of date. How other than by sheer scale is it really worse than pip
> if you follow some standards and a consistent process?

What Thomas is trying to say, and I think other people in this thread
also agreed with, is that it's not "only" because of the sheer number of
dependencies. My personal opinion is that the Javascript ecosystem is
currently where Perl/CPAN was 25 years ago, Python was between 15 and 20
years ago, and Ruby was 10-15 years ago: quite popular, attracting many
people who "just want to write a couple of lines of code to solve this
simple task", and, as a very logical consequence, full of small
libraries that various people developed to fix their own itches and just
released out into the wild without very much thought of long-term
maintenance. Now, this has several consequences (most of them have been
pointed out already):

- there are many (not all, but many) developers who do not even try to
  keep their own libraries backwards-compatible

- there are many (not all, but many) developers who, once they have
  written a piece of code that uses three libraries from other people,
  do not really bother to follow the development of those libraries and
  try to make their own piece of code compatible with their new versions
  (this holds even more if there are not three, but fifteen libraries
  from other people; it can be a bit hard to keep up with them all if
  their authors do not care about API stability)

- there are many libraries that lock the versions of their dependencies,
  thus bringing back what was once known as "DLL hell", over and over
  and over again (and yes, this happens in other languages, too)

- there are many, many, *many* libraries that solve the same problems
  over and over again in subtly different ways, either because their
  authors were not aware of the other implementations or because said
  other implementations could not exactly scratch the author's itch and
  it was easier to write their own instead of spend some more time
  trying to adapt the other one and propose changes to its author
  (and, yes, I myself have been guilty of this in C, Perl, and Python
  projects in the past; NIH is a very, very easy slope to slide down
  along)

I *think* that, with time, many Javascript developers will realize that
this situation is unsustainable in the long term, and, one by one, they
will start doing what C/C++, Perl, Python, and Ruby people have been
doing for some time now:

- start thinking about backwards compatibility, think really hard before
  making an incompatible change and, if they really have to, use
  something like semantic versioning (not necessarily exactly semver,
  but something similar) to signal the API breakage

- once the authors of the libraries they depend on start doing this,
  start encoding loose version requirements (not strictly pinned), such
  as "dep >= 1.2.1, dep < 3". This is already done in many Python
  packages, and OpenStack's upper-constraints machinery is a wonderful
  example of how this can be maintained in a conservative manner that
  virtually guarantees that the end result will work.

- start wondering whether it is really worth it to maintain their own
  pet implementation instead of extending a more-widely-used one, thus
  eventually having the community settle on a well-known set of
  more-or-less comprehensive and very widely tested packages for most
  tasks. Once this happens, the authors of these widely-used libraries
  absolutely *have* to keep some degree of backwards compatibility and
  some kind of reasonable versioning scheme to signal changes.

So, I'm kind of optimistic and I believe that, with time, the Javascript
ecosystem will become better. Unfortunately, this process has taken many
years for the other languages I've mentioned, and is not really fully
complete in any of them: any module repository has its share of
mostly-maintained reimplementations of various shapes and sizes of the
wheel. So I guess the point of all this was mostly to explain the
problem (once again) more than propose any short-term solutions :/

G'luck,
Peter

-- 
Peter Pentchev  roam at ringlet.net roam at debian.org pp at storpool.com
PGP key:        http://people.FreeBSD.org/~roam/roam.key.asc
Key fingerprint 2EE7 A7A5 17FC 124C F115  C354 651E EFB0 2527 DF13
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20200628/fa1c31e5/attachment.sig>