Hi, We can discuss Horizon v next today during our mid-cycle call: http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016346.ht... Regards, Ivan Kolodyazhny, http://blog.e0ne.info/ On Sun, Jun 28, 2020 at 5:03 PM Peter Pentchev <roam@ringlet.net> wrote:
On Wed, Jun 24, 2020 at 02:07:06PM +1200, Adrian Turjak wrote:
Anyone dismissing how huge of a problem this is, isn't doing serious programming, for serious production use. That person would just be doing script kiddy work in the playground. Yes, it works, yes, it's shiny and all. The upstream code may even be super nice, well written and all. But it's *NOT* serious to put such JS bundling approach in production. And yet people are running huge projects in production like this just fine. So clearly people are finding sane practices around it that give
On 24/06/20 1:05 am, Thomas Goirand wrote: them enough security to feel safe that don't involve packaging each npm requirement as an OS package. How exactly are all the huge powerhouses doing it then when most of the internet's front end is giant js bundles built from npm dependencies? How does gitlab do it for their omnibus? From a cursory glance it did seem like they did use npm, and had a rake job to compile the js. Gitlab most definitely isn't "script kiddy work".
I'm mostly a python dev, so I don't deal with npm often. When it comes to python though, other than underlying OS packages for python/pip itself, I use pip for installing my versions (in a venv or container). I've had too many painful cases of weird OS package versions, and I dislike the idea of relying on the OS when there is a perfectly valid and working package management system for my application requirements. I can audit the versions installed against known CVEs, and because I control the requirements, I can ensure I'm never using out of date libraries.
Javascript and npm is only different because the sheer number of dependencies. Which is terrifying, don't get me wrong, but you can lock versions, you can audit them against CVEs, you can be warned if they are out of date. How other than by sheer scale is it really worse than pip if you follow some standards and a consistent process?
What Thomas is trying to say, and I think other people in this thread also agreed with, is that it's not "only" because of the sheer number of dependencies. My personal opinion is that the Javascript ecosystem is currently where Perl/CPAN was 25 years ago, Python was between 15 and 20 years ago, and Ruby was 10-15 years ago: quite popular, attracting many people who "just want to write a couple of lines of code to solve this simple task", and, as a very logical consequence, full of small libraries that various people developed to fix their own itches and just released out into the wild without very much thought of long-term maintenance. Now, this has several consequences (most of them have been pointed out already):
- there are many (not all, but many) developers who do not even try to keep their own libraries backwards-compatible
- there are many (not all, but many) developers who, once they have written a piece of code that uses three libraries from other people, do not really bother to follow the development of those libraries and try to make their own piece of code compatible with their new versions (this holds even more if there are not three, but fifteen libraries from other people; it can be a bit hard to keep up with them all if their authors do not care about API stability)
- there are many libraries that lock the versions of their dependencies, thus bringing back what was once known as "DLL hell", over and over and over again (and yes, this happens in other languages, too)
- there are many, many, *many* libraries that solve the same problems over and over again in subtly different ways, either because their authors were not aware of the other implementations or because said other implementations could not exactly scratch the author's itch and it was easier to write their own instead of spend some more time trying to adapt the other one and propose changes to its author (and, yes, I myself have been guilty of this in C, Perl, and Python projects in the past; NIH is a very, very easy slope to slide down along)
I *think* that, with time, many Javascript developers will realize that this situation is unsustainable in the long term, and, one by one, they will start doing what C/C++, Perl, Python, and Ruby people have been doing for some time now:
- start thinking about backwards compatibility, think really hard before making an incompatible change and, if they really have to, use something like semantic versioning (not necessarily exactly semver, but something similar) to signal the API breakage
- once the authors of the libraries they depend on start doing this, start encoding loose version requirements (not strictly pinned), such as "dep >= 1.2.1, dep < 3". This is already done in many Python packages, and OpenStack's upper-constraints machinery is a wonderful example of how this can be maintained in a conservative manner that virtually guarantees that the end result will work.
- start wondering whether it is really worth it to maintain their own pet implementation instead of extending a more-widely-used one, thus eventually having the community settle on a well-known set of more-or-less comprehensive and very widely tested packages for most tasks. Once this happens, the authors of these widely-used libraries absolutely *have* to keep some degree of backwards compatibility and some kind of reasonable versioning scheme to signal changes.
So, I'm kind of optimistic and I believe that, with time, the Javascript ecosystem will become better. Unfortunately, this process has taken many years for the other languages I've mentioned, and is not really fully complete in any of them: any module repository has its share of mostly-maintained reimplementations of various shapes and sizes of the wheel. So I guess the point of all this was mostly to explain the problem (once again) more than propose any short-term solutions :/
G'luck, Peter
-- Peter Pentchev roam@ringlet.net roam@debian.org pp@storpool.com PGP key: http://people.FreeBSD.org/~roam/roam.key.asc Key fingerprint 2EE7 A7A5 17FC 124C F115 C354 651E EFB0 2527 DF13