Open Stack

On 11/12/2014 11:12 PM, Jiri Tomasek wrote:
> As Monty Taylor said, nodejs itself is not a blocker as multiple
> versions of it should not be needed by our tools. (That's also what npm
> and bower are taking care of, right?) Only thing that is required is
> that all tools/js libs we want to use would eventually have to be
> packaged. This is just a bunch of work for packagers.
> 
> Approach on using Xstatic packages vs Js tooling:
> 
> As only problem with using js tooling should be just actual packaging of
> it, I think it makes sense to use these tools and make development
> simpler then going other way around and using Xstatic packages - which
> means devs would have to care about getting stuff packaged as xstatic
> and added to the code, while maintaining proper versions and making sure
> that they work ok together. NPM and Bower do this for us. Common sense
> tells me packagers should take care of packaging.
> Packaging of these tools will have to get resolved somehow anyway, as
> there will be rise in requirements of using them not just from Horizon...

I see an obvious issue with your approach.

Just like with pip and PyPi, it's going to be just one line of patch for
Horizon people to add "yet-another-javascript-dependency". It's already
a dependency hell. I have created 21 new python-xstatic packages in
Debian, and at least, for half a dozen of them (maybe more), I also
packaged the libjs-* packages. It took a long time to have them in
order, and it just got stabilized for Juno (there's still something to
fix in Jessie, but I don't really care as it's Icehouse there...).

Now, your motivation to go into the direction of using tooling seems to
be motivated so that it's super easy to add more. And more... And more
again. This leads me to believe that it's going to be even more crazy.
When is this going to stop? We're definitively enlarging the potential
surface of attack and potential security breaches. And OpenStack at
large is not at all doing great in terms of security. [1]

I just don't understand why all of this is needed. I did some JS
programming myself back in the days (10 years ago, using PHP...). I
could do Ajax by hand, without using a single library. What is it that
you're doing in Horizon that needs so many libs? When I see Horizon in
Juno, I don't even get it. Or is it because you just want to have fancy
animation? Frankly, I don't care these. I care a lot more that we're not
adding new potential security problems.

On 11/13/2014 12:18 AM, Julie Pichon wrote:
> There are instructions already on how to create xstatic packages [1],
> it's not very complicated and just takes some review time.

Exactly!!!

And if someone can't make the effort to package something as an XStatic
lib, then he doesn't deserve the rights to add a new libjs depends on
the project. :)

Also, why do we need to:
1/ Change our procedure starting on each new cycle
2/ Constantly reinvent the wheel
3/ Make my life miserable... :)

On 11/13/2014 01:23 AM, Jiri Tomasek wrote:
> I might have expressed myself wrong about XStatic packages. But as you
> say as well, to use XStatic packages, we need to most often create
> them and maintain the correct versions we require in Horizon and they
> don't help to packagers either.

They do! Well, except Ubuntu guys who found it funny to re-embed all the
Javascript back in Horizon, but for all sane people doing packaging in
this world, it's very helpful!!!

> Similar thing is npm and bower doing on the Angular side except for we
> don't have to create these libraries as they already exist. NPM and
> Bower are taking care of including the right versions of js libs our
> dev env and our application needs. They use similar description files
> as requirements.txt in Django.

How are you going to express the changes in the global-requirements.txt?
Is this just going to live within Horizon? How about projects like
tuskar-ui, or murano-dashboard? There simply wont be a single unique
place for me to look at to see what work I need to process. And no
unique place either to have an overview and see how bad we're doing in
terms of dependency (ie: we have a way too many...).

> It makes no sense not to use them and try to include js libraries
> using XStatic packages and listing them in requirements.txt because
> this way we don't know which version of js lib to use etc.

Not correct. XStatic package versions are following the one of upstream
libjs packages. For example, python-xstatic-jsencrypt in Debian has
version 2.0.0.2, and depends on version 2.0.0 of jsencrypt. If there's
evolution of the dependency, then the XStatic package version will also
change. I think it's very easy to follow, and a very good idea.

The thing is, using all these NodeJS stuff, we'll end-up depending on
Node so much, that it's going to pull about a 100 more dependencies (or
even more). Maybe it's going to be only build-depends, but that's the
same from my point of view: it's going to be packages that we'll need to
take care of in the distributions, while it's not needed with XStatic
packages.

Last, the very good thing that XStatic packages offer for distributions,
is the possibility to *not* package libjs packages separately, and just
use whatever the XStatic package embeds. I did that for bootstrap 3,
because I didn't want to do the packaging work for the ruby part of it
(which I didn't want to care / maintain). I think that's a very nice
feature, which I would like to keep. With your tooling method, this wont
be possible anymore.

On 11/12/2014 09:35 PM, Monty Taylor wrote:
> That a bunch of javascript libraries will need to be distro pacakged
> should not be a blocker (although I don't think that anyone is saying
> it is) That is, after all, the important work the distros do. At this
> point, given the popularity of javascript and javascript tooling, I'm
> pretty sure the problem is going to have to be solved at some point.

There's no way the NodeJS and javascript library version and dependency
hell can be solved anytime soon. Have you ever consider that we have
about 90 reverse dependencies on jquery in Debian? Guess what happens if
we upgrade to the latest version... Is OpenStack as a project ready to
make the transitions at the same time as the distributions for these
libraries? If that's the path we're choosing, I hope so. And I hope also
that I wont be the only one working on these issues (just like I did
with the Django 1.7 transition).

On this last release cycle, Horizon has been, by large, the most
demanding project of all of OpenStack for me. I just hope this doesn't
happen again, because that way, I can't focus on other important things
(like ease of deployment, validating with tempest using packages, etc.).

On 11/12/2014 09:31 PM, Monty Taylor wrote:
>> jshint is NOT free software.
>> >
>> > https://github.com/jshint/jshint/blob/master/src/jshint.js#L19
> Reasonable people disagree on this point.

Feel free to have this debate with the entire Debian community. When
you're done, then come back to us, and we can use jshint. In the mean
while, let's not use it. (and by the way, read again the Debian Free
Software Guideline and especially point number 5 and 6, I'm not saying
that this is *my* point of argumentation, but it has been so within some
threads in Debian).

On 11/12/2014 04:28 PM, Matthias Runge wrote:
> Looking at es5-shim, it pulls in additional 28 dependent packages,
> json3 has 12 dependencies (including a circular dependency, one
> circular depencency in dependencies),
>
> Not counting dependencies in dependent packages, I can only suspect,
> this will bring us at least 100 new packages required.

PLEASE DON'T DO THAT !!!
Or should I kill myself just right away now? :)

Yes, I did read the self-reply of Matthias. However, what he described
above is exactly what I fear will happen if we don't take enough care.

Cheers,

Thomas Goirand (zigo)

[1] We're currently at the rate of 1 CVE per week. If you don't believe
me, just count by yourself!!!