[openstack-dev] [javascript] Linters

Robert Collins robertc at robertcollins.net
Wed Jun 10 18:30:49 UTC 2015

On 11 June 2015 at 03:29, Michael Krotscheck <krotscheck at gmail.com> wrote:
> On Tue, Jun 9, 2015 at 3:37 PM Robert Collins <robertc at robertcollins.net>
> wrote:

> There are two package managers in the JavaScript world right now, one that
> focuses on node.js/server dependencies (karma, lint, express, etc), and one
> that focuses on in-browser dependencies (angular, bootstrap, etc). They're
> both required for thick browser clients, but for server apps you only really
> need package.json
> https://docs.npmjs.com/files/package.json

Ok, thanks. So within that only the *Dependencies fields seem
interesting, no? With our multiple ways of expressing requirements in
Python we fold all the things that might be in any of them into
global-requirements.txt. I think we're going to want to do the same
here : e.g. /not/ literal package.json, but something very similar -
say one with all the project specific metadata missing and just a
dependencies sub-dict.

>> In the absence of information I'm assuming we should make a subdir
>> 'js' and put bower.json and package.json in there (and do the same for
>> the python things in a subdir called python for symmetry, though
>> backwards compat and tooling considerations may mean that we have to
>> prepare for that. The reason being that if package.json is js
>> specific, its just confusing to have it live at the top level. Also we
>> may want to call them global-$thing, since if we do have js in the
>> requirements repo itself at some point, it would be good not to
>> conflict on names.
> FYI, it looks like the monasca team may want to start doing similar things
> with Java. More information after I hunt them down this morning :)

And swift w/go. Yes.

>> I'm not at all sure that update.py should handle the json files per
>> se; will the policy for those files be the same as we use for python?
> Add linting rule sets to this list, contained either in .jscsrc or
> .eslintrc. Javascript does not have the privilege of pep8 being baked into
> the language tooling, so we have to sync it ourselves.

Ah so - I don't think that linting rule sets fit global-requirements
at all. pep8 (the tool) isn't baked into the language tooling, it was
created and maintained by a third party based on PEP 8 (the
specification), and is more opinionated than PEP8 was ever intended to
be (and in fact PEP8 isn't intended to apply outside of the stdlib,
even though folk like us use it as a starting point).

The set of folk that are interested in requirements management
(distributors want a control point to review licensing and whether its
in their distro, the community consensus also cared about maturity,
supportability and Python3 readiness)... and the set of folk
interested in linting configuration are very different and IMO it
makes no sense to squash them together.

>> If not it might be cleaner to have a new entry point. OTOH I'll need
>> to look closely at a few real {bower,package}.json files to proffer an
>> useful opinion. [Perhaps you covered this in IRC - it didn't come
>> through in your summary].
> It depends on what you think update.py is supposed to do. If I look at that
> repository, I would argue that the purpose is to "Synchronize common files
> and properties across registered openstack projects". In this case, a
> "requirement" is defined as something required to meet a minimum set of
> project "sanity" standards.

There are two related bits of tooling. We have openstack/requirements
which is a central place for dependency ('requirements' inspired from
requirements.txt and the -r parameter to pip). And we have a separate
tool that runs scripts *like* requirements/update.py and proposes
things from them:

So - we have a bunch of places where we synchronise things across
projects in various ways. The code in openstack/requirements today is
all and only about dependency management. I think it should stay that
way. Adding js and other languages makes a tonne of sense; whether the
/code/ involved in adding those languages should be Python or $other
language, and whether it should be added to the existing update.py or
a new entry point in the same project - those are two questions whose
answers are not entirely clear to me.

By 'same policy' earlier - let me expand: the policy update.py
implements for Python today is:
0: global requirements are defined in global-requirements.txt
1: projects MUST NOT have dependencies not present in global-requirements.
2: dependencies must be an exact textual match for those in global-requirements
3: project requirements are in requirements.txt or test-requirements.txt

Adding JS to update.py specifically involves:
- abstracting out the location and parsing of requirements
- deciding if the same rules 1 and 2 above apply to JS
- teaching the tool how to merge its output into structured JSON

The changes we have planned (from the requirements-management spec) to
update.py that aren't visible as patches in gerrit are:
 - teach it to handle PEP-426 markers
 - move from exact text match to a 'contains the global requirement'
comparison. That is, we want to allow projects to have wider
dependencies locally than we specify globally, which will be wider
than we test with (which will be a single clamped version everywhere -
its how we will guarantee co-installability). This will allow projects
to fix things that need doing so urgently locally. OTOH it may be that
we were optimistic about this change at the summit - I need to work
through the potential races between the constraints lists and local
project deps very carefully before doing this bit.
 - add a tool to update constraints/requirements files a bit like
iniset (to make certain things like running with a git dependency in
devstack easier.

Sidebar: The dependency management thing is entirely functional - its
not loosely defined like('sanity'): its what is required to deliver
co-installability of the projects that honour that. In the node world
thats much less of an issue in the npm world where dependencies are
localised - so localised you can have hundreds of different versions
of the same thing in one resolved tree ;). I'm not pushing back on
having the js deps centrally managed - I just find the differences
interesting to think about.

>> Also, I'm in the middle of rearranging update.py to handle PEP 426 and so
>> we should probably coordinate so we don't tromp on each other.
> Do you have a patch?

I have a number open in gerrit, and am adding more as I prepare them.
I was hoping to have the one handling PEP-426 markers for update.py
(which is causing a number of my other ones to fail) ready yesterday,
but things happened ;)


Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud

More information about the OpenStack-dev mailing list