[Pkg-javascript-devel] Javascript team policy and rejection of node-three binary package [and 1 more messages]

[Pirate Praveen]

On വെള്ളി 09 മാർച്ച് 2018 08:39 വൈകു, Simon McVittie wrote:
> I think saying "script" is perhaps unhelpful here, because outside
> Javascript, that usually refers to something executable with #! at the
> beginning.
> 
> It might be clearer to think about this in terms of libraries and
> executables: some Javascript packages are libraries (reusable code
> intended to be depended on by something else), some Javascript packages
> are executables (an executable script in /usr/bin or similar, starting
> with #!/usr/bin/nodejs or similar), and some have both.

I have followed the convention used in that document. I agree, changing
it to libraries and executables is better.

> As far as I know, Javascript has two interesting issues from a packaging
> point of view, which make it require some extra thought compared with
> Perl or Python:
> 
> * Some Javascript modules are very small, resulting in lots of small
>   packages

I think we need to balance the small packages concern with number of
times such small packages are used.

node-has was rejected recently because it was "too small". I agree if
only one, two or even three modules depend on it embedding it is okay (I
have been doing that already). But when more than three modules need a
tiny module? Is it really better to embed it that many times?

It is a question of number of packages vs number of duplicate copies and
duplicate manual work.

To summarize,

1. If more than three modules require a tiny module, it should be
packaged separately.

2. If only a single package is depending on a module, it could be
embedded to reduce the number of packages. But if a second package also
require it, then it should be packaged separately.

3. If a module involves some build/transformation steps, it should be
packaged separately t make managing it easier.

> * There are multiple Javascript runtimes/interpreters (nodejs, gjs,
>   seed, web browsers) and each Javascript library is usable by one or
>   more of those interpreters, but not necessarily by all of them
> 
> The second issue applies equally to Python: some Python libraries
> only work in CPython 2 (python-foo), some only work in CPython 3
> (python3-foo), and some also work in PyPy (pypy-foo). For Python, we've
> chosen to distribute libraries for those three runtimes as separate
> binary packages, because that doesn't result in an overwhelmingly large
> number of binary packages.
> 
> For Javascript, because some libraries (particularly in the nodejs
> ecosystem) are very small and numerous, following the same convention as
> Python would result in a very large number of binary packages, and the ftp
> team are uncomfortable about accepting those; so we might want to have
> a different policy, where (whenever possible) a single binary package
> libjs-foo provides the 'foo' library for all the runtimes it supports.

The main two runtimes, we currently target are browsers and nodejs. So
libjs-foo providing node-foo.

>> Exceptions (cases where a provides may be insufficient):
>> 1. should generate binary package 'foo' ('node-foo' in case of a name
>> conflict) and declare dependency on nodejs, if script includes a command
>> line tool
> 
> Perhaps a better way to think about this would be something like:
> 
> """
> If a Javascript source package includes reusable library code, then that
> part of the package should be packaged as a library:
> 
> * the binary package that enables you to require('foo') in nodejs should
>   either have Provides: node-foo or be named node-foo
> * if the library is designed for use by nodejs only, it should be in a
>   package named node-foo
> * if the library is usable by multiple Javascript interpreters, it should
>   be in a package named libjs-foo that Provides: node-foo
> """

Agreed.

Also we need to consider source package names as well. I think if a
project (library/tool) is distributed as a node module (it has a
package.json and available from npmjs.com) then the source package
should also be node-foo.

This is also helpful for 'npm2deb search' to determine if a module is
packaged already. Because of non standard names, sometimes we have to
duplicate efforts to get some modules packaged.

> (In case you haven't noticed, I'm a big fan of "names are APIs and
> APIs are names": I think the systematic policies that we have for some
> languages, where libfoo2 always means you can run binaries with
> DT_NEEDED: libfoo.so.2, libfoo-bar-perl always means you can "use
> Foo::Bar" in Perl, and python3-foo always means you can "import foo"
> in Python 3 are a very good idea.)

Agreed.

> And for executables, perhaps something like this:
> 
> """
> If a Javascript package includes an executable or executables that are
> useful in their own right, then they should be packaged in a binary package
> that represents those executables, as opposed to the library:
> 
> * must depend on the necessary interpreter, usually nodejs

Agreed.

> * must depend on any Javascript libraries that those executables need
> * should not be named node-* without a suffix like -bin or -tools (?)

I don't think there is any particular benefit in this restriction.
Having a uniform was to map node module name -> package name without
having extra effort to check if it is an executable or library is
helpful in automation (especially because we are talking about a large
number of modules here). Manually finding if a module is already
packaged vs automatically finding.

> """
> 
> I'm not sure how this would be phrased in a policy, but I think the
> goal should perhaps be: if the user wants to run the uglifyjs(1)
> command-line utility, they should be able to install a package that
> represents the uglifyjs command-line tool, without needing to know
> or care whether uglifyjs is itself written in Javascript, or whether
> it's been reimplemented in Python or C or Perl. The goal of the user
> of uglifyjs(1) is to minify/compress some Javascript, and the fact that
> they do that by running a tool that is itself written in Javascript is
> just an implementation detail.

Agreed.

> This gets annoying if the name of the executable would be too generic
> (namespace pollution) without a node- prefix, like node-static (I think
> "static" would be a bad name for an executable), but perhaps that could be
> packaged as node-static-tools or node-static-bin or something?

node-static itself is enough to avoid conflict I think. See above.

>> 2. should generate binary package 'node-foo' in addition to 'libjs-foo',
>> if the script requires a newer version of nodejs than available in
>> testing. This is to facilitate proper testing migration (provides will
>> not be sufficient to block migration to testing before required nodejs
>> also migrates).
> 
> Breaks might be a better mechanism for this, but I'm not sure about the
> corner cases. I think Breaks would prevent migration as long as there is
> some other package that depends on both nodejs and the library (because
> migrating the library would reduce installability), but it would not
> prevent migration if there is no such package.

I'd like to get a definitive answer on this.

> I think I should also point out here that these libraries shouldn't be
> in unstable until the corresponding nodejs is ready or almost ready
> to migrate: in general, packages in unstable should be "almost ready"
> for migration to testing, and they should certainly be *installable*.
> If you're uploading libraries or tools that require a version of nodejs
> that is not yet available in unstable, or that require a version of
> nodejs that is in unstable but is not going to migrate for a while (for
> instance if it has RC bugs that are not going to be easy to solve),
> then they should probably be going to experimental instead.
> 
> If packages that require nodejs 6 don't hit unstable until nodejs 6
> is (about to be) in testing, then the corner cases where Breaks would
> be a problem are perhaps not something that will occur in practice?

Well, that is the theory, it is different in practice. The package was
building fine in experimental but failed to build in mips when moved to
unstable.

nodejs 6 was uploaded to unstable on 2017-08-15 but nodejs >= 6 was
available in testing only on 2018-02-26 (more than 6 months).

> Ian Jackson wrote:
>>> Also your comments about namespace make me wonder: do many of these
>>> node scripts have poorly chosen command names ?  Can I get a list of
>>> them easily somehow ?
>>
>> apt-file find /usr/bin |grep ^node-
>>
>> will give most of it, but some packages like mocha only provides
>> node-mocha so the search should include source package names too.
> 
> I think things like mocha are OK: that's a "brand name" that isn't just
> a description of what it does or a commonly used jargon term.

There is also ruby-mocha (I was thinking about this when I mentioned
when there is a conflict) but it does not have any executables.

> If the command-line tool provided by src:node-static was just called
> static, I'd be uncomfortable with that. (But it's called node-static,
> which seems fine.)
> 
>>> I think "handlebars" and "uglify" are OK for this, but I don't know
>>> what the whole ecosystem is like.
> 
> Those seem fine to me too.
> 
>>> In practice, as an easier guideline, maybe it would be better to say
>>> that the command and package should *usually* be renamed, unless the
>>> script is high propfile and has a good name which is unlikely to
>>> conflict.
>>
>> I don't agree. I don't think the benefit outweigh the cost. Changing
>> node to nodejs costed us so much effort in patching at so many places.
>> And as soon as we could use node again, we switched back.
> 
> I think "should usually be renamed" might be going too far, for the
> reasons Pirate mentioned; but it's certainly something the maintainer
> (and the sponsor if any, and the ftp team reviewer) should at least be
> thinking about, to the same extent as for any other executable in the
> flat global namespace of $PATH.
> 
> If the executable is a non-obvious "brand name" like mocha or coveralls
> or gyp or uglifyjs, then that seems reasonable to ship as-is without
> renaming.

Agreed.

> If the executable is something really generic like po2json or static
> (I'm looking at the output of `apt-file search /usr/bin/node` for
> inspiration here) then I think it would be necessary to rename for
> disambiguation. (Note that the real executables in Debian here are
> node-po2json and node-static, so they're fine; I don't know whether
> that's because they're OK upstream or whether they've been renamed in
> Debian.)
> 
> I don't think there's any substitute for the maintainer and reviewers
> putting case-by-case thought into this, tbh.

I'd say when there is an actual conflict, then only we should rename.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.alioth.debian.org/pipermail/pkg-javascript-devel/attachments/20180311/4a4d186b/attachment.sig>