[Pkg-javascript-devel] npm tarballs can be trusted, was: Re: Bug#943389: Bug#943389: node-lodash: source package does not contain upstream source

[Paolo Greppi]

Hi Jeremy, 

I am replying here to the side note.

On 24/10/19 19:47, Jérémy Lal wrote:
> Le jeu. 24 oct. 2019 à 19:33, Jonas Smedegaard <dr at jones.dk <mailto:dr at jones.dk>> a écrit :
> ..
> Side note: downloading from npmjs.org <http://npmjs.org> should be avoided and maybe a
> good candidate for a lintian error; for it is an unreliable source
> (no checksum, no guarantee, unlike git, afaik).
> 
> Jérémy, trying to help.

I do not agree that npm registry tarballs should be distrusted.

Actually they are (supposed to be) immutable and they always had a shasum field (based on sha1), it is documented here:
https://github.com/npm/registry/blob/master/docs/REGISTRY-API.md#version

For example I looked at async module, version 0.8.0.

There is an archived copy of the npm registry JSON from 4 years ago (17 Jul 2015): http://archive.is/acGAB
from there, the dist object is for version 0.8.0 is:
{"shasum":"ee65ec77298c2ff1456bc4418a052d0f06435112","tarball":"http://registry.npmjs.org/async/-/async-0.8.0.tgz"},

The same shasum is available today:
curl -s https://registry.npmjs.org/async | jq '.versions["0.8.0"].dist'
{
  "shasum": "ee65ec77298c2ff1456bc4418a052d0f06435112",
  "tarball": "https://registry.npmjs.org/async/-/async-0.8.0.tgz"
}
and this can be verified:

wget https://registry.npmjs.org/async/-/async-0.8.0.tgz
sha1sum async-0.8.0.tgz 
ee65ec77298c2ff1456bc4418a052d0f06435112  async-0.8.0.tgz

Additionally recent tarballs have new fields:
- integrity (based on https://w3c.github.io/webappsec-subresource-integrity/#integrity-metadata, for modules published with npm at 5 or later)
- and npm-signature (see https://blog.npmjs.org/post/172999548390/new-pgp-machinery)

For example from here:
curl -s https://registry.npmjs.org/yarn | jq '.versions["1.19.1"].dist'
{
  "shasum": "14b92410dd1ba5bab87a12b4a3d807f4569bea97",
  "tarball": "https://registry.npmjs.org/yarn/-/yarn-1.19.1.tgz",
  "integrity": "sha512-gBnfbL9rYY05Gt0cjJhs/siqQXHYlZalTjK3nXn2QO20xbkIFPob+LlH44ML47GcR4VU9/2dYck1BWFM0Javxw==",
  "fileCount": 12,
  "unpackedSize": 5311732,
  "npm-signature": "-----BEGIN PGP SIGNATURE-----\r\nVersion: OpenPGP.js v3.0.4\r\nComment: https://openpgpjs.org\r\n\r\nwsFcBAEBCAAQBQJdnH9RCRA9TVsSAnZWagAAi4kP/0b67Q6Ra3wfw8bZmY0z\nouSHtZ7Ua3Ke6T4VPUJgcGz4nuRUVu9eRQBX0pHbyek7YBgE5+AIYQ5Gkua5\ndIv182/OS6DtOVFALfyXJogs+51IiD7K+Zd9RExppBiZuFg1yNVRqxK7Ox6X\nLyPw8G20qQFpC/DgRDkj5dur8mZBAGFIxHrbFPTckXKWyEZdtABfE8vT2rp/\nzsnjMLSeS+OORjMBop6GujXxie2ctTKzHiGyrmcrJIxFaYl+rPTep39lxNMM\nKg/Ad/YiE4p9RDFyWsVqHIWSdphwNUyDiB3c9l/JjP31Pvvqiqh5Yq49B8U7\nafcNA3Uh456zYTXvbf+IM1bL3ZYwxh9DD4yuAWcM+D1fbbpQAaow1QFiAxV+\nBP/EBe9FHJlQR6Y3thqMaTwV5SNu9UdS2iYkC2fbilBcYMagUEOuL9ooQ1VU\nxVb2OPIPebpqodt+O5xfioIqAkS3fv9b1BfD8p0akoRRYHwKf4McftO+MM0Q\nr55SRwdqBPm2y/oZVjgXUFmA9nMj8Y8SJ5+yUUwRysGU6qEKgZs0oy16aypk\nZp2Cz2Df+8WuJoDzADWo8mkWSQHaw2/kAVz3P0JtBz4yfVPaip4LMYh6pa4H\nL2XclAkNDOoN+0IQV64miyq6+bbpYjaQFzxYA/SytDoBPvym1p+P91nj0aaR\nmFWt\r\n=QESU\r\n-----END PGP SIGNATURE-----\r\n"
}

shasum and integrity can be verified like this:

openssl dgst -sha512 -binary yarn-1.19.1.tgz | openssl base64 -A
gBnfbL9rYY05Gt0cjJhs/siqQXHYlZalTjK3nXn2QO20xbkIFPob+LlH44ML47GcR4VU9/2dYck1BWFM0Javxw==
sha1sum yarn-1.19.1.tgz
14b92410dd1ba5bab87a12b4a3d807f4569bea97  yarn-1.19.1.tgz

Verifying npm-signature is left as an exercise (they use Keybase for sharing the public keys)

Paolo