[Pkg-javascript-devel] script to generate debian/watch for embedding nodejs modules

[Xavier]

Le 26/11/2018 à 15:55, Paolo Greppi a écrit :
> I am thinking of using this approach:
> https://wiki.debian.org/Javascript/GroupSourcesTutorial
> to embed the modules required for yarkpkg.
> 
> I have created a Python script to automatically generate required the debian/watch file:
> https://salsa.debian.org/paolog-guest/create-bundled-watch
> (to use it, modify the data dictionary as required and invoke it)
> 
> Running uscan on the generated yarnpkg debian/watch file, it spits out a bunch of tar errors but also successfully downloads these files:
> 
> babel-plugin-transform-inline-imports-commonjsyarnpkg_babel-plugin-transform-inline-imports-commonjs_dnscache_normalize-url_tar-fs_v8-compile-cache-0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.tar.gz
> dnscacheyarnpkg_babel-plugin-transform-inline-imports-commonjs_dnscache_normalize-url_tar-fs_v8-compile-cache-0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.tar.gz
> normalize-urlyarnpkg_babel-plugin-transform-inline-imports-commonjs_dnscache_normalize-url_tar-fs_v8-compile-cache-0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.tar.gz
> tar-fsyarnpkg_babel-plugin-transform-inline-imports-commonjs_dnscache_normalize-url_tar-fs_v8-compile-cache-0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.tar.gz
> v8-compile-cacheyarnpkg_babel-plugin-transform-inline-imports-commonjs_dnscache_normalize-url_tar-fs_v8-compile-cache-0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.tar.gz
> yarnpkg_babel-plugin-transform-inline-imports-commonjs_dnscache_normalize-url_tar-fs_v8-compile-cache-0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.tar.gz
> 
> It further creates these soft links:
> 
> node-yarnpkg_0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.orig-babel-plugin-transform-inline-imports-commonjs.tar.gz -> babel-plugin-transform-inline-imports-commonjsyarnpkg_babel-plugin-transform-inline-imports-commonjs_dnscache_normalize-url_tar-fs_v8-compile-cache-0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.tar.gz
> node-yarnpkg_0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.orig-dnscache.tar.gz -> dnscacheyarnpkg_babel-plugin-transform-inline-imports-commonjs_dnscache_normalize-url_tar-fs_v8-compile-cache-0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.tar.gz
> node-yarnpkg_0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.orig-normalize-url.tar.gz -> normalize-urlyarnpkg_babel-plugin-transform-inline-imports-commonjs_dnscache_normalize-url_tar-fs_v8-compile-cache-0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.tar.gz
> node-yarnpkg_0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.orig-tar-fs.tar.gz -> tar-fsyarnpkg_babel-plugin-transform-inline-imports-commonjs_dnscache_normalize-url_tar-fs_v8-compile-cache-0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.tar.gz
> node-yarnpkg_0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.orig.tar.gz -> yarnpkg_babel-plugin-transform-inline-imports-commonjs_dnscache_normalize-url_tar-fs_v8-compile-cache-0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.tar.gz
> node-yarnpkg_0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.orig-v8-compile-cache.tar.gz -> v8-compile-cacheyarnpkg_babel-plugin-transform-inline-imports-commonjs_dnscache_normalize-url_tar-fs_v8-compile-cache-0.15.1+~1.2.0+~1.0.1+~4.0.0+~1.16.3+~2.0.2.tar.gz
> 
> The file names are crazy ! is this the way the Debian JavaScript Maintainers team wants to go ?
> If you are brave, generate the debian/watch for npm and try running uscan on it ...

Hi all,

new uscan provides a way to avoid part of this (fakeupstream.cgi was
used to workaround old-uscan). See below.

Embedding components without following them may be a lack of security. I
think we should have a policy for embedding:
 - components without major risks   => not used in version
 - components that must be followed => declared as "group" in
   debian/watch
 - components that must be followed and used in many other packages
   => packaged separately

Example with node-mongodb (accepted by ftpmaster), 3 "group" components.
It is easy to add non followed component (replace "group" by "ignore"):

# debian/watch:
opts="searchmode=plain,pgpmode=none" \
 https://registry.npmjs.org/mongodb
https://registry.npmjs.org/mongodb/-/mongodb-(\d[\d\.]*)@ARCHIVE_EXT@ group

opts="searchmode=plain,pgpmode=none,component=bson" \
 https://registry.npmjs.org/bson
https://registry.npmjs.org/bson/-/bson-(\d[\d\.]*)@ARCHIVE_EXT@ group

opts="searchmode=plain,pgpmode=none,component=mongodb-core" \
 https://registry.npmjs.org/mongodb-core
https://registry.npmjs.org/mongodb-core/-/mongodb-core-(\d[\d\.]*)@ARCHIVE_EXT@
group

opts="searchmode=plain,pgpmode=none,component=requireoptional" \
 https://registry.npmjs.org/require_optional
https://registry.npmjs.org/require_optional/-/require_optional-(\d[\d\.]*)@ARCHIVE_EXT@
group

# uscan result:
bson-4.0.0.tgz
mongodb-3.1.10.tgz
mongodb-core-3.1.9.tgz
node-mongodb/
node-mongodb_3.1.10+~4.0.0+~3.1.9+~1.0.1.orig-bson.tar.gz
 -> bson-4.0.0.tgz
node-mongodb_3.1.10+~4.0.0+~3.1.9+~1.0.1.orig-mongodb-core.tar.gz
 -> mongodb-core-3.1.9.tgz
node-mongodb_3.1.10+~4.0.0+~3.1.9+~1.0.1.orig-requireoptional.tar.gz
 -> require_optional-1.0.1.tgz
node-mongodb_3.1.10+~4.0.0+~3.1.9+~1.0.1.orig.tar.gz
 -> mongodb-3.1.10.tgz
require_optional-1.0.1.tgz


In this case, require_optional may be declared as "ignore". This gives:

bson-4.0.0.tgz
mongodb-3.1.10.tgz
mongodb-core-3.1.9.tgz
node-mongodb/
node-mongodb_3.1.10+~4.0.0+~3.1.9.orig-bson.tar.gz
 -> bson-4.0.0.tgz
node-mongodb_3.1.10+~4.0.0+~3.1.9.orig-mongodb-core.tar.gz
 -> mongodb-core-3.1.9.tgz
node-mongodb_3.1.10+~4.0.0+~3.1.9.orig-requireoptional.tar.gz
 -> require_optional-1.0.1.tgz
node-mongodb_3.1.10+~4.0.0+~3.1.9.orig.tar.gz
 -> mongodb-3.1.10.tgz
require_optional-1.0.1.tgz


Looks acceptable IMO