[Debian-science-sagemath] <DKIM> Re: <DKIM> Re: widgetsnbextension situation

[Julien Puydt]

Hi,

On 15/11/2016 11:26, Ximin Luo wrote:
> Julien Puydt:
>> On 15/11/2016 02:51, Ximin Luo wrote:
>>
>>> Perhaps we can use this information to regenerate extension.js without using webpack itself. Julien, have you seen anything approximating this approach before?
>>>
>>
>> Yes.
>>
>> I would try using the compile-modules command from my package node-es6-module-transpiler -- and that might not even be necessary.
>>
>> As I already mentioned (did I really?), webpack is mostly used to put together various chunks of javascript code and "compress" them so they're smaller for web use -- so there's a chance we can get away by not compressing anything and just installing the files as-is. That would mean just writing a debian/install file putting the un-modified source files at a right place so they find each other at runtime!
>>
>
> I'm not sure if runtime-linking would be easy to make work. Do you have an example of this somewhere?

Well, as far as I know, you just have to "foo=require('foo');" and you 
can then use module foo... or did I miss something?

> For now, I'm trying to write a python script to generate extension.js from "sources" and "sourcesContent" in extension.js.map. I'm about 80% of the way there...
>
> For widgetsnbextension, webpack seems to do more sophisticated things than what you said. For a start, it wraps all of the component files with something like

The "compile-modules" I pointed you to is supposed to do bundling, that 
is put all sources of a javascript module in a single file -- including 
the deps if needed.

> function(module, exports, __webpack_require__) {
>     [..internal javascript..]
> }
>
> then rewrites the internal javascript's calls to require(name) with __webpack_require__(i) where i is the index into webpack's own module list of modules, 56 of them for widgetsnbextension.
>
> I have managed to recreate something similar to this rewriting in python, however webpack also does other rewriting, like this:
>
> widgetsnbextension-1.2.6/widgetsnbextension/static$ ../../../ipywidgets/regen-webpack-source.py ./filelist ./module_list . > output; diff output extension.js
>
> # output is what I can auto-generate so far; extension.js is what we're aiming for
> --- output      2016-11-15 11:22:16.206545596 +0100
> +++ extension.js        2016-07-14 22:31:07.000000000 +0200
> @@ -865,9 +865,9 @@
>
>  /***/ },
>  /* 2 */
> -/***/ function(module, exports) {
> +/***/ function(module, exports, __webpack_require__) {
>
> -       //     Underscore.js 1.8.3
> +       var __WEBPACK_AMD_DEFINE_ARRAY__, __WEBPACK_AMD_DEFINE_RESULT__;//     Underscore.js 1.8.3
>         //     http://underscorejs.org
>         //     (c) 2009-2015 Jeremy Ashkenas, DocumentCloud and Investigative Reporters & Editors
>         //     Underscore may be freely distributed under the MIT license.
> @@ -914,7 +914,7 @@
>           // Export the Underscore object for **Node.js**, with
>           // backwards-compatibility for the old `require()` API. If we're in
>           // the browser, add `_` as a global object.
> -         if (typeof exports !== 'undefined') {
> +         if (true) {
>             if (typeof module !== 'undefined' && module.exports) {
>               exports = module.exports = _;
>             }
> @@ -2409,10 +2409,10 @@
>           // popular enough to be bundled in a third party lib, but not be part of
>           // an AMD load request. Those cases could generate an error when an
>           // anonymous define() is called outside of a loader request.
> -         if (typeof define === 'function' && define.amd) {
> -           define('underscore', [], function() {
> +         if (true) {
> +           !(__WEBPACK_AMD_DEFINE_ARRAY__ = [], __WEBPACK_AMD_DEFINE_RESULT__ = function() {
>               return _;
> -           });
> +           }.apply(exports, __WEBPACK_AMD_DEFINE_ARRAY__), __WEBPACK_AMD_DEFINE_RESULT__ !== undefined && (module.exports = __WEBPACK_AMD_DEFINE_RESULT__));
>           }
>         }.call(this));
>
> @@ -2421,7 +2421,7 @@
>  /* 3 */
> [..]
>
> etc etc. I'm not sure if this is feasible to recreate with regex in python, I would probably have to look for a Javascript AST parser and write rewrite rules for that. Or do you have an alternative suggestion?

What you do seems impressive. I have already pointed you to my 
node-es6-module-transpiler package, but you might also be interested by 
node-recast and node-ast-util.

Of course, rollup would be nice to have, but I still haven't managed to 
bootstrap it : I have two RFS on deps (node-vlq and 
node-sourcemap-codec) and I'm still fighting with trying to understand 
acorn upstream and what happened to acorn-in-Debian...

> After this is done, then there is the question of actually generating the "sourcesContent" files. I can see that "widgets.min.css" needs to be generated via other rules, but hopefully the *.js files can be copied directly from their sources...

The ".min." means it has gone through a minimization process, which 
uglifyjs might help with : things get one-letter names, some functions 
are inlined, as many spaces/newlines are removed as possible... it's to 
optimize bandwith use.

Snark on #debian-js