[Pkg-mozext-maintainers] Bug#877040: New upstream version, including transition to webext

[Sean Whitton]

Source: ublock-origin
Version: 1.13.8+dfsg-1
Severity: normal
Tags: help

uBlock Origin in Debian is several versions behind upstream.  This is
because packaging a recent upstream release requires

- installing the webext version, as a new binary package
  webext-ublock-origin, rather than the traditional XUL extension; and

- providing a script for the user to transition their data from the XUL
  extension to the new webext.

I am filing this bug because I know that I won't be able to work on this
any time soon.  I hope to provide enough information that someone else
is able to work on it independently.

This bug will become release-critical when every version of Firefox in
unstable is greater than or equal to 57, assuming Mozilla's plans do not
change.  At that point, XUL extensions will stop working.

Note that none of this affects the Chrome extension binary package,
unless Chrome, too, decides to support only web extensions.

Installing the webext
=====================

Ideally this would be achieved by fixing #866997.  But it would be fine
for this package to install the web extension without a helper; indeed,
it's not clear we need a helper for web extensions.

There have been several discussions about what needs to be done.  In
addition to those linked from #866997, see [3].  In summary:

- use the binary package namespace 'webext-'

- install to /usr/share/webext/ublock-origin

- create a symlink in the appropriate subdirectory of /usr/share/firefox
  such that Firefox loads the web extension (requires searching through
  the source of Firefox; possibly no such directory exists yet)

The purpose of the symlinking is to accommodate the fact that some web
extensions work with only one browser but others work with multiple
browsers.  We can manage this with symlinks.

See the webext branch of ublock-origin's Vcs-Git repository for first
attempts at installing the webext.

Providing the transition script
===============================

It is not possible to automatically transition the user's data -- by
patching the web extension to import the data, say.  This is because web
extensions cannot access the parts of the filesystem where the XUL
extension stored its data.

Upstream attempted a hybrid XUL and web extension which would import the
data.  They abandoned this project, but in any case, it would have been
no use for Debian: later point updates of stretch and the initial
release of buster will contain no version of Firefox capable of loading
XUL extensions.  So we must provide a script that users can run to
transition their data.  I would suggest a NEWS.Debian entry suggesting
that users run the script.

Upstream (Raymond Hill) has very helpfully provided guidance on writing
such a script.[1] For archival purposes, I reproduce the most
significant parts of Raymond's messages in the linked thread here:

> So anyways, basically this is how things are:
> 
> - With uBO/legacy, the settings are located in
>   ~/.config/firefox/[profile]/extension-data/ublock0.sqlite
>   - The content of the ublock0.sqlite db is one table, named settings,
>     with two columns, named name and value.
>   - Each name is unique, and is the name of a setting.
>   - Each value is a string, and is JSON.parse'd after being read, and
>     before being sent to the caller.
> - With uBO/webext, the settings are located in
>   ~/.config/firefox/[profile]/browser-extension-data/uBlock0 at raymondhill.net/storage.js
>   (that is a Firefox's controlled location).
>   - The content of storage.js is simple JSON data, in which each setting
>     is a [setting name]: [setting value] entry. The value can be any
>     valid JSON value: number, string, array, etc.
> 
> Now given all this, it's a matter of fetching all entries in the sqlite
> table, and converting them into an object property in one single
> destination JSON object structure { ... }.
> 
> Now not all entries in the table are to be transferred to the JSON
> structure, many entries are strictly for internal use by uBO. Any entry
> which name starts with cache/ must be ignored. Additionally, the
> following entries must also be ignored: assetCacheRegistry,
> assetSourceRegistry, selfie.
> 
> The code to migrate is located in this bootstrap.js file.[2] The
> function named getStorageMigrator contains all the code to deal with
> reading all entries one by one from the settings table.
> 
> [...]
> 
> So far I got something with this command -- needs more work though to
> get a valid JSON file -- so before I lose that command:
> 
>     sqlite3 ublock0.sqlite ".separator "\\t " SELECT * FROM settings;" | awk '!/^(?:assetCacheRegistry|assetSourceRegistry|cache\/.+|selfie)$/' > output.txt
> 
> This outputs two fields per line, separated by \t.
> 
> [field1]	[field2]
> 
> After this is for each line to be output this way:
> 
> "[field1]": [field2]
> 
> Then to separate all the lines with a , character, then finally wrap the
> result in { and }.

[1]  https://github.com/gorhill/uBlock/issues/2948
[2]  https://github.com/gorhill/uBlock/blob/1.13.10/platform/webext/bootstrap.js#L39
[3]  https://lists.alioth.debian.org/pipermail/pkg-mozext-maintainers/2017-August/006273.html

-- 
Sean Whitton
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-mozext-maintainers/attachments/20170927/b3f9b3de/attachment.sig>