Bug#888112: diffoscope: compare JSON files using Python jsondiff module

[Mattia Rizzolo]

On Tue, Jan 23, 2018 at 09:41:18PM +0800, Paul Wise wrote:
> The Python jsondiff module allows comparing JSON files in a better way
> than just comparing the plain text like diffoscope currently does.

Actually, that's a problem.
Diffoscope has a json comparator, that works great, but it's failing for
your example:

> $ curl -s 'https://salsa.debian.org/api/v4/users/1' > 1
> $ curl -s 'https://salsa.debian.org/api/v4/users/2' > 2

mattia at warren /tmp % diffoscope --debug 1 2
2018-01-23 15:45:31 D: diffoscope.presenters.formats: Will generate the following formats: text
2018-01-23 15:45:31 D: diffoscope.main: Starting diffoscope 90
2018-01-23 15:45:31 D: diffoscope.locale: Normalising locale, timezone, etc.
2018-01-23 14:45:31 D: diffoscope.main: Starting comparison
2018-01-23 14:45:31 D: diffoscope.comparators: Loaded 63 comparator classes
2018-01-23 14:45:31 D: diffoscope.comparators.utils.specialize: Using TextFile for 1
2018-01-23 14:45:31 D: diffoscope.comparators.utils.specialize: Using TextFile for 2
2018-01-23 14:45:31 D: diffoscope.comparators.utils.compare: Comparing 1 (TextFile) and 2 (TextFile)
...

I.e. it's not recognizing them as json files.
If I simply rename them to 1.json and 2.json and run diffoscope on them:

mattia at warren /tmp % diffoscope --debug 1.json 2.json
2018-01-23 15:46:40 D: diffoscope.presenters.formats: Will generate the following formats: text
2018-01-23 15:46:40 D: diffoscope.main: Starting diffoscope 90
2018-01-23 15:46:40 D: diffoscope.locale: Normalising locale, timezone, etc.
2018-01-23 14:46:40 D: diffoscope.main: Starting comparison
2018-01-23 14:46:40 D: diffoscope.comparators: Loaded 63 comparator classes
2018-01-23 14:46:40 D: diffoscope.comparators.utils.specialize: Using JSONFile for 1.json
2018-01-23 14:46:40 D: diffoscope.comparators.utils.specialize: Using JSONFile for 2.json
2018-01-23 14:46:40 D: diffoscope.comparators.utils.compare: Comparing 1.json (JSONFile) and 2.json (JSONFile)
2018-01-23 14:46:40 D: diffoscope.comparators.utils.file: File.has_same_content: <<class 'abc.JSONFile'> 1.json> <<class 'abc.JSONFile'> 2.json>
2018-01-23 14:46:40 D: diffoscope.diff: Running diff -aU7 /tmp/tmpfpv5c6dm_diffoscope/fifo1 /tmp/tmpfpv5c6dm_diffoscope/fifo2
2018-01-23 14:46:40 D: diffoscope.diff: diff -aU7 /tmp/tmpfpv5c6dm_diffoscope/fifo1 /tmp/tmpfpv5c6dm_diffoscope/fifo2: returncode 1, parsed True
 |##################################################################################################################################|  100%                             Time: 0:00:00 
2018-01-23 14:46:40 D: diffoscope.presenters.formats: Generating 'text' output at '-'
--- 1.json
+++ 2.json
│   --- 1.json
├── +++ 2.json
│ @@ -1,16 +1,16 @@
│  {
│ -    "avatar_url": "https://seccdn.libravatar.org/avatar/9a5dac7fca5fa69fcb4fa36b948ff766?s=80&d=identicon",
│ -    "bio": "",
│ -    "created_at": "2017-12-17T12:04:05.482Z",
│ -    "id": 1,
│ +    "avatar_url": "https://seccdn.libravatar.org/avatar/a0177f306e580e136af48558989f2782?s=80&d=identicon",
│ +    "bio": null,
│ +    "created_at": "2017-12-17T12:46:16.992Z",
│ +    "id": 2,
│      "linkedin": "",
│ -    "location": "",
│ -    "name": "Administrator",
│ -    "organization": "",
│ +    "location": null,
│ +    "name": "Paul Martin",
│ +    "organization": null,
│      "skype": "",
│      "state": "active",
│      "twitter": "",
│ -    "username": "root",
│ -    "web_url": "https://salsa.debian.org/root",
│ +    "username": "pm",
│ +    "web_url": "https://salsa.debian.org/pm",
│      "website_url": ""
│  }
1 mattia at warren /tmp %                                                                                                                                                             :(


Which is arguably even much better than what you get from that jsondiff
command :)


FTR, it seems the only check used to decide whether a file is a json
file, is to look for a '.json' as a filename suffix (and I'm not sure we
can do much better without looking into the file, which is probably too
expensive a check).

-- 
regards,
                        Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540      .''`.
more about me:  https://mapreri.org                             : :'  :
Launchpad user: https://launchpad.net/~mapreri                  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/reproducible-builds/attachments/20180123/66651be1/attachment.sig>