Bug#850485: diffoscope: APK support issues - traceback on existent directory & missing zipinfo & misleading apktool.yml file

[Emanuel Bronshtein]

Package: diffoscope
Version: 60
Severity: normal

Dear Maintainer,

3 issues regarding APK files (apk.py comparator) below:

#1 - Diffoscope fail to run on APKs if supplied via absolute paths.

Running: (using diffoscope from GIT)

/data/repbdiffs/repos/diffoscope/bin/diffoscope /tmp/1.apk /tmp/2.apk

Result:

Destination directory (/tmp/1.apk) already exists. Use -f switch if you want to overwrite it.
Traceback (most recent call last):
  File "/data/repbdiffs/repos/diffoscope/diffoscope/main.py", line 260, in main
    sys.exit(run_diffoscope(parsed_args))
  File "/data/repbdiffs/repos/diffoscope/diffoscope/main.py", line 236, in run_diffoscope
    parsed_args.path1, parsed_args.path2)
  File "/data/repbdiffs/repos/diffoscope/diffoscope/comparators/utils/compare.py", line 61, in compare_root_paths
    return compare_files(file1, file2)
  File "/data/repbdiffs/repos/diffoscope/diffoscope/comparators/utils/compare.py", line 78, in compare_files
    return file1.compare(file2, source)
  File "/data/repbdiffs/repos/diffoscope/diffoscope/comparators/utils/file.py", line 199, in compare
    if hasattr(self, 'compare_details') or self.as_container:
  File "/data/repbdiffs/repos/diffoscope/diffoscope/comparators/utils/file.py", line 108, in as_container
    self._as_container = self.__class__.CONTAINER_CLASS(self)
  File "/data/repbdiffs/repos/diffoscope/diffoscope/comparators/utils/archive.py", line 44, in __init__
    self._archive = self.open_archive()
  File "/data/repbdiffs/repos/diffoscope/diffoscope/tools.py", line 50, in tool_check
    return original_function(*args, **kwargs)
  File "/data/repbdiffs/repos/diffoscope/diffoscope/comparators/apk.py", line 45, in open_archive
    shell=False, stderr=None, stdout=subprocess.PIPE)
  File "/usr/lib/python3.5/subprocess.py", line 271, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['apktool', 'd', '-k', '-m', '-o', '/tmp/1.apk', '/tmp/1.apk']' returned non-zero exit status 1

it does work when running as:
cd /tmp && /data/repbdiffs/repos/diffoscope/bin/diffoscope 1.apk 2.apk

fix:
	use temporary directory for apktool unpacking.

#2 - apktool.yml file created by apktool is shown as file from APK & contain input filenames (might be unrelated to files content)

apktool generate apktool.yml which contain metadata about the APK, more information:
https://ibotpeaches.github.io/Apktool/documentation/
but shown as file from APK which is incorrect, for example:
diffoscope 1.apk 2.apk
1.apk is: https://f-droid.org/repo/com.poinsart.votar_9.apk
2.apk is: https://verification.f-droid.org/com.poinsart.votar_9.apk

Result:

--- 1.apk
+++ 2.apk
├── apktool.yml
@@ -1,9 +1,9 @@
│  !!brut.androlib.meta.MetaInfo
│ -apkFileName: 1.apk
│ +apkFileName: 2.apk
│  compressionType: false
│  doNotCompress:
│  - arsc
│  isFrameworkApk: false
│  packageInfo: null
│  sdkInfo:
│    minSdkVersion: '9'


it's better to show it as "APK metadata" (similar to "file list","metadata", etc..) instead of apktool.yml

also the apktool.yml contain the filename recevied by apktool at apkFileName field, thus if apktool was run directly on files supplied via command-line (instead of files inside archive) it will show difference that not related to APK content, example above and in:
https://verification.f-droid.org/org.sufficientlysecure.ical_54.apk.diffoscope.html

thus apkFileName field need to be striped from apktool.yml file. (the archive case is supported via zipinfo information, see next issue)

fix:
	1. show apktool.yml difference as "APK metadata" instead of apktool.yml file
	2. remove apkFileName field from apktool.yml result.

#3 missing zipinfo information

on ZIP files the zipinfo utility used to list files inside the archive (may contain difference in file-ordering/permissions/timestamps/etc..), but it is not used on APK files which are ZIP/JAR files.
for example, comparing the zipinfo on APKs:
https://f-droid.org/repo/com.nbossard.packlist_16.apk
https://verification.f-droid.org/com.nbossard.packlist_16.apk
show that there are new-files added & there is file-ordering issue, as happened before apk.py was added. (zip.py handled APK files)

fix:
	use also the zipinfo mechanism as used currently on ZIP files via zip.py comparator on APK files.