Bug#1128981: jsonpath-ng: please fix d/watch for the new PyPi archive naming policy

[Alexandre Detiste]

Source: jsonpath-ng
Version: 1.7.0-4
Severity: normal

Hi Antonio,

PyPi has changed the way it mangles archive names a while ago.

The old d/watch files cannot detect the new archive.

I'm using an UDD script to track those (hereunder).

Please tweak the watch file here to accept [_-].

I'm not quite sure that the watch v5 logic version
with the template is up-to-par;
you may want to try for yourself


https://pypi.debian.net/jsonpath-ng

- jsonpath-ng-1.6.0.tar.gz
- jsonpath-ng-1.6.1.tar.gz
- jsonpath-ng-1.7.0.tar.gz
- jsonpath_ng-1.8.0.tar.gz





$ cat pypi.py
#!/usr/bin/python3

import re

from looseversion import LooseVersion2 as V
import requests
import psycopg

RE = re.compile(r'(https?://pypi\.debian\.net/.*)/(.*)\-')

conn = psycopg.connect("postgresql://udd-mirror:udd-mirror@udd-mirror.debian.net/udd")
cursor = conn.cursor()
cursor.execute("""
    SELECT DISTINCT source, version, watch_file
    FROM upstream
    WHERE watch_file LIKE '%://pypi.debian.net/%'
    AND status = 'up to date'   -- well it _thinks_ it is up-to-date
""")

count = 0
for row in cursor.fetchall():
    source_, version_, watch_ = row
    source = source_.decode('ascii', 'ignore')
    version = version_.decode('ascii', 'ignore')
    watch = watch_.decode('ascii', 'ignore')
    current = V(version)
    if not current:
        print('cannot parse %s' % version)
        continue
    watch = watch.replace('@PACKAGE@', source)
    watch = watch.replace('@ANY_VERSION@', '-any_version')
    match = RE.search(watch)
    if not match:
        continue
    base = match[1]
    archive = match[2]

    transform = archive.replace('-', '_').lower()
    if transform == archive:
        continue

    count += 1
    #print(base, version, archive)
    r = requests.get(base)
    for line in r.text.splitlines():
        if transform in line:
            #print(line)
            new_v = line.split('>')[1].split('<')[0]
            next_v = new_v.split('-')[1]
            if next_v.endswith('.tar.gz'):
                next_v = next_v[:len(next_v)-len('.tar.gz')]
            if next_v.endswith('.tgz'):
                next_v = next_v[:len(next_v)-len('.tgz')]
            next_version = V(next_v)
            if not next_version:
                print('cannot parse %s' % next_version)
            elif next_version > current:
                print(source, version, base, new_v, archive)

print(count)