[Piuparts-devel] Summary.json out of date

[David Steele]

On 7/17/21 11:08 AM, David Steele wrote:
> I just noticed that the top summary.json file is old.
> 
> {
>   "_comment": "Debian Piuparts Package Results - 
> https://salsa.debian.org/debian/piuparts/raw/develop/piupartslib/pkgsummary.py 
> <https://salsa.debian.org/debian/piuparts/raw/develop/piupartslib/pkgsummary.py>",
>   "_date": "Thu Jul 15 13:29:51 UTC 2021",
>   "_id": "Piuparts Package Test Results Summary",
>   "_type": "source",
>   "_version": "1.0",
>   "packages": {
>    "0ad": {
> ...
> 
> 
> in piuparts-report.txt
> 
> 22:57:33 Writing static pages
> 22:57:36 Generating global summary
> Traceback (most recent call last):
>    File "/srv/piuparts.debian.org/share/piuparts/piuparts-report 
> <http://piuparts.debian.org/share/piuparts/piuparts-report>", line 1911, 
> in <module>
>      main()
>    File "/srv/piuparts.debian.org/share/piuparts/piuparts-report 
> <http://piuparts.debian.org/share/piuparts/piuparts-report>", line 1905, 
> in main
>      generate_global_summary(output_directory, section_names)
>    File "/srv/piuparts.debian.org/share/piuparts/piuparts-report 
> <http://piuparts.debian.org/share/piuparts/piuparts-report>", line 1689, 
> in generate_global_summary
>      summary = pkgsummary.merge_summary(summary, sec_summ)
>    File 
> "/srv/piuparts.debian.org/lib/python3/dist-packages/piupartslib/pkgsummary.py 
> <http://piuparts.debian.org/lib/python3/dist-packages/piupartslib/pkgsummary.py>", 
> line 188, in merge_summary
>      add_summary(gbl_summ, rep_sec, pkg, flag, block_cnt, url)
>    File 
> "/srv/piuparts.debian.org/lib/python3/dist-packages/piupartslib/pkgsummary.py 
> <http://piuparts.debian.org/lib/python3/dist-packages/piupartslib/pkgsummary.py>", 
> line 168, in add_summary
>      raise SummaryException("Invalid summary argument")
> piupartslib.pkgsummary.SummaryException: Invalid summary argument
> 
> 
> The code:
> 
> def add_summary(summary, rep_sec, pkg, flag, block_cnt, url):
> if not flag in flaginfo or not isinstance(block_cnt, int) \
> or not url.startswith('http'):
> raise SummaryException("Invalid summary argument")
> 
> -- 

Still failing.

It looks this is driven by two packages in the "sid2experimental" 
section, "diaspora" and "ruby-device-two-factor", which report as being 
in the "unknown" state [1]. I don't think pkgsummary has seen this state 
as an input before. It is reporting the url for these packages as the 
default "invalid url" [2], which causes the Traceback above.

We are in freeze, and piuparts is part of the release chain. You really 
should mitigate this now by setting "sid2experimental" "json-section = 
none", fixing piuparts to report the state more accurately, and/or 
fixing pkgsummary to provide a valid url for this case.

I'd also recommend adding a log entry in piuparts-report to point to the 
offending section in the future.

     #!/usr/bin/python3

     import sys
     import json

     # print(sys.argv[1])

     with open(sys.argv[1], 'r') as fp:
         summary = json.load(fp)

     for pkg in summary["packages"]:
         for dist in summary["packages"][pkg]:
             # print(summary["packages"][pkg])
             (flag, count, url) = summary["packages"][pkg][dist]
             if (
                 flag not in "PFXW"
                 or not isinstance(count, int)
                 or not url.startswith("http")
             ):
                 print(
                     "{}: {} - {}".format(
                         sys.argv[1],
                         pkg,
                         summary["packages"][pkg][dist]
                     )
                 )



[1]: https://piuparts.debian.org/sid2experimental/state-unknown.html
[2]: https://piuparts.debian.org/sid2experimental/summary.json

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <http://alioth-lists.debian.net/pipermail/piuparts-devel/attachments/20210723/d340c547/attachment.sig>