Bug#1033406: licensecheck: scan-copyrights fails to create copyright file for texlive-extra

[Vignesh Raman]

Hi Dominique, Jonas

On Sun, 26 Mar 2023 19:18:33 +0200 Dominique Dumont <dod at debian.org> wrote:

 > On Fri, 24 Mar 2023 21:22:33 +0530 Vignesh Raman 
<vignesh.raman at collabora.com> wrote:
 > > Only when we run scan-copyrights with all the source files, it crashes.
 >
 > With texlive-extra-2022.20230122 source, scan-copyright emits some 
warnings but does not fail.
 >
 > Could you try scan-copyright on your side by running this command in 
texlive-extra directory ?
 >
 > $ scan-copyright > copyright.debian
 >
 > Since the source is quite big (the output of license-check weights 
32MB), it's possible that your system runs out of memory.
 > Please check for kernel message.
 >
 > On my system, scan-copyright uses ~130MB when scanning texlive-extra.
 >

I was able to successfully run scan-copyrights with 
texlive-extra-2022.20230122 sources, but encountered an issue when using 
debian/copyright-scan-patterns.yml. I apologize for not including this 
information in my earlier report.

To recreate the issue, I followed these steps:

1. Running scan-copyright > copyright.debian was successful.

2. After adding the content of debian/copyright-scan-patterns.yml, 
running scan-copyright > copyright-with-scan-pattern.debian failed and 
resulted in an empty copyright-with-scan-pattern.debian file.

debian/copyright-scan-patterns.yml:
"check":
   "pattern":
   - |-
     .*
"ignore":
   "pattern":
   - |-
     ^(\./)?(?s:debian/apertis/[^/]*)\Z
   - |-
     ^(\./)?(?s:debian/(?>[^/]*?/)[^/]*\.pgp)\Z
   - |-
     ^(\./)?(?s:debian/(?>[^/]*?/)[^/]*\.gpg)\Z
   - |-
     ^(\./)?(?s:debian/(?>[^/]*?/)[^/]*\.asc)\Z
   - |-
     ^(\./)?(?s:(?>[^/]*?/po/)[^/]*)\Z
   - |-
     ^(\./)?(?s:.*\~)\Z
   - |-
     ^(\./)?(?s:\.\#.*)\Z
   - |-
     ^(\./)?(?s:\..*\.swp)\Z
   - |-
     ^(\./)?(?s:DEADJOE)\Z
   - |-
     ^(\./)?(?s:\.cvsignore)\Z
   - |-
     ^(\./)?(?s:\.arch\-inventory)\Z
   - |-
     ^(\./)?(?s:\.bzrignore)\Z
   - |-
     ^(\./)?(?s:\.gitignore)\Z
   - |-
     ^(\./)?(?s:CVS)\Z
   - |-
     ^(\./)?(?s:RCS)\Z
   - |-
     ^(\./)?(?s:\.pc)\Z
   - |-
     ^(\./)?(?s:\.deps)\Z
   - |-
     ^(\./)?(?s:\{arch\})\Z
   - |-
     ^(\./)?(?s:\.arch\-ids)\Z
   - |-
     ^(\./)?(?s:\.svn)\Z
   - |-
     ^(\./)?(?s:\.hg)\Z
   - |-
     ^(\./)?(?s:\.hgtags)\Z
   - |-
     ^(\./)?(?s:_darcs)\Z
   - |-
     ^(\./)?(?s:\.git)\Z
   - |-
     ^(\./)?(?s:\.shelf)\Z
   - |-
     ^(\./)?(?s:_MTN)\Z
   - |-
     ^(\./)?(?s:\.bzr)\Z
   - |-
     ^(\./)?(?s:\.bzr\.backup)\Z
   - |-
     ^(\./)?(?s:\.bzrtags)\Z
   - |-
     ^(\./)?(?s:config\.guess)\Z
   - |-
     ^(\./)?(?s:config\.sub)\Z

3. After removing the below files,
texmf-dist/fonts/type1/public/baskervillef/BaskervilleF-Bold.pfb
texmf-dist/fonts/type1/public/baskervillef/BaskervilleF-BoldItalic.pfb
texmf-dist/tex/latex/exp-testopt/exp-testopt.sty

running scan-copyright > copyright-with-scan-pattern-remove-files.debian 
was successful and contained the necessary license details.

The same debian/copyright-scan-patterns.yml file has worked without 
issue for other sources.

licensecheck returns the same output consistently on the problematic files,

$ licensecheck --encoding utf8 --copyright --machine --lines=300 
--shortname-scheme=debian,spdx --recursive 
'--ignore=^(\./)?(?s:debian/apertis/[^/]*)\Z|^(\./)?(?s:debian/(?>[^/]*?/)[^/]*\.pgp)\Z|^(\./)?(?s:debian/(?>[^/]*?/)[^/]*\.gpg)\Z|^(\./)?(?s:debian/(?>[^/]*?/)[^/]*\.asc)\Z|^(\./)?(?s:(?>[^/]*?/po/)[^/]*)\Z|^(\./)?(?s:.*\~)\Z|^(\./)?(?s:\.\#.*)\Z|^(\./)?(?s:\..*\.swp)\Z|^(\./)?(?s:DEADJOE)\Z|^(\./)?(?s:\.cvsignore)\Z|^(\./)?(?s:\.arch\-inventory)\Z|^(\./)?(?s:\.bzrignore)\Z|^(\./)?(?s:\.gitignore)\Z|^(\./)?(?s:CVS)\Z|^(\./)?(?s:RCS)\Z|^(\./)?(?s:\.pc)\Z|^(\./)?(?s:\.deps)\Z|^(\./)?(?s:\{arch\})\Z|^(\./)?(?s:\.arch\-ids)\Z|^(\./)?(?s:\.svn)\Z|^(\./)?(?s:\.hg)\Z|^(\./)?(?s:\.hgtags)\Z|^(\./)?(?s:_darcs)\Z|^(\./)?(?s:\.git)\Z|^(\./)?(?s:\.shelf)\Z|^(\./)?(?s:_MTN)\Z|^(\./)?(?s:\.bzr)\Z|^(\./)?(?s:\.bzr\.backup)\Z|^(\./)?(?s:\.bzrtags)\Z|^(\./)?(?s:config\.guess)\Z|^(\./)?(?s:config\.sub)\Z|~$|(^|/)\.\.swp$|(^|/),,.*(?:$|/.*$)|(^|/)(DEADJOE|\.cvsignore|\.arch-inventory|\.bzrignore|\.gitignore)$|(^|/)(CVS|RCS|\.pc|\.deps|\{arch\}|\.arch-ids|\.svn|\.hg|_darcs|\.git|\.shelf|_MTN|\.bzr(?:\.backup|tags)?)(?:$|/.*$)|(?:^|/)debian/((fix.scanned.)?copyright|changelog|NEWS|compat|.*yml|docs|source|patches/series)|(png|jpg|pdf|ico|bmp|jpe?g)$' 
.
failed decoding file BaskervilleF-Bold.pfb as utf8, will try iso-8859-1
./BaskervilleF-Bold.pfb    UNKNOWN    2016 / 2012 Rodrigo Fuenzalida 
(www.rfuenzalida.com|hello :rfuenzalida.com) / 2012 Pablo Impallari 
(www.impallari.com|impallari at gmail.com),
./BaskervilleF-BoldItalic.pfb    UNKNOWN    2016 / 2012 Rodrigo 
Fuenzalida (www.rfuenzalida.com|hello :rfuenzalida.com) / 2012 Pablo 
Impallari (www.impallari.com|impallari at gmail.com),

$ licensecheck --encoding utf8 --copyright --machine --lines=300 
--shortname-scheme=debian,spdx --recursive 
'--ignore=^(\./)?(?s:debian/apertis/[^/]*)\Z|^(\./)?(?s:debian/(?>[^/]*?/)[^/]*\.pgp)\Z|^(\./)?(?s:debian/(?>[^/]*?/)[^/]*\.gpg)\Z|^(\./)?(?s:debian/(?>[^/]*?/)[^/]*\.asc)\Z|^(\./)?(?s:(?>[^/]*?/po/)[^/]*)\Z|^(\./)?(?s:.*\~)\Z|^(\./)?(?s:\.\#.*)\Z|^(\./)?(?s:\..*\.swp)\Z|^(\./)?(?s:DEADJOE)\Z|^(\./)?(?s:\.cvsignore)\Z|^(\./)?(?s:\.arch\-inventory)\Z|^(\./)?(?s:\.bzrignore)\Z|^(\./)?(?s:\.gitignore)\Z|^(\./)?(?s:CVS)\Z|^(\./)?(?s:RCS)\Z|^(\./)?(?s:\.pc)\Z|^(\./)?(?s:\.deps)\Z|^(\./)?(?s:\{arch\})\Z|^(\./)?(?s:\.arch\-ids)\Z|^(\./)?(?s:\.svn)\Z|^(\./)?(?s:\.hg)\Z|^(\./)?(?s:\.hgtags)\Z|^(\./)?(?s:_darcs)\Z|^(\./)?(?s:\.git)\Z|^(\./)?(?s:\.shelf)\Z|^(\./)?(?s:_MTN)\Z|^(\./)?(?s:\.bzr)\Z|^(\./)?(?s:\.bzr\.backup)\Z|^(\./)?(?s:\.bzrtags)\Z|^(\./)?(?s:config\.guess)\Z|^(\./)?(?s:config\.sub)\Z|~$|(^|/)\.\.swp$|(^|/),,.*(?:$|/.*$)|(^|/)(DEADJOE|\.cvsignore|\.arch-inventory|\.bzrignore|\.gitignore)$|(^|/)(CVS|RCS|\.pc|\.deps|\{arch\}|\.arch-ids|\.svn|\.hg|_darcs|\.git|\.shelf|_MTN|\.bzr(?:\.backup|tags)?)(?:$|/.*$)|(?:^|/)debian/((fix.scanned.)?copyright|changelog|NEWS|compat|.*yml|docs|source|patches/series)|(png|jpg|pdf|ico|bmp|jpe?g)$' 
.
./exp-testopt.sty    UNKNOWN    2009 Paul Ebermann. | / 2008 Paul Ebermann

Could you please look into this issue with the details provided? Thank you.

Regards,
Vignesh