Bug#1033406: licensecheck: scan-copyrights fails to create copyright file for texlive-extra

[Jonas Smedegaard]

Quoting Vignesh Raman (2023-03-24 16:52:33)
> On 24/03/23 8:09 pm, Jonas Smedegaard wrote:
> > Please try isolate the exact licensecheck command that scan-copyrights
> > executes (e.g. by hacking that script to print to stderr before
> > executing).
> >
> > When that exact command - including options and arguments - is known, we
> > can move on to examine why the failure isn't deterministic.
> 
> This is the exact command.
> 
> licensecheck --encoding utf8 --copyright --machine --lines=300 
> --shortname-scheme=debian,spdx --recursive 
> '--ignore=^(\./)?(?s:(?>[^/]*?/po/)[^/]*)\Z|^(\./)?(?s:.*\~)\Z|^(\./)?(?s:CVS)\Z|^(\./)?(?s:DEADJOE)\Z|^(\./)?(?s:RCS)\Z|^(\./)?(?s:\..*\.swp)\Z|^(\./)?(?s:\.\#.*)\Z|^(\./)?(?s:\.arch\-ids)\Z|^(\./)?(?s:\.arch\-inventory)\Z|^(\./)?(?s:\.bzr)\Z|^(\./)?(?s:\.bzr\.backup)\Z|^(\./)?(?s:\.bzrignore)\Z|^(\./)?(?s:\.bzrtags)\Z|^(\./)?(?s:\.cvsignore)\Z|^(\./)?(?s:\.deps)\Z|^(\./)?(?s:\.git)\Z|^(\./)?(?s:\.gitignore)\Z|^(\./)?(?s:\.hg)\Z|^(\./)?(?s:\.hgtags)\Z|^(\./)?(?s:\.pc)\Z|^(\./)?(?s:\.shelf)\Z|^(\./)?(?s:\.svn)\Z|^(\./)?(?s:\{arch\})\Z|^(\./)?(?s:_MTN)\Z|^(\./)?(?s:_darcs)\Z|^(\./)?(?s:config\.guess)\Z|^(\./)?(?s:config\.sub)\Z|^(\./)?(?s:debian/(?>[^/]*?/)[^/]*\.asc)\Z|^(\./)?(?s:debian/(?>[^/]*?/)[^/]*\.gpg)\Z|^(\./)?(?s:debian/(?>[^/]*?/)[^/]*\.pgp)\Z|^(\./)?(?s:debian/apertis/[^/]*)\Z|~$|(^|/)\.\.swp$|(^|/),,.*(?:$|/.*$)|(^|/)(DEADJOE|\.cvsignore|\.arch-inventory|\.bzrignore|\.gitignore)$|(^|/)(CVS|RCS|\.pc|\.deps|\{arch\}|\.arch-ids|\.svn|\.hg|_darcs|\.git|\.shelf|_MTN|\.bzr(?:\.backup|tags)?)(?:$|/.*$)|(?:^|/)debian/((fix.scanned.)?copyright|changelog|NEWS|compat|.*yml|docs|source|patches/series)|(png|jpg|pdf|ico|bmp|jpe?g)$' 
> .
> 
> > Your mentioning above that licensecheck failing is "not consistently
> > reproducible" is another bug that is more serious, however: Licensecheck
> > should - with same version also of its libraries - behave the same at
> > each execution.
> >
> > So it would be helpful to me, independent of the issue of not parsing
> > binary data - if you can help isolate the command which is unreliable.
> >
> When I run the above command the results for the problematic files are,
> 
> ./texmf-dist/tex/latex/exp-testopt/exp-testopt.sty    UNKNOWN 2009 Paul 
> Ebermann. | / 2008 Paul Ebermann
> ./texmf-dist/fonts/type1/public/baskervillef/BaskervilleF-Bold.pfb 
> UNKNOWN    2016 / 2012 Rodrigo Fuenzalida (www.rfuenzalida.com|hello 
> :rfuenzalida.com) / 2012 Pablo Impallari 
> (www.impallari.com|impallari at gmail.com),
> ./texmf-dist/fonts/type1/public/baskervillef/BaskervilleF-BoldItalic.pfb 
> UNKNOWN    2016 / 2012 Rodrigo Fuenzalida (www.rfuenzalida.com|hello 
> :rfuenzalida.com) / 2012 Pablo Impallari 
> (www.impallari.com|impallari at gmail.com),
> 
> Going with the above information, scan-copyrights should not crash. If I 
> remove all other files and keep only the above in same directory 
> structure, it does not crash.
> 
> Only when we run scan-copyrights with all the source files, it crashes.

Thanks for the additional details of the crash.

When you wrote "not consistently reproducible", do you then mean that
the exact same command when executed multiple times only sometimes cause
a crash, or do you mean that the crash stops occuring when you remove
all but binary files, as described here above?


> > It is unclear what you are referring to as "this issue": What you
> > summarise in the subject of this bugreport strictly speaking is an issue
> > with libconfig-model-dpkg-perl, and above I have mentioned 2 potential
> > underlying issues that might be your main concern here (as I assume you
> > deliberately chose to file this bugreport against licensecheck).
> 
> Yes, I created the bug report against licensecheck because I suspected 
> that parsing the binary files was causing the issue.
> 
> > If you are not really reporting a new issue here, but instead are asking
> > for best practice of using licensecheck with sources that include binary
> > data, then my most up-to-date notes on that is here:
> > https://wiki.debian.org/CopyrightReviewTools#licensecheck
> 
> Thanks for the link.
> 
> If there are no issues with licensecheck, will report the issue in 
> libconfig-model-dpkg-perl with all this information.

If licensecheck behaves unreliably - i.e. emits varying output for
identical content, then it is a bug in licensecheck that I would like to
hunt down and fix (because that type of bugs can cause confusion
elsewhere and is difficult to locate).

But if the bug you experience is that you *logically* expect that a
crash for some input should also crash for different input, then it
seems like the wrapperscript wreaks havoc in varying ways when receiving
data it fails to handle, which is a bug in the wrapper.

If you agree, then please reassign this bugreport, to carry over my
potentially helpful reasoning.


Kind regards,

 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/
 * Sponsorship: https://ko-fi.com/drjones

 [x] quote me freely  [ ] ask before reusing  [ ] keep private
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: signature
URL: <http://alioth-lists.debian.net/pipermail/pkg-perl-maintainers/attachments/20230324/ca631902/attachment-0001.sig>