Bug#1003449: texlive-base: Reproducible configuration file ls-R: sychronise both variants of mktexlsr
Roland Clobus
rclobus at rclobus.nl
Mon Jan 10 11:41:50 GMT 2022
Package: texlive-base
Version: 2021.20211217-1
Severity: wishlist
Tags: patch
User: reproducible-builds at lists.alioth.debian.org
Usertags: randomness
X-Debbugs-Cc: reproducible-bugs at lists.alioth.debian.org
Hello Debian TeX Task Force,
While working on the “reproducible builds” effort [1], I have noticed that the
output of the shell script and the output of the Perl script for 'mktexlsr'
produce different output, as seen in '/var/lib/texmf/ls-R'.
This was demonstrated in the Cinnamon live image for bookworm and sid [2], as
built on Jenkins [3].
If the Perl version is used as the last step in postinst (e.g. when invoked
from fmtutil), the output is not reproducible.
I've attached two patches, feel free to use either of them.
The file synchronise_mktexlsr_shell_and_perl.patch makes the output identical,
assuming that the Perl script is started with '--sort':
* The shell script occasionally outputs a double empty line, 'cat -s' removes
this
* The Perl script had an empty line before './:' and had a trailing empty line
The file synchronise_mktexlsr_shell_and_perl_and_remove_sort.patch contains the
same and additionally removes the functionality of the '--sort' flag (by always
sorting), while still allowing the flag to be passed.
[1]: https://wiki.debian.org/ReproducibleBuilds
[2]: https://tests.reproducible-builds.org/debian_live_build/cinnamon-
unstable.html
[3]:
https://jenkins.debian.net/view/live/job/reproducible_debian_live_build_cinnamon_sid
-- Package-specific info:
IMPORTANT INFORMATION: We will only consider bug reports concerning
the packaging of TeX Live as relevant. If you have problems with
combination of packages in a LaTeX document, please consult your
local TeX User Group, the comp.text.tex user group, the author of
the original .sty file, or any other help resource.
In particular, bugs that are related to up-upstream, i.e., neither
Debian nor TeX Live (upstream), but the original package authors,
will be closed immediately.
*** The Debian TeX Team is *not* a LaTeX Help Desk ***
If you report an error when running one of the TeX-related binaries
(latex, pdftex, metafont,...), or if the bug is related to bad or wrong
output, please include a MINIMAL example input file that produces the
error in your report.
Please run your example with
(pdf)latex -recorder ...
(or any other program that supports -recorder) and send us the generated
file with the extension .fls, it lists all the files loaded during
the run and can easily explain problems induced by outdated files in
your home directory.
Don't forget to also include minimal examples of other files that are
needed, e.g. bibtex databases. Often it also helps
to include the logfile. Please, never send included pictures!
If your example file isn't short or produces more than one page of
output (except when multiple pages are needed to show the problem),
you can probably minimize it further. Instructions on how to do that
can be found at
http://www.minimalbeispiel.de/mini-en.html (english)
or
http://www.minimalbeispiel.de/mini.html (german)
##################################
minimal input file
##################################
other files
######################################
List of ls-R files
-rw-rw-r-- 1 root staff 81 Jan 9 11:33 /usr/local/share/texmf/ls-R
-rw-r--r-- 1 root root 1769 Jan 9 11:33 /var/lib/texmf/ls-R
lrwxrwxrwx 1 root root 29 Sep 4 11:34 /usr/share/texmf/ls-R -> /var/lib/texmf/ls-R-TEXMFMAIN
lrwxrwxrwx 1 root root 31 Dec 18 00:07 /usr/share/texlive/texmf-dist/ls-R -> /var/lib/texmf/ls-R-TEXLIVEDIST
lrwxrwxrwx 1 root root 31 Dec 18 00:07 /usr/share/texlive/texmf-dist/ls-R -> /var/lib/texmf/ls-R-TEXLIVEDIST
######################################
Config files
-rw-r--r-- 1 root root 475 Sep 10 06:56 /etc/texmf/web2c/texmf.cnf
lrwxrwxrwx 1 root root 33 Dec 18 00:07 /usr/share/texmf/web2c/fmtutil.cnf -> /var/lib/texmf/fmtutil.cnf-DEBIAN
lrwxrwxrwx 1 root root 32 Dec 18 00:07 /usr/share/texmf/web2c/updmap.cfg -> /var/lib/texmf/updmap.cfg-DEBIAN
-rw-r--r-- 1 root root 2862 Dec 20 17:56 /var/lib/texmf/tex/generic/config/language.dat
######################################
Files in /etc/texmf/web2c/
total 8
-rw-r--r-- 1 root root 283 Jun 15 2013 mktex.cnf
-rw-r--r-- 1 root root 475 Sep 10 06:56 texmf.cnf
######################################
md5sums of texmf.d
ca40c66f144b4bafc3e59a2dd32ecb9c /etc/texmf/texmf.d/00debian.cnf
-- System Information:
Debian Release: bookworm/sid
APT prefers testing
APT policy: (990, 'testing'), (500, 'testing-debug'), (50, 'unstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 5.15.0-2-amd64 (SMP w/8 CPU threads)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages texlive-base depends on:
ii debconf [debconf-2.0] 1.5.79
ii libpaper-utils 1.1.28+b1
ii sensible-utils 0.0.17
ii tex-common 6.17
ii texlive-binaries 2021.20210626.59705-1
ii ucf 3.0043
ii xdg-utils 1.1.3-4.1
Versions of packages texlive-base recommends:
ii lmodern 2.004.5-6.1
Versions of packages texlive-base suggests:
ii evince [postscript-viewer] 41.3-1
ii ghostscript [postscript-viewer] 9.55.0~dfsg-3
ii perl-tk 1:804.035-0.1+b1
pn xpdf | pdf-viewer <none>
pn xzdec <none>
Versions of packages tex-common depends on:
ii dpkg 1.21.1
ii ucf 3.0043
Versions of packages tex-common suggests:
ii debhelper 13.6
Versions of packages texlive-base is related to:
ii tex-common 6.17
ii texlive-binaries 2021.20210626.59705-1
-- debconf information excluded
-------------- next part --------------
diff -r -u /usr/share/texlive/texmf-dist/scripts/texlive/mktexlsr /usr/share/texlive/texmf-dist/scripts/texlive.modified/mktexlsr
--- /usr/share/texlive/texmf-dist/scripts/texlive/mktexlsr 2021-09-21 02:32:32.000000000 +0000
+++ /usr/share/texlive/texmf-dist/scripts/texlive.modified/mktexlsr 2022-01-10 10:45:41.309887577 +0000
@@ -213,15 +213,15 @@
# inserts the leading ./ for directory names, and removes ., .., and
# version control entries from the list. Also omit contents of any
# the version directories; sed apparently requires that we do that
- # operation in a separate invocation. We do not try to support colons
- # in directory names.
+ # operation in a separate invocation. Also remove double empty
+ # lines. We do not try to support colons in directory names.
#
echo "./:" >>"$db_file_tmp"
vc_dirs='\.\(bzr\|git\|hg\|svn\)\|_darcs'
(cd "$TEXMFLS_R" && \ls -LRa 2>/dev/null) \
| sed -e '/^$/{n;s%^\./%%;s%^%./%;}; /^\.$/d; /^\.\.$/d; /^'$vc_dirs'$/d;' \
-e '/^[\.\/]*lsR[0-9]*\.tmp:*$/d' \
- | sed -e /$vc_dirs'.*:$/,/^$/d' \
+ | sed -e /$vc_dirs'.*:$/,/^$/d' | cat -s \
>>"$db_file_tmp"
# To be really safe, a loop.
diff -r -u /usr/share/texlive/texmf-dist/scripts/texlive/mktexlsr.pl /usr/share/texlive/texmf-dist/scripts/texlive.modified/mktexlsr.pl
--- /usr/share/texlive/texmf-dist/scripts/texlive/mktexlsr.pl 2021-09-21 02:32:32.000000000 +0000
+++ /usr/share/texlive/texmf-dist/scripts/texlive.modified/mktexlsr.pl 2022-01-10 10:55:57.296887527 +0000
@@ -362,21 +362,20 @@
return 0;
}
open (LSR, ">$fn") || die "TeX::LSR writable but cannot open??; $!";
- print LSR "$lsrmagic\n\n";
- print LSR "./:\n"; # hardwired ./ for top-level files
+ print LSR "$lsrmagic\n";
+ print LSR "./:"; # hardwired ./ for top-level files
do_entry($self->{'tree'}, ".", $dosort);
close LSR;
return 1;
sub do_entry {
my ($t, $n, $sortit) = @_;
- print LSR "$n:\n";
+ print LSR "\n$n:\n";
my @sd;
for my $st ($sortit ? sort(keys %$t) : keys %$t) {
push (@sd, $st) if (ref($t->{$st}) eq 'HASH');
print LSR "$st\n";
}
- print LSR "\n";
for my $st ($sortit ? sort @sd : @sd) {
do_entry($t->{$st}, "$n/$st", $sortit);
}
-------------- next part --------------
diff -r -u /usr/share/texlive/texmf-dist/scripts/texlive/mktexlsr /usr/share/texlive/texmf-dist/scripts/texlive.modified/mktexlsr
--- /usr/share/texlive/texmf-dist/scripts/texlive/mktexlsr 2021-09-21 02:32:32.000000000 +0000
+++ /usr/share/texlive/texmf-dist/scripts/texlive.modified/mktexlsr 2022-01-10 10:45:41.309887577 +0000
@@ -213,15 +213,15 @@
# inserts the leading ./ for directory names, and removes ., .., and
# version control entries from the list. Also omit contents of any
# the version directories; sed apparently requires that we do that
- # operation in a separate invocation. We do not try to support colons
- # in directory names.
+ # operation in a separate invocation. Also remove double empty
+ # lines. We do not try to support colons in directory names.
#
echo "./:" >>"$db_file_tmp"
vc_dirs='\.\(bzr\|git\|hg\|svn\)\|_darcs'
(cd "$TEXMFLS_R" && \ls -LRa 2>/dev/null) \
| sed -e '/^$/{n;s%^\./%%;s%^%./%;}; /^\.$/d; /^\.\.$/d; /^'$vc_dirs'$/d;' \
-e '/^[\.\/]*lsR[0-9]*\.tmp:*$/d' \
- | sed -e /$vc_dirs'.*:$/,/^$/d' \
+ | sed -e /$vc_dirs'.*:$/,/^$/d' | cat -s \
>>"$db_file_tmp"
# To be really safe, a loop.
diff -r -u /usr/share/texlive/texmf-dist/scripts/texlive/mktexlsr.pl /usr/share/texlive/texmf-dist/scripts/texlive.modified/mktexlsr.pl
--- /usr/share/texlive/texmf-dist/scripts/texlive/mktexlsr.pl 2021-09-21 02:32:32.000000000 +0000
+++ /usr/share/texlive/texmf-dist/scripts/texlive.modified/mktexlsr.pl 2022-01-10 11:06:49.803902045 +0000
@@ -104,7 +104,7 @@
my $opt_verbose = (-t STDIN); # test whether connected to a terminal
my $opt_version = 0;
my $opt_output;
-my $opt_sort = 0; # for debugging sort output
+my $opt_sort = 1; # deprecated, kept for backward compatibility
my $opt_follow = 1; # follow links - check whether they are dirs or not
(my $prg = basename($0)) =~ s/\.pl$//;
@@ -141,7 +141,7 @@
$lsr = new TeX::LSR( root => $texmftree );
$lsr->loadtree();
$lsr->loadfile();
- $lsr->write( [filename => $fn, sort => $do_sort ] );
+ $lsr->write( filename => $fn );
$lsr->addfiles ( @files );
=head1 Methods
@@ -327,12 +327,10 @@
=pod
-=item C<< $lsr->write( [ filename => "$fn", sort => $val) >>
+=item C<< $lsr->write( filename => "$fn") >>
Writes out the C<ls-R> file, either to the default file name, or
-to C<$fn> if given. Entries within a directory are not sorted
-(not necessary), but sorting can be enforced by passing a true
-value to C<sort>.
+to C<$fn> if given.
Returns 1 on success, 0 on failure (and give warning).
@@ -342,9 +340,7 @@
my $self = shift;
my %params = @_;
my $fn;
- my $dosort = 0;
$fn = $params{'filename'} if $params{'filename'};
- $dosort = $params{'sort'};
if (!defined($self->{'root'})) {
warn "TeX::LSR: root undefined, cannot write.\n";
return 0;
@@ -362,23 +358,22 @@
return 0;
}
open (LSR, ">$fn") || die "TeX::LSR writable but cannot open??; $!";
- print LSR "$lsrmagic\n\n";
- print LSR "./:\n"; # hardwired ./ for top-level files
- do_entry($self->{'tree'}, ".", $dosort);
+ print LSR "$lsrmagic\n";
+ print LSR "./:"; # hardwired ./ for top-level files
+ do_entry($self->{'tree'}, ".");
close LSR;
return 1;
sub do_entry {
- my ($t, $n, $sortit) = @_;
- print LSR "$n:\n";
+ my ($t, $n) = @_;
+ print LSR "\n$n:\n";
my @sd;
- for my $st ($sortit ? sort(keys %$t) : keys %$t) {
+ for my $st (sort(keys %$t)) {
push (@sd, $st) if (ref($t->{$st}) eq 'HASH');
print LSR "$st\n";
}
- print LSR "\n";
- for my $st ($sortit ? sort @sd : @sd) {
- do_entry($t->{$st}, "$n/$st", $sortit);
+ for my $st (sort @sd) {
+ do_entry($t->{$st}, "$n/$st");
}
}
}
@@ -633,10 +628,10 @@
print "$prg: Dry run, not writing files.\n" if $opt_dryrun;
} elsif ($opt_output) {
#warn "writing to $opt_output\n";
- $lsr->write(filename => $opt_output, sort => $opt_sort);
+ $lsr->write(filename => $opt_output);
} else {
- #warn "writing with sort=$opt_sort\n";
- $lsr->write(sort => $opt_sort);
+ #warn "writing\n";
+ $lsr->write();
}
} else {
warn "$prg: cannot read files, skipping: $t\n";
More information about the Reproducible-bugs
mailing list