Bug#1050397: 1: yosys-src is huge

[Daniel Gröber]

Hi,

On Thu, Aug 24, 2023 at 03:00:36AM +0200, Daniel Gröber wrote:
> As long as nobody installs this package at least it won't waste space in
> the archive as the deb is compressed anyway. Post mortem wise, I'm not sure
> what's going on here. When I unpack the tarball the directory only ends up
> being around 50M (as it should be).

Ok so the deb(s) are actually still around 350-400M depending depending on
the architecture, because I also forgot to set Arch:all (ouch) and the
buildd built amd64 deb is actually 2.2G when unpacked.

The yosys-src I had build locally didn't end up that large tho.

I'm not entirely sure why the tarball got so big. Due to a misplaced
dependency the tarball was packed after dh_auto_build/_test, but I used
--files-from to prevent including any build artifacts.

This is the d/rule that builds yosys.tar:

    ORIG_TARBALL := ../$(DEB_SOURCE)_$(DEB_VERSION_UPSTREAM).orig.tar.gz
    debian/yosys-src/usr/src/yosys/yosys.tar: SHELL := /bin/bash
    debian/yosys-src/usr/src/yosys/yosys.tar: FORCE
            mkdir -p debian/yosys-src/usr/src/yosys
            tar --files-from <(tar -taf $(ORIG_TARBALL) | sed -r -e 's,[^/]+/,,') \
                --ignore-failed-read --transform 's,^,yosys/,' \
                --sort=name --mtime=@"$$SOURCE_DATE_EPOCH" \
                --owner=0 --group=0 --numeric-owner --format=gnu \
                -cf $@ \
                debian/{control,changelog,source,patches,rules}

Testing this again now on a dirty build tree that's 2.6G in size I get a
yosys.tar that's 70k. What the hell went wrong on the buildds?

Looking at the amd64 log I see no (relevant) errors and the expanded tar
command looks right:

    make[1]: Entering directory '/<<PKGBUILDDIR>>'
    mkdir -p debian/yosys-src/usr/src/yosys
    tar --files-from <(tar -taf ../yosys_0.30.orig.tar.gz | sed -r -e 's,[^/]+/,,') \
        --ignore-failed-read --transform 's,^,yosys/,' \
        --sort=name --mtime=@"$SOURCE_DATE_EPOCH" \
        --owner=0 --group=0 --numeric-owner --format=gnu \
        -cf debian/yosys-src/usr/src/yosys/yosys.tar \
        debian/{control,changelog,source,patches,rules}
    tar: tests/simple_abc9/abc9.v: Warning: Cannot stat: No such file or directory

The abc9.v thing is expected, hence --ignore-failed-read. Yet yosys.tar
ends up huge.

    -rw-r--r-- root/root 2289633280 2023-08-23 14:40 ./usr/src/yosys/yosys.tar

Looking at the file sizes of top offenders inside the buildd built amd64
yosys-src:

    $ tar -tvf ./usr/src/yosys/yosys.tar | sort -n -k3 -r | head -n20

    -rwxr-xr-x 0/0        22706286 2023-08-23 16:40 yosys/tests/bram/temp/tb_00_03.tb
    -rw-r--r-- 0/0        19693600 2023-08-23 16:40 yosys/passes/sat/sim.o
    -rw-r--r-- 0/0        18143464 2023-08-23 16:40 yosys/kernel/rtlil.o
    -rw-r--r-- 0/0        16335168 2023-08-23 16:40 yosys/passes/sat/qbfsat.o
    -rw-r--r-- 0/0        15009576 2023-08-23 16:40 yosys/passes/opt/share.o
    -rw-r--r-- 0/0        14109856 2023-08-23 16:40 yosys/backends/cxxrtl/cxxrtl_backend.o
    -rw-r--r-- 0/0        14010040 2023-08-23 16:40 yosys/passes/sat/recover_names.o
    -rw-r--r-- 0/0        13766864 2023-08-23 16:40 yosys/passes/opt/opt_expr.o
    -rw-r--r-- 0/0        13337960 2023-08-23 16:40 yosys/passes/techmap/abc.o
    -rw-r--r-- 0/0        13268272 2023-08-23 16:40 yosys/passes/techmap/abc9_ops.o
    -rw-r--r-- 0/0        13182088 2023-08-23 16:40 yosys/backends/smt2/smt2.o
    -rw-r--r-- 0/0        12897840 2023-08-23 16:40 yosys/passes/memory/memory_libmap.o
    -rw-r--r-- 0/0        12610552 2023-08-23 16:40 yosys/passes/pmgen/xilinx_dsp.o
    -rw-r--r-- 0/0        12223360 2023-08-23 16:40 yosys/passes/pmgen/test_pmgen.o
    -rw-r--r-- 0/0        12123208 2023-08-23 16:40 yosys/passes/techmap/flowmap.o
    -rw-r--r-- 0/0        12104440 2023-08-23 16:40 yosys/passes/techmap/techmap.o
    -rw-r--r-- 0/0        12051072 2023-08-23 16:40 yosys/libs/subcircuit/subcircuit.o
    -rw-r--r-- 0/0        11872992 2023-08-23 16:40 yosys/passes/sat/sat.o
    -rwxr-xr-x 0/0        11443185 2023-08-23 16:40 yosys/tests/bram/temp/tb_01_03.tb
    -rwxr-xr-x 0/0        11413171 2023-08-23 16:40 yosys/tests/memlib/t_async_big_block.out/t_async_big_block_tb_syn0

That's all build output that shouldn't have been included because of the
explicit --files-files.

I'm mystified.

--Daniel