Bug#877813: reprotest: regression between 0.7.1 and 0.7.2

gregor herrmann gregoa at debian.org
Thu Oct 26 23:13:35 UTC 2017 

On Thu, 26 Oct 2017 22:40:00 +0000, Ximin Luo wrote:

> gregor herrmann:
> > - reprotest called as 
> >   env -u TMPDIR -- reprotest --variations=+all,-build_path,-user_group --verbosity 1 . -- schroot default 2>&1 | [..]
> Thanks, this helped me reproduce the bug locally. I was having
> trouble before, because the bug only occurs when fileordering is
> switched on but user_group is not switched on.

Thanks, that's excellent news.
And sorry for not providing the command I used earlier.
 
> The bug is because disorderfs was being run as root inside the
> schroot, but the build was being run as the normal unprivileged
> user. This was because I was dropping privs in the wrong place,
> I've fixed in commit e367967, see if it works for you:
> https://anonscm.debian.org/git/reproducible/reprotest.git/commit/?id=e367967

I've now built a package from git HEAD (3efb86f), and .. hm .. I guess
this bug is fixed, it's just that I'm running into another issue
which seems related to another recent commit (the domain_host
feature):

#v+
INFO:root:build successful, copying artifacts
INFO:root:copying /tmp/reprotest.5Ab5ie/artifacts-control/ back from virtual server's /tmp/tmpo1_ukgqc/control
INFO:root:build "experiment-1": vary environment, FIX build_path, FIX user_group, vary fileordering, vary domain_host, vary home, vary kernel, vary locales, vary exec_path, vary time, vary timezone, vary umask
WARNING:root:Not using sudo for domain_host; it is recommended. Your build may fail.
WARNING:root:Be sure to `echo 1 > /proc/sys/kernel/unprivileged_userns_clone` if on a Debian system.
INFO:root:copying . over to virtual server's /tmp/reprotest.5Ab5ie/build-experiment-1/
INFO:root:starting build with source directory: /tmp/reprotest.5Ab5ie/build-experiment-1/, artifact pattern: ./../*.deb
Note, using directory './.' to get the build dependencies
Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
INFO:root:executing build in /tmp/reprotest.5Ab5ie/const_build_path/
unshare: unshare failed: ²»ÔÊÐíµIJÙ×÷
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/reprotest/__init__.py", line 802, in run
    return 0 if check_func(*check_args) else 1
  File "/usr/lib/python3/dist-packages/reprotest/__init__.py", line 351, in check
    local_dists = [proc.send(nv) for nv in zip(bnames, build_variations)]
  File "/usr/lib/python3/dist-packages/reprotest/__init__.py", line 351, in <listcomp>
    local_dists = [proc.send(nv) for nv in zip(bnames, build_variations)]
  File "/usr/lib/python3/dist-packages/reprotest/__init__.py", line 317, in corun_builds
    bctx.run_build(testbed, build, os.environ, artifact_pattern, testbed_build_pre, no_clean_on_error)
  File "/usr/lib/python3/dist-packages/reprotest/__init__.py", line 211, in run_build
    kind='build')
  File "/usr/lib/python3/dist-packages/reprotest/__init__.py", line 63, in check_exec2
    adtlog.AutopkgtestError)
  File "/usr/lib/python3/dist-packages/reprotest/__init__.py", line 69, in bomb
    raise _type(m)
reprotest.lib.adtlog.AutopkgtestError: "su -s /bin/sh gregoa -c set -e; run_build() {
    mkdir -p /tmp/reprotest.5Ab5ie/build-experiment-1-aux && \
    mv /tmp/reprotest.5Ab5ie/build-experiment-1/ /tmp/reprotest.5Ab5ie/const_build_path && \
    mv /tmp/reprotest.5Ab5ie/const_build_path/ /tmp/reprotest.5Ab5ie/const_build_path-before-disorderfs/ && \
    mkdir -p /tmp/reprotest.5Ab5ie/const_build_path/ && \
    disorderfs -q --shuffle-dirents=yes /tmp/reprotest.5Ab5ie/const_build_path-before-disorderfs/ /tmp/reprotest.5Ab5ie/const_build_path/ && \
    umask 0002 && \
    export REPROTEST_BUILD_PATH=/tmp/reprotest.5Ab5ie/const_build_path/ && \
    export REPROTEST_UMASK=$(umask) && \
    faketime +373days+7hours+13minutes \
    linux32 \
    unshare -r --uts sh -ec '
            hostname reprotest-capture-hostname
            domainname "reprotest-capture-domainname"
            "$@"' - \
    sh -ec 'cd "$REPROTEST_BUILD_PATH"; unset REPROTEST_BUILD_PATH; umask "$REPROTEST_UMASK"; unset REPROTEST_UMASK; dpkg-buildpackage --no-sign -b'
}

cleanup() {
    __c=0; \
    export PATH="/tmp/reprotest.5Ab5ie/bin:$PATH" || __c=$?; \
    fusermount -u /tmp/reprotest.5Ab5ie/const_build_path/ || __c=$?; \
    rmdir /tmp/reprotest.5Ab5ie/const_build_path/ || __c=$?; \
    mv /tmp/reprotest.5Ab5ie/const_build_path-before-disorderfs/ /tmp/reprotest.5Ab5ie/const_build_path/ || __c=$?; \
    mv /tmp/reprotest.5Ab5ie/const_build_path /tmp/reprotest.5Ab5ie/build-experiment-1/ || __c=$?; \
    rm -rf /tmp/reprotest.5Ab5ie/build-experiment-1-aux || __c=$?; \
    exit $__c
}

trap '( cleanup )' HUP INT QUIT ABRT TERM PIPE # FIXME doesn't quite work reliably yet

if ( run_build ); then ( cleanup ); else
    __x=$?; # save the exit code of run_build
    if ( ! false ); then
        if ( cleanup ); then :; else echo >&2 "cleanup failed with exit code $?"; fi;
    fi
    exit $__x
fi" failed with status 1
#v-

The recommended `echo 1 > /proc/sys/kernel/unprivileged_userns_clone'
(as root on the host) doesn't help.

Adding -domain_host to --variations worksaround the issue as
expected, and confirms that the original issue is indeed fixed.

Yay! Thanks again.


> > What I don't understand: The second build starts with:
> > 
> > [..]
> > INFO:root:starting build with source directory: /tmp/autopkgtest.tgbi5X/build-experiment-1/, artifact pattern: ./../*.deb
> > ..
> > INFO:root:executing build in /tmp/autopkgtest.tgbi5X/const_build_path/
> > tail: Î޷¨´ò¿ª'debian/changelog' ¶ÁȡÊý¾: ȨÏ޲»¹»
> > dpkg-buildpackage: error: tail of debian/changelog subprocess returned exit status 1
> > INFO:root:build successful, copying artifacts
> > /bin/sh: 2: cd: can't cd to /tmp/autopkgtest.tgbi5X/build-experiment-1/
> > 
> > but there is no /tmp/autopkgtest.tgbi5X/build-experiment-1/ directory
> > anywhere:
> > 
> > [..]
> 
> This is explained by the fact that reprotest moves stuff from
> build-experiment-1 to const_build_path, in order to run both builds
> in the same build path. The build failed, so reprotest didn't run
> the cleanup to move the directory back. Then, reprotest thought the
> build was successful (it says "INFO:root:build successful" in your
> log) so it tried to do more stuff with
> /tmp/autopkgtest.tgbi5X/build-experiment-1/, but it's not there
> because the build actually failed.
> 
> The fact that reprotest thought the build was successful, was a bug
> in 0.7.3 with --no-clean-on-error that I just noticed, which I
> accidentally fixed in a refactoring just before I fixed this bug.

Oh, I see.
Glad you found & fixed this bug as well.


Cheers,
gregor

-- 
 .''`.  https://info.comodo.priv.at/ - Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
   `-   NP: Dire Straits: Tunnel Of Love
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 963 bytes
Desc: Digital Signature
URL: <http://lists.alioth.debian.org/pipermail/reproducible-builds/attachments/20171027/a9ab9d57/attachment.sig>

More information about the Reproducible-builds mailing list