[From nobody Sat Oct 31 13:07:06 2026 Received: (at submit) by bugs.debian.org; 11 Jul 2025 13:51:19 +0000 X-Spam-Checker-Version: SpamAssassin 4.0.1-bugs.debian.org_2005_01_02 (2024-03-25) on buxtehude.debian.org X-Spam-Level: X-Spam-Status: No, score=-105.1 required=4.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FOURLA, FROMDEVELOPER,SPF_HELO_NONE,SPF_NONE,UNPARSEABLE_RELAY, USER_IN_DKIM_WELCOMELIST autolearn=ham autolearn_force=no version=4.0.1-bugs.debian.org_2005_01_02 X-Spam-Bayes: score:0.0000 Tokens: new, 24; hammy, 150; neutral, 315; spammy, 0. spammytokens: hammytokens:0.000-+--Hx-spam-relays-external:sk:stravin, 0.000-+--H*RT:sk:stravin, 0.000-+--Hx-spam-relays-external:311, 0.000-+--H*RT:311, 0.000-+--H*RT:108 Return-path: <smcv@debian.org> Received: from stravinsky.debian.org ([2001:41b8:202:deb::311:108]:48140) from C=NA, ST=NA, L=Ankh Morpork, O=Debian SMTP, OU=Debian SMTP CA, CN=stravinsky.debian.org, EMAIL=hostmaster@stravinsky.debian.org (verified) by buxtehude.debian.org with esmtps (TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.96) (envelope-from <smcv@debian.org>) id 1uaE9n-00ABgV-1m for submit@bugs.debian.org; Fri, 11 Jul 2025 13:51:19 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debian.org; s=smtpauto.stravinsky; h=X-Debian-User:Content-Transfer-Encoding:Content-Type :MIME-Version:Message-ID:Subject:To:From:Date:Reply-To:Cc:Content-ID: Content-Description:In-Reply-To:References; bh=76VXEYeaqZYQoxPPCRzdOWqqgpwxiY3pINWoYNJxydw=; b=k36wF1E239ZU1YcANsxJ+kx9aQ Pe5tgQuiaoRjXJiqOle/FlABOg+44oQSWALtb66FOVlFa2Y9sOxxp6hc+qcSPls4sFbbOUZtKi8TJ bhXPd79ySp7cq4YDLqV6VNfair1/O6s6AkXgRqDZYwUnk0K4C0YrEXLUf4KjGYGxzYlckNC58OXPp w/O7zU7OHAiuiseyZfCipRyhBQV2dXtxKSG3TXCt8OH0PrpR/N5fcrGF7mHjzM6ogwhgtQF/uz+P+ /hNTyLlJTyXYLQ2PDhJ1/kACB+5Il0ICHPBjOcevM/2J1XhZoLLeAiTe5VZz2B63B3h3rbihZRlE1 m7TY86rA==; Received: from authenticated user by stravinsky.debian.org with esmtpsa (TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.94.2) (envelope-from <smcv@debian.org>) id 1uaE9l-00GE0O-Cu for submit@bugs.debian.org; Fri, 11 Jul 2025 13:51:17 +0000 Date: Fri, 11 Jul 2025 14:51:16 +0100 From: Simon McVittie <smcv@debian.org> To: Debian Bug Tracking System <submit@bugs.debian.org> Subject: libsoup3: intermittent test failure: memory corruption in multithread-test Message-ID: <aHEW1LouPJc29dS7@remnant.pseudorandom.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-Mutt-Fcc: =.lists.debian/ X-Reportbug-Version: 13.2.0 X-Debian-User: smcv Delivered-To: submit@bugs.debian.org Source: libsoup3 Version: 3.6.4-2 Severity: important Tags: ftbfs help moreinfo Control: block 1035983 by -1 In a previous build of libsoup3 on the official buildds, multithread-test failed with evidence of memory corruption: https://buildd.debian.org/status/fetch.php?pkg=libsoup3&arch=amd64&ver=3.6.4-2&stamp=1737574120&raw=0 > 17/38 multithread-test RUNNING > >>> MALLOC_PERTURB_=181 G_TEST_SRCDIR=/build/reproducible-path/libsoup3-3.6.4/tests MESON_TEST_ITERATION=1 LD_LIBRARY_PATH=/build/reproducible-path/libsoup3-3.6.4/obj-x86_64-linux-gnu/tests:/build/reproducible-path/libsoup3-3.6.4/obj-x86_64-linux-gnu/libsoup MALLOC_CHECK_=2 MSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 G_TEST_BUILDDIR=/build/reproducible-path/libsoup3-3.6.4/obj-x86_64-linux-gnu/tests G_DEBUG=gc-friendly UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 /build/reproducible-path/libsoup3-3.6.4/obj-x86_64-linux-gnu/tests/multithread-test --debug > ▶ 15/38 /misc/cancel-while-reading/msg OK > ▶ 15/38 /misc/cancel-while-reading/req/immediate OK > ▶ 17/38 /multithread/no-main-context OK > ▶ 17/38 /multithread/basic/async OK > ▶ 17/38 /multithread/basic/sync OK > ▶ 17/38 /multithread/basic-ssl/async OK > ▶ 17/38 /multithread/basic-ssl/sync OK > ▶ 17/38 /multithread/basic-proxy/async OK > ▶ 17/38 /multithread/basic-proxy/sync OK > ▶ 17/38 /multithread/basic-no-main-thread/async OK > ▶ 17/38 /multithread/basic-no-main-thread/sync OK > ▶ 17/38 /multithread/basic-ssl-proxy/async OK > ▶ 17/38 /multithread/basic-ssl-proxy/sync OK > ▶ 17/38 /multithread/basic-http2/async OK > 17/38 multithread-test ERROR 0.09s killed by signal 6 SIGABRT > ――――――――――――――――――――――――――――――――――――― ✀ ――――――――――――――――――――――――――――――――――――― > stderr: > malloc(): unsorted double linked list corrupted > > (test program exited with status code -6) When the build was retried, all tests succeeded, so this is presumably intermittent or otherwise unreproducible. This is **not** the same as the failure mode that has been the most common in the past, where tests that use Apache fail with "Address already in use: AH00072: make_sock: could not bind to address 127.0.0.1:xxx". Similarly when I tried to add Salsa-CI to this package, my first attempt failed with a different indication of memory corruption: https://salsa.debian.org/gnome-team/libsoup3/-/jobs/7814730 > 17/38 multithread-test ERROR 15.76s killed by signal 6 SIGABRT > ――――――――――――――――――――――――――――――――――――― ✀ ――――――――――――――――――――――――――――――――――――― > stderr: > tcache_thread_shutdown(): unaligned tcache chunk detected > (test program exited with status code -6) > TAP parsing error: Too few tests run (expected 21, got 11) I think we can probably treat any evidence of memory corruption in this test as being essentially equivalent - if we corrupt the heap, then glibc can fail in several different ways as a result, none of which are meaningfully different. There seems to be a second failure mode where multithread-test times out (the default timeout is 60 seconds, but we use a 6x multiplier in the Debian packaging to accommodate slower architectures). That failure mode should be treated as a separate bug and is out of scope for this particular bug report, although it's possible that it has the same root cause. I will report that failure mode as a separate bug. To get an idea of how frequent this is, I tried these steps on the amd64 porterbox, barriere: 1. build libsoup3 (from unstable): schroot -c $chroot -r -- \ env DEB_BUILD_PROFILES=noudeb \ debuild -e CCACHE_DIR=$HOME/.ccache -e PATH=/usr/lib/ccache:$PATH -us -uc -B 2. run multithread-test repeatedly: schroot -c $chroot -r -- \ env -C obj-x86_64-linux-gnu \ DEB_BUILD_PROFILES=noudeb CCACHE_DIR=$HOME/.ccache PATH=/usr/lib/ccache:$PATH \ DEB_PYTHON_INSTALL_LAYOUT=deb LC_ALL=C.UTF-8 \ meson test --repeat 100 -j1 multithread-test (I tried this 3 times; optionally add --timeout-multiplier=6 to the `meson test` command-line to emulate the original package build more accurately) 3. read obj-x86_64-linux-gnu/meson-logs/testlog.txt for details of the failures, if any and my results were as follows: - 7 successes, 1 timeout, 1 failure with memory corruption - 19 successes, 1 timeout, 6 more successes, 1 more timeout, I cancelled the run at this point - 10 successes, 1 timeout, 15 more successes, 1 failure with memory corruption Anyone who wants libsoup3 tests to pass more often is invited to help to debug and fix this. If the failure is reproducible under valgrind, probably the easiest way is to build it in an environment that is suitable for interactive debugging, then run multithread-test repeatedly under valgrind, using something like meson test --repeat 100 --wrapper=./valgrind.sh multithread-test to get a backtrace for the memory corruption and figure out how it is happening. But this might not be possible if using valgrind perturbs the timing enough that the failure mode never actually happens. Or it might be possible to build libsoup3 (and ideally GLib too) with -fsanitize=address,undefined, and then run multithread-test repeatedly, as above; but, again, AddressSanitizer slows down the binaries, which could perturb the timing enough that the failure mode never actually happens. Annoyingly, it is not possible to run two or more copies of this test in parallel, so that cannot be used to get to a failure sooner (this is because each run of this test uses the same fixed filenames and port numbers). I am a member of the GNOME team, but not an Uploader of this particular package. I am aware that some project members believe that, because I have solved test issues it in the past, I should be held personally responsible for every test failure that occurs in GNOME. As per the Debian Social Contract §2.1.1, I decline that responsibility: I am not able to fix everything all of the time, and I'm sorry if the project considers my contributions to be inadequate. smcv ]