[buildd-tools-devel] Some Debian package upgrades are corrupting rsync "quick check" backups

Mon Aug 28 06:06:06 UTC 2017

On Sat, 2017-05-13 at 22:48 +0200, Aurelien Jarno wrote:
> On 2017-05-13 21:34, Aurelien Jarno wrote:
> > On 2017-05-13 17:52, Mattia Rizzolo wrote:
> > > On Sat, May 13, 2017 at 03:44:57PM +0100, Chris Lamb wrote:
> > > >  a) Has anything changed in the meantime?
> > > 
> > > Yes: sbuild stopped repeating the changelog time taking it from
> > > the last
> > > entry, and will instead generate a new timestamp based on the
> > > current
> > > time:
> > > 
> > >   * For binNMUs, instead of copying the timestamp of the last
> > > changelog entry,
> > >     generate a new one (closes: #843773)
> > > 
> > > In version 0.73.0-1.
> > 
> > And I am glad that after all that months with people talking about
> > the
> > issue, I finally got a detailed description of the issue and a
> > pointer
> > to the commit to backport. I'll work on that.
> 
> The above change should now be deployed on most jessie based buildds,
> it's only missing on the buildds that are currently down.

Original thread author here reporting to beware that some rsync data
corruption can still become apparent after all this time.

It's been a while since I did a full rsync checksum test. Decided to do
one after a recent upgrade that includes clang 3.9 related files. These
are Debian systems that default to unstable BUT include all debian apt
sources including experimental/unstable/testing/stable/oldstable.

I found these four corrupted files in my rsync backups:

var/lib/dpkg/info/clang-3.9.md5sums
var/lib/dpkg/info/libclang-common-3.9-dev.md5sums
var/lib/dpkg/info/libclang1-3.9:amd64.md5sums
var/lib/dpkg/info/libllvm3.9:amd64.md5sums

The latest packages were installed from this repository:

Get:17 https://cdn-aws.deb.debian.org/debian unstable/main amd64 clang-3.9 amd64 1:3.9.1-11 [37.3 MB]
Get:18 https://cdn-aws.deb.debian.org/debian unstable/main amd64 libclang-common-3.9-dev amd64 1:3.9.1-11 [2,587 kB]
Get:19 https://cdn-aws.deb.debian.org/debian unstable/main amd64 libllvm3.9 amd64 1:3.9.1-11 [11.4 MB]
Get:20 https://cdn-aws.deb.debian.org/debian unstable/main amd64 libclang1-3.9 amd64 1:3.9.1-11 [5,896 kB]                                                                  

Here are the mtimes of the newly install files:

# ls -la /var/lib/dpkg/info/clang-3.9.md5sums /var/lib/dpkg/info/libclang-common-3.9-dev.md5sums /var/lib/dpkg/info/libclang1-3.9:amd64.md5sums /var/lib/dpkg/info/libllvm3.9:amd64.md5sums
-rw-r--r-- 1 root root 10855 Jun 19 05:12 /var/lib/dpkg/info/clang-3.9.md5sums
-rw-r--r-- 1 root root   384 Jun 19 05:12 /var/lib/dpkg/info/libclang1-3.9:amd64.md5sums
-rw-r--r-- 1 root root 18320 Jun 19 05:12 /var/lib/dpkg/info/libclang-common-3.9-dev.md5sums
-rw-r--r-- 1 root root   371 Jun 19 05:12 /var/lib/dpkg/info/libllvm3.9:amd64.md5sums

Here's a diff of /var/lib/dpkg/info/clang-3.9.md5sums (rsync backup vs current file):

2,11c2,11
< 3cbbedf0c71035cd3029dfe1b052e581  usr/lib/llvm-3.9/bin/c-index-test
< 0fdde07673439d60670efed27456169a  usr/lib/llvm-3.9/bin/clang
< aa1260a7741de9cdf0c748f8b7462c1c  usr/lib/llvm-3.9/bin/clang-apply-replacements
< 996c76e1c1ab4075d016d4a860a616a2  usr/lib/llvm-3.9/bin/clang-check
< 8021eba7391ad79567cf0d2cc71bacfa  usr/lib/llvm-3.9/bin/clang-include-fixer
< 048cc918522e9099d7085d734e7356cc  usr/lib/llvm-3.9/bin/clang-query
< 7dae8990cdbf679b19e370d62fea05e1  usr/lib/llvm-3.9/bin/clang-rename
< 3f7c675b0be7c94c88e6b27f665236d2  usr/lib/llvm-3.9/bin/find-all-symbols
< a2920ac65d5085773747078cdd099f87  usr/lib/llvm-3.9/bin/modularize
< dd36711af0597df2c78f3583fb914728  usr/lib/llvm-3.9/bin/sancov
---
> e48899b8ed6e12602edaec4be4cf5ba8  usr/lib/llvm-3.9/bin/c-index-test
> c523712273fb46731e4ad88de23a00a6  usr/lib/llvm-3.9/bin/clang
> 1d59562b0f0167b20b3f47af8da65bb2  usr/lib/llvm-3.9/bin/clang-apply-replacements
> de8b1949a36135e7f5eac0016e99c813  usr/lib/llvm-3.9/bin/clang-check
> 458274598da467da470be2404374432e  usr/lib/llvm-3.9/bin/clang-include-fixer
> cca520673bf5586037e66b8384944576  usr/lib/llvm-3.9/bin/clang-query
> 3d3b408633a6ffe416c28a281082ffde  usr/lib/llvm-3.9/bin/clang-rename
> 288d8965a7fa9389c37233d8751683a9  usr/lib/llvm-3.9/bin/find-all-symbols
> ec0b482ab1af827ea958b51a87a096f8  usr/lib/llvm-3.9/bin/modularize
> 21c34b02091b53c37a90306f1dea5d6a  usr/lib/llvm-3.9/bin/sancov
109c109
< 32db7068b426f045c64d8642c1e6c022  usr/share/doc/clang-3.9/changelog.Debian.gz
---
> 2ec59c559a5c304cacddea2f02d74697  usr/share/doc/clang-3.9/changelog.Debian.gz
116,123c116,123
< 3a8deaf272eff95b110efb05ebc9ee9d  usr/share/man/man1/clang-apply-replacements-3.9.1.gz
< efe297e129ca7d6823ac46f5b7a58644  usr/share/man/man1/clang-check-3.9.1.gz
< 114d671d66872a188832cad20062cd7b  usr/share/man/man1/clang-include-fixer-3.9.1.gz
< 8992eb9feffb6b90c015a28785d0b33b  usr/share/man/man1/clang-query-3.9.1.gz
< c69a128b81404ecc2eac7e759608fd15  usr/share/man/man1/clang-rename-3.9.1.gz
< 14f265858cf398647aa0be918e65dd6b  usr/share/man/man1/find-all-symbols-3.9.1.gz
< e54fd298f2fea34b272d44b7c8c372a2  usr/share/man/man1/modularize-3.9.1.gz
< c062a1192879288d88211d8804c740c4  usr/share/man/man1/sancov-3.9.1.gz
---
> 184960744b144151add1be27f96e5a2f  usr/share/man/man1/clang-apply-replacements-3.9.1.gz
> 68480b6ce324eae9ce6771eb30ae0093  usr/share/man/man1/clang-check-3.9.1.gz
> 1f16d5c2d51a41aa723ae820817b3b5b  usr/share/man/man1/clang-include-fixer-3.9.1.gz
> 2da1698364052c490c02bae9a9c81907  usr/share/man/man1/clang-query-3.9.1.gz
> cae329a9d569ef1cdf1d13754c872131  usr/share/man/man1/clang-rename-3.9.1.gz
> 7c29374033ef542c082fa437ff158765  usr/share/man/man1/find-all-symbols-3.9.1.gz
> a1522b2d84d25ca07c5fc195b0a1ffcf  usr/share/man/man1/modularize-3.9.1.gz
> 41fb9f4cfb33c8a49bac4f88dc5a762f  usr/share/man/man1/sancov-3.9.1.gz

It appears the rsync backups were corrupted by new packages recently
build for unstable that were source-modified in June:
<http://metadata.ftp-master.debian.org/changelogs/main/l/llvm-toolchain-3.9/llvm-toolchain-3.9_3.9.1-11_changelog>

Regards,
Adam Warner