[Python-apps-team] Bug#781844: rdiff-backup: crashed after hitting process memory limit

[Matt Taggart]

Package: rdiff-backup
Version: 1.2.8-7

I am running into a situation where rdiff-backup grows in memory usage 
until it crashes.

Client: amd64 userspace, amd64 kernel, 16gb of RAM
Server: i386 userspace, amd64 kernel, 32gb of RAM
Both are using rdiff-backup 1.2.8-7 and librsync 0.9.7-9 (on wheezy)

When backing up a filesystem that contains tens of thousands of files, I 
see the memory usage of rdiff-backup on the server grow until it reaches 
4gb and then I get the following error:

Feb 28 05:16:06 Error: Found interrupted initial backup. Removing... 
Exception '' raised of class '<type 'exceptions.MemoryError'>': File 
"/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in 
error_check_Main try: Main(arglist) File "/usr/lib/python2.7/dist-packages/r
diff_backup/Main.py", line 324, in Main take_action(rps) File 
"/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 280, in 
take_action elif action == "backup": Backup(rps[0], rps[1]) File 
"/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 346, in 
Backup backup.Mirror(rpin, rpout) File "/usr/lib/python2.7/dist-packages/rdi
ff_backup/backup.py", line 38, in Mirror DestS.patch(dest_rpath, 
source_diffiter) File "/usr/lib/python2.7/dist-packages/rdiff_backup/connect
ion.py", line 450, in __call__ return apply(self.connection.reval, 
(self.name,) + args) File "/usr/lib/python2.7/dist-packages/rdiff_backup/con
nection.py", line 370, in reval if isinstance(result, Exception): raise 
result Traceback (most recent call last): File "/usr/bin/rdiff-backup", 
line 30, in <module> rdiff_backup.Main.error_check_Main(sys.argv[1:]) File 
"/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 304, in 
error_check_Main try: Main(arglist) File "/usr/lib/python2.7/dist-packages/r
diff_backup/Main.py", line 324, in Main take_action(rps) File 
"/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 280, in 
take_action elif action == "backup": Backup(rps[0], rps[1]) File 
"/usr/lib/python2.7/dist-packages/rdiff_backup/Main.py", line 346, in 
Backup backup.Mirror(rpin, rpout) File "/usr/lib/python2.7/dist-packages/rdi
ff_backup/backup.py", line 38, in Mirror DestS.patch(dest_rpath, 
source_diffiter) File "/usr/lib/python2.7/dist-packages/rdiff_backup/connect
ion.py", line 450, in __call__ return apply(self.connection.reval, 
(self.name,) + args) File "/usr/lib/python2.7/dist-packages/rdiff_backup/con
nection.py", line 370, in reval if isinstance(result, Exception): raise 
result MemoryError Fatal Error: Lost connection to the remote system

The server was recently upgraded from older i386 to new amd64 hardware, so 
that is why we have the i386 userspace but are running the amd64 kernel (to 
support the new hardware and utilize all the RAM).

I think there might be multiple issues:

A) Why is it using so much memory? I thought even if you have a filesystem 
with tons of hardlinks to keep track of it was only supposed to use tens of 
megabytes, is growing to 4gb is maybe a memory leak bug?

B) It didn't used to crash before the upgrade to an amd64 kernel, and given 
it crashes at 4gb (32bit per process limit) I suspect maybe having the 
mixed userspace/kernel has uncovered another bug.

I found a post on the mailing list that may be related

https://lists.nongnu.org/archive/html/rdiff-backup-users/2015-02/msg00011.ht
ml

Thanks,

-- 
Matt Taggart
taggart at debian.org