saslauthd memory leak with PAM

[Gabor Gombas]

Hi,

I got annoyed by saslauthd consuming more than 2Gig of RAM so I started
looking into this issue. My findings:

- The leak does NOT happen on successful authentication. I sent 500000
  valid auth. requests to saslauthd and its memory usage did not
  increase.

- I sent just a couple of invalid authentication requests and
  saslauthd's memory usage started to climb. So this is a trivially
  exploitable remote DoS (send a large amount of bad passwords to any
  sasl-using service and wait until the OOM killer kicks in and renders
  your box useless).

- The leak is NOT related to libpam-mysql, it happens with the plain
  pam_unix module as well.

- When using just pam_unix, valgrind gives the following trace segment:

==17824== 68 bytes in 17 blocks are definitely lost in loss record 7 of 7
==17824==    at 0x40064B0: malloc (vg_replace_malloc.c:149)
==17824==    by 0x425AAF12: (within /lib/ld-2.5.so)
==17824==    by 0x425AC5B4: (within /lib/ld-2.5.so)
==17824==    by 0x425B6450: (within /lib/ld-2.5.so)
==17824==    by 0x425B2401: (within /lib/ld-2.5.so)
==17824==    by 0x425B5E9D: (within /lib/ld-2.5.so)
==17824==    by 0x42709C2C: (within /lib/i686/cmov/libdl-2.5.so)
==17824==    by 0x425B2401: (within /lib/ld-2.5.so)
==17824==    by 0x4270A2AB: (within /lib/i686/cmov/libdl-2.5.so)
==17824==    by 0x42709B60: dlopen (in /lib/i686/cmov/libdl-2.5.so)
==17824==    by 0x4352838F: (within /lib/libpam.so.0.79)
==17824==    by 0x4352852B: (within /lib/libpam.so.0.79)
==17824==    by 0x435292F3: _pam_init_handlers (in /lib/libpam.so.0.79)
==17824==    by 0x4352726E: pam_start (in /lib/libpam.so.0.79)
==17824==    by 0x804B1F4: auth_pam (auth_pam.c:207)

The number of lost blocks equals to the invalid authentication requests
I sent to saslauthd. This seems to suggest that something forgets to
clean up when an authentication request fails.

The amount of leaked memory seems to be dependent on the PAM module
being used. pam_unix seems to be the 'nicest'; with libpam_mysql, I get
about 60 KiB of memory lost for every failed authentication attempt,
according to 'ps' output.

Gabor

-- 
     ---------------------------------------------------------
     MTA SZTAKI Computer and Automation Research Institute
                Hungarian Academy of Sciences
     ---------------------------------------------------------