[Pkg-openldap-devel] Bug#660917: fileno ulimit regression: slapd rejects connections approaching 1024 simultaneous connections

[Chris Hiestand]

Package: slapd
Version: 2.4.23-7.2
Severity: normal
Tags: patch

I believe in previous versions of slapd there was a line in /etc/init.d/slapd
setting the ulimit -n higher. However this is not present in the current version.

The original bug was reported in #378261. I am re-using the perl script in that bug report
to reproduce the problem.

A vanilla squeeze install of slapd (amd64) will max out before 1024 connections,
and the nofiles limit is 1024 (verified in /proc/$pid/limits)

The syslog errors are:
Feb 21 09:13:49 test slapd[3194]: warning: cannot open /etc/hosts.allow: Too many open files
Feb 21 09:13:49 test slapd[3194]: warning: cannot open /etc/hosts.deny: Too many open files
Feb 21 09:13:49 test slapd[3194]: warning: cannot open /etc/hosts.allow: Too many open files
Feb 21 09:13:49 test slapd[3194]: warning: cannot open /etc/hosts.deny: Too many open files
...

As in the previous bug report (378261) I tried recompiling without libwrap, and can still
reproduce the problem, though the error message changes slightly.

Proposed patches are inline below. Varying PAM stacks might effect setting the ulimit,
so I thought it was important to add a check and warn if the ulimit cannot be set.

With these patches in place, I can no longer reproduce the problem up until ~8192 connections.

Relatedly I was curious what the ceiling was, I am guessing it is whenever you run out of one of:
A. free tcp ports
B. free memory

In simple connection-only (no-load) testing I can get to ~28000 connections before my client
runs out of tcp ports to test with, and my VM server with 756MB of RAM runs out of memory.
This suggests 8192 is a conservative default value.


--- slapd.default	2012-02-22 11:29:01.000000000 -0800
+++ slapd.default.new	2012-02-22 11:28:43.000000000 -0800
@@ -43,3 +43,7 @@
 
 # Additional options to pass to slapd
 SLAPD_OPTIONS=""
+
+# The nofile user limit: should be larger than the maximum number of
+# simultaneous connections.
+SLAPD_NOFILES=8192



--- slapd.init	2012-02-17 10:58:42.000000000 -0800
+++ slapd.init.new	2012-02-22 11:37:48.000000000 -0800
@@ -57,6 +57,11 @@
 	SLAPD_OPTIONS="-F $SLAPD_CONF $SLAPD_OPTIONS"
 fi
 
+# Set the nofiles limit to a reasonable value if not set
+if [ -z "$SLAPD_NOFILES" ]; then
+  SLAPD_NOFILES=8192
+fi
+
 # Find out the name of slapd's pid file
 if [ -z "$SLAPD_PIDFILE" ]; then
 	# If using old one-file configuration scheme
@@ -132,6 +137,11 @@
 # Start the slapd daemon and capture the error message if any to 
 # $reason.
 start_slapd() {
+  ulimit -Hn $SLAPD_NOFILES
+  ulimit -Sn $SLAPD_NOFILES
+  if [ "`ulimit -n`" -ne "$SLAPD_NOFILES" ]; then
+    log_warning_msg "Could not set ulimit, slapd may prematurely reject connections under load"
+  fi
 	if [ -z "$SLAPD_SERVICES" ]; then
 		reason="`start-stop-daemon --start --quiet --oknodo \
 			--pidfile "$SLAPD_PIDFILE" \




-- System Information:
Debian Release: 6.0.4
  APT prefers stable
  APT policy: (900, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.32-5-amd64 (SMP w/12 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages slapd depends on:
ii  adduser                3.112+nmu2        add and remove users and groups
ii  coreutils              8.5-1             GNU core utilities
ii  debconf [debconf-2.0]  1.5.36.1          Debian configuration management sy
ii  libc6                  2.11.3-2          Embedded GNU C Library: Shared lib
ii  libdb4.8               4.8.30-2          Berkeley v4.8 Database Libraries [
ii  libgnutls26            2.8.6-1+squeeze1  the GNU TLS library - runtime libr
ii  libldap-2.4-2          2.4.23-7.2        OpenLDAP libraries
ii  libltdl7               2.2.6b-2          A system independent dlopen wrappe
ii  libperl5.10            5.10.1-17squeeze3 shared Perl library
ii  libsasl2-2             2.1.23.dfsg1-7    Cyrus SASL - authentication abstra
ii  libslp1                1.2.1-7.8         OpenSLP libraries
ii  libwrap0               7.6.q-19          Wietse Venema's TCP wrappers libra
ii  lsb-base               3.2-23.2squeeze1  Linux Standard Base 3.2 init scrip
ii  perl [libmime-base64-p 5.10.1-17squeeze3 Larry Wall's Practical Extraction 
ii  psmisc                 22.11-1           utilities that use the proc file s
ii  unixodbc               2.2.14p2-1        ODBC tools libraries

Versions of packages slapd recommends:
ii  libsasl2-modules          2.1.23.dfsg1-7 Cyrus SASL - pluggable authenticat

Versions of packages slapd suggests:
ii  ldap-utils                    2.4.23-7.2 OpenLDAP utilities

-- Configuration Files:
/etc/default/slapd changed [not included]
/etc/init.d/slapd changed [not included]

-- debconf-show failed