[Pkg-openldap-devel] Bug#725091: Bug#725091: slapd with memory leak in active sync

[Thomas Sesselmann]

Hi Ryan,

On 02.10.2013 00:52, wrote Ryan Tandy:
> Hi Thomas,
> 
> On Tue, Oct 1, 2013 at 4:10 AM, Thomas Sesselmann
> <thomas.sesselmann at uni-jena.de> wrote:
>> We installed a multimaster replication setup.
>>
>> Now if we modifies some attributes and groups-memberships and the memory
>> use of the slapd on the 'master' increase extremely (>10G) until out of memory.
> 
> I think you might be experiencing ITS#7292. What do you think? If you
> bring up a new server with the same configuration and an empty
> database and allow it to start replicating, like in the ITS, do you
> observe the same memory usage?
> 
> http://www.openldap.org/its/index.cgi/Software%20Bugs?id=7292

I don't know, on initial update/sync there are no problems.
The problem exists only if the two master servers are in sync
with the persist connection (syncrepl type=refreshAndPersist).

> It was fixed upstream in 2.4.32 by a series of commits:
> 
> http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=shortlog;h=dc912776;hp=65c0cd19
> 
> If it's easy for you to rebuild a package and deploy it in a testing
> environment, you might try doing so with those changes integrated.

We now build an independent test-environment. So we can install there
test packages ...
Rebuild a source packages with some diffs is not impossible, but will
spend some time, because we have to learn it first ;)


>> We use the online config. In attachment is the corresponding slapd.conf.
> 
> I tried to reproduce your report but wasn't successful yet. Is it
> possible for you to reduce the testcase any further? For example, can
> you still trigger it if you omit some of the overlays?

We delete all overlays (except syncprov) and the problem still exists.
The modules are loaded but not used in the configuration,
if this should change something?

We modifies the member of some groups (delete/add). There are much users
in the groups (>20000), perhaps the result is so worse.

Here are the Skript we used:
-------modify.pl ----------------
#!/usr/bin/perl
use strict;
use warnings;
use Net::LDAPS;
use Net::LDAP::Util qw(ldap_error_text);

my $hostname    = ['ldaptest2.rz.uni-jena.de'];
my $binddn      = 'ou=local,dc=uni-jena, dc=de';
my $auth        = 'uid=xxxy';
my $passwd      = 'xxxx';

my $basedn = "ou=users,dc=uni-jena,dc=de";
my $ldap = Net::LDAP->new($hostname) or die "Unable to connect to LDAP server $hostname: $@\n";
my $mesg = $ldap-> start_tls() or die "Unable to StartTLS: $@\n";
$ldap->bind( "$auth,$binddn", password => $passwd, version => 3 ) or die "Binding to the LDAP-Server";

##searching on LDAP
my $searchresult = $ldap->search(base =>$basedn, filter  => "(&(uid=*)(mail=*)(|(mailHost=mail1.rz.uni-jena.de)))", attrs => ['*', 'memberOf']) or die "Searching the
LDAP-Server";
print STDOUT "(".$searchresult->count.")\n";

sub in_array {
    my ($arr,$search_for) = @_;
    return grep {$search_for eq $_} @$arr;
}

### For all LDAP-USERS ###

foreach my $entry ($searchresult->entries) {
    my @AddArray        = [];
    my @ReplaceArray    = [];
    my @DeleteArray     = [];

    print STDOUT $entry->dn, "\n";

    my $memberOfs = $entry->get_value('memberOf', asref => 1);

    ### DELETE IN GROUP SMTP ###
    if(in_array($memberOfs,'cn=smtp,ou=local,ou=groups,dc=uni-jena,dc=de')){
        $ldap->modify('cn=smtp,ou=local,ou=groups,dc=uni-jena,dc=de', delete => {member => [$entry->dn]}) or die "Delete group SMTP";
    }

    ### DELETE IN GROUP SMTP-AUTH ###
    if(in_array($memberOfs,'cn=smtp-auth,ou=local,ou=groups,dc=uni-jena,dc=de')){
        $ldap->modify('cn=smtp-auth,ou=local,ou=groups,dc=uni-jena,dc=de', delete => {member => [$entry->dn]}) or die "Delete group SMTP-AUTH";
    }

    ### DELETE IN GROUP IMAP ###
    if(in_array($memberOfs,'cn=imap,ou=local,ou=groups,dc=uni-jena,dc=de')){
        $ldap->modify('cn=imap,ou=local,ou=groups,dc=uni-jena,dc=de', delete => {member => [$entry->dn]}) or die "Delete group IMAP";
    }

    ### WRITE USER MODIFICATIONS TO LDAP ###
    push @{$ReplaceArray[0]}, mailhost => "mail1-deleted";
    $ldap->modify( $entry->dn, changes => [add => @AddArray, replace => @ReplaceArray, delete => @DeleteArray]) or die "Modify User on LDAP";

}
$ldap->unbind();
---------------------------------------


best regards

Thomas


-- 
Thomas Sesselmann, Dipl.-Inf.
Friedrich-Schiller-Universität Jena
Rechenzentrum
Am Johannisfriedhof 2
D-07743 Jena
Tel.: 03641/9-40530
Fax.: 03641/9-40630

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4829 bytes
Desc: S/MIME Kryptografische Unterschrift
URL: <http://lists.alioth.debian.org/pipermail/pkg-openldap-devel/attachments/20131002/8fe0f727/attachment-0001.bin>