[Pkg-openldap-devel] RE: LDAP/BDB log purging (fwd)

Mon Mar 5 23:30:16 UTC 2007

This comes off a discussion of db_recover destroying a user's environment 
on Sarge, where they built their own OpenLDAP 2.3.32, but were using the 
Sarge version of BDB (4.2.52-18).  There are two messages below of the 
discussion between the Debian BDB folks and Lesley.

Basically, it looks like even the current Debian BDB 4.2.52 builds do not 
include patches #4 & #5 from

<http://www.oracle.com/technology/products/berkeley-db/db/update/4.2.52/patch.4.2.52.html>

according to this email thread.

This could well be the cause of the database corruption issue that was 
reported here.

The two patches notes say:

Patch #4: Fix a bug where recovery of a page split after a 
non-transactional update to the next page would fail to update the back 
pointer. This bug could result in log sequence errors and missing data when 
when the database is walked with a cursor, after data was initially loaded 
into a database non-transactionally (for example, by using the db_load 
utility).

Patch #5: Fix a bug where cache buffer retrieval could race with a 
checkpoint call, potentially causing database environment recovery to fail. 
[#14657]

I discussed the impact of these two patches with Howard Chu, as I felt #4 
wasn't an issue, and he agreed.  However, #5 can be a source of problems 
for OpenLDAP, and should be in the Debian BDB 4.2.52 package.  It would 
probably be good to escalate this for the etch release.

--Quanah

------------ Forwarded Message ------------
Date: Tuesday, March 06, 2007 10:14 AM +1300
From: Lesley Walker <lesley.walker at opus.co.nz>
To: 'Quanah Gibson-Mount' <quanah at stanford.edu>, 
openldap-software at openldap.org
Subject: RE: LDAP/BDB log purging

> >>> So, are you running BDB 4.2 with or without the patches that
> >>> sleepycat says you must use to use BDB 4.2?
> >>
> >> I would have to investigate that. We're running the Debian packaged
> >> version 4.2.52-18. I'm guessing possibly not.

I asked the Debian BDB maintainers, and the answer is copied below.

> The version of BDB in Debian is currently 4.2.52-24.  BDB
> 4.2.52-18 is part of Sarge.  Sarge ships with OpenLDAP 2.2.23,
> which is a very old release of
> OpenLDAP, and there were some bugs fixed with checkpointing and open
> transactions *after* that release.  So that may be the issue you are
> hitting.  Are you *sure* you are running OpenLDAP 2.3.32 on
> those boxes?

Yes, I built it myself, and included the sync logging patch that Aaron
Richton kindly pointed me to back in January.

We build most of our critical apps (OpenLDAP, Samba, Apache and a few
others) but BDB isn't on that list.

Reply from Debian maintainers re the BDB patches:

Clint Adams wrote:
> Sent: Wednesday, 28 February 2007 5:11 p.m.
> To: Lesley Walker
> Cc: pkg-db-devel at lists.alioth.debian.org
> Subject: Re: [Pkg-db-devel] Db4.2 patch query
>
> > Can you confirm/deny that all 5 of the patches listed at
> > the following URL
> > have been incorporated into the Debian build?
>
> I can confirm that patches #1 and #2 have, and that #4 and #5
> have NOT.

The URL I gave was
http://www.oracle.com/technology/products/berkeley-db/db/update/4.2.52/patch
.4.2.52.html
(beware link breakage)

-- 
Lesley Walker
Linux Systems Administrator
Opus International Consultants Ltd
Email lesley.walker at opus.co.nz
Tel +64 4 471 7002, Fax +64 4 473 3017
http://www.opus.co.nz
Level 9  Majestic Centre, 100 Willis Street, PO Box 12 343
Wellington, New Zealand

---------- End Forwarded Message ----------

------------ Forwarded Message ------------
Date: Tuesday, March 06, 2007 11:19 AM +1300
From: Lesley Walker <lesley.walker at opus.co.nz>
To: 'Quanah Gibson-Mount' <quanah at stanford.edu>, 
openldap-software at openldap.org
Subject: RE: LDAP/BDB log purging

Quanah wrote regarding DBD patches on Debian:
> Interesting... Did they say if they were included in later releases?

I wasn't 100% sure which release he was referring to since he didn't say,
and I probably didn't phrase my original question as clearly as I could
have. But I asked another question about future plans and got a reply that
pretty much answers that one:

Clint Adams wrote:
> Sent: Thursday, 1 March 2007 3:16 p.m.
> To: Lesley Walker
> Cc: pkg-db-devel at lists.alioth.debian.org
> Subject: Re: [Pkg-db-devel] Db4.2 patch query
>
> > Thanks for the quick answer!
> >
> > Are you planning to incorporate the later ones at some stage?
>
> As far as I am aware, there are no firm plans either way.  Chances of
> them getting into sarge are infinitesimal.  For etch, there is a
> possibility, at the discretion of the release managers, assuming
> release-critical bugs are filed.  For lenny, we will either drop db4.2
> entirely or apply the patches.
>
> Please file bugs regarding the deficiencies that would be corrected
> by the application of the missing patches, if you wish to facilitate
> this process.
>
> If you are unfamiliar with the bug reporting process, see
> http://www.debian.org/Bugs/Reporting
>
> If you have a use case which exposes the data loss corrected by
> the patch, list it and set the severity according to
> http://www.debian.org/Bugs/Developer#severities
>
> Hope this helps.

I replied that if I could pin down any specific problems I would file bug a
report, but it probably makes more sense to move forward to 4.4.

-- 
Lesley Walker
Linux Systems Administrator
Opus International Consultants Ltd
Email lesley.walker at opus.co.nz
Tel +64 4 471 7002, Fax +64 4 473 3017
http://www.opus.co.nz
Level 9  Majestic Centre, 100 Willis Street, PO Box 12 343
Wellington, New Zealand

---------- End Forwarded Message ----------

--
Quanah Gibson-Mount
Principal Software Developer
ITS/Shared Application Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html