RE: OpenLDAP as an enterprise level LDAP provider

["Fortin, John {PBG}" <John.Fortin@pepsi.com>]

Hi Mathew,

Not quite sure what Outlook is doing to my posts!!  I'm posting (according
to outlook anyway, in plain text.

1)  I did apply the patches, but without success.  I'm going to try the head
source along with the bdb 4.3 code to see if this resolves the issue.

2b)  Hopefully not many times!!  However, if the hierarchy changes
significantly, it may be needed on occasion.  
2c)  That is what I normally do.  However, slapadd never completed with
indexing on.  We had to turn off indexing, load the database, then run
slapindex.  I was very surprised!!  In a response to Quanah, I detailed the
system I am using, but basically it is a Dell 2650 with Dual 2+ GHz Zeon
processors and 2 Gig mem.

Thanks!
--John

John Fortin
PBG Middleware and Web Services
(914) 767-7844


>-----Original Message-----
>From: Matthew Hardin [mailto:mhardin@symas.com] 
>Sent: Wednesday, November 24, 2004 4:18 PM
>To: Fortin, John {PBG}; 'OpenLDAP Mail List'
>Subject: RE: OpenLDAP as an enterprise level LDAP provider
>
>
>The message was pretty badly mangled, but mostly complete. 
>Maybe post in
>plain text next time?
>
>Our customers routinely use our distributions of OpenLDAP, 
>called Connexitor
>Directory Services, in enterprise directory applications with 
>a great deal
>of success.
>
>I'll address your issues one at a time:
>
>1. Log archiving and deleting - Because of a nasty deadlocking 
>problem, the
>author of back-bdb/back-hdb (our own Howard Chu) needed to 
>make a change to
>those backends that created a long-lived transaction. This 
>transaction is
>what prevents log files from being rotated/deleted. He 
>produced a patch to
>Berkeley DB 4.2 that would address this problem, but it's not 
>an official
>Sleepycat patch and so you probably do not have it. It is available
>publicly, though, and our distributions are built with this 
>it. In version
>4.3 Sleepycat addressed the functionality introduced by the 
>patch, but to
>make use of it you will need updates that are currently only in
>OPENLDAP-RELEASE. Your alternatives are to live with the 
>problem, patch and
>rebuild Berkeley DB and OpenLDAP, or use Berkeley DB 4.3.
>
>2. The ability to back up data - 
>a) You are correct: Don't use the Berkeley native tools. Also, 
>to insulate
>you from changes to the internal database, don't back up the 
>database files
>themselves; use ldapsearch or slapcat to dump the db in LDIF 
>format. Because
>of the fine-grained locking and the transactional nature of the bdb
>backends, slapcat can be used while slapd is running.
>
>b) The load times you are complaining about can be reduced 
>through tuning
>and setup changes. How often do you anticipate reloading the 
>DB? I hope not
>very often...
>
>c) You are best served by letting slapadd create the indexes 
>when it creates
>the new db. It's much faster.
>
>Hope this helps...
>
>Matthew Hardin
>Symas Corporation
>Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
>http://www.symas.com
>
>-----Original Message-----
>From: owner-openldap-software@OpenLDAP.org
>[mailto:owner-openldap-software@OpenLDAP.org] On Behalf Of Fortin, John
>{PBG}
>Sent: Wednesday, November 24, 2004 9:21 AM
>To: OpenLDAP Mail List
>Subject: OpenLDAP as an enterprise level LDAP provider
>
>First of this, this message is =ntended to open a discussion 
>about using
>OpenLDAP in the =nterprise. I do not want to start a flame war 
>concerning
>the =ros and cons of various LDAP implementations. Currently 
>we are using
>OpenLDAP as our =nitial implementation for authentication and 
>authorization
>with =eblogic and other J2EE providers for our enterprise 
>application. =ur
>initial rollout was successful, although we did not have a 
>large =opulation
>of users in the directory (<1000) Performance was =ine, and we 
>had no issue
>with loading data etc as the ldif files were =mall. However, 
>as we are now
>looking to roll =his out to a much larger population (600K+) 
>we are starting
>to run =nto some issues, one of which I sent a note about recently. The
>=ssues we are currently seeing, and could potentially be a 
>show stopper =or
>us are as follows: 1) Log archiving and transactions - With 
>the =urrent bdb
>and version of OpenLDAP (2.2.18), I cannot =rchive/delete files without
>stopping slapd. This manifested =tself as we were testing bulk 
>loading of
>data and consistently ran out =f log space. I have tested with 
>the various
>patched suggested to =o avail. I have not tested with the 
>newest version of
>bdb (4.3) =s I have no indication that this fixes the issue. 
>2) The ability
>to backup data - Using the bdb utilities =db_load and db_dump) 
>do not work.
>It seems that this is based on =penLDAP using custom hashes in 
>the creation
>of the configured indexes. =This is based on some discussion I 
>found in the
>maillist =rchives). The two workarounds that I am aware of, 
>creating ldif
>=iles with slapcat, and backing up the bdb files themselves so 
>not seem =o
>be adequate for the following reasons: = slapadd - with 600K 
>users and no
>=ndexes it takes about 2 hrs to load. The creation of indexes 
>=fterwards
>with slapindex takes an additional 6-12 hours. To me, =his 
>seems like too
>long of a time for recovery. = *.bdb file backup - we've had 
>limited =uccess
>with this. There also seems to be an issue, even after =oing a 
>db_checkpoint
>and a db_recover of a dependency on logs =iles. As we are 
>looking to do a
>'cold' backup of our master ldap =irectory, we do not want to 
>be dependent
>on logs files. I have searched the archives quite a =it 
>looking for similar
>issues with limited success. I know the =asics of how OpenLDAP 
>works and
>tuning of the system, but I am by no =eans a guro in the 
>internals. At this
>point, I am looking for =ome direction as to how to proceed. 
>System: OS: RH
>ES 3.0 OpenLDAP 2.2.18 BDB 4.2.52 (with current =atches) 
>Thanks!! --John
>John Fortin PBG Middleware and Web =ervices (914) 767-7844
>