[Date Prev][Date Next] [Chronological] [Thread] [Top]

Fwd: [ldap] Implementation Suggestions

Just a small bit of background, I'm bringing this conversation over from ldap@umich.edu.

We have been running our campus LDAP service for a little while now and
are starting to see some potential causes for concern. So I had a
couple of implementation questions for others who may or may not have
run into these issues. Let me start off by saying we currently have a
"readonly LDAP service". All of our data is pulled from other sources
so we are effectively just presenting it all in an easy to get to
manner. We are running OpenLDAP with a one master and 3 slaves setup.
(master effectively just pulls information from the various data
sources, puts it together, and performs updates that are sync'd with the
slaves) We have 'directory' type data and 'account' type data together
in the same pool. All of these are running under Solaris.

That said, we've run into a periodic problem where the slaves are
unresponsive due to a very large amount of updates occuring on a
particular day, typically due to directory data changes, and the
response time on the slaves slows down quite a bit. As far as I can
tell, this is due primarily to the relentless disk activity going on
during updates. This isn't a concern for things like the directory,
where if it takes a little longer, no big whoop. However, for account
type information, this can cause a lot of complaints. (yesterday, as an
example, queries that normally returned 'almost instantly' would return
in 8 seconds)

So I've been trying to think through ways to get around this slowness.
Here's my "unresearched" ideas so far:

1. Separate directory from account data . . . perhaps using refers of
some sort to "make it look" like they're all one server/service.
Directory stuff is by far the most intensely searched, updated, and
involves "unusual" queries instead of a simple "give me this one entry,
thanks".

2. Is Solaris causing too much of a bottleneck I/O wise? It seems to be
notorious for having slower I/O than Linux, so I'm wondering if that's
part of my problem. This is Solaris 8 btw.

3. Maybe I have Berkeley DB configured like crap? Thing is I can't seem
to understand the various config options for BDB (the ones you'd put in
DB_CONFIG) well enough to reallly fine tune it.

4. Is there a way to throttle changes? Like tell the master "don't send
ALL of your changes immediately, give the slaves some time ... maybe
send 1K at a time or something like that".

Anyone else run into this before?  What did you do about it?  We're
currently running the 2.2 series.  I've also considered that some of
this might be improved in 2.3 but I don't know about that.

Is BDB considered the best backend for performance? Like I said before,
this is primarily a read-only service with daily updates pushed from the
master. If there's a better backend I should be using given these
parameters, I'm all ears! =) A coworker suggested a MySQL backend but
my gut feeling is that isn't the ideal for performance. Gut could
always be wrong. =)

Thanks for any insight you all may provide!!!

My first suggestion is that you take your questions to the openldap- software@openldap.org list, since the ldap@umich list is for general LDAP questions, and your issues are specific to OpenLDAP.

My second suggestion would be to provide:

Your slapd.conf on the replicas
Your DB_CONFIG file
The size of the *.bdb files in your database directory
Your /etc/vfstab file

I'll attach all of these (and their components) that are actual files. The *.bdb files are:
-rw------- 1 root other 86319104 Dec 2 23:53 cn.bdb
-rw------- 1 root other 663552 Dec 2 23:50 departmentNumber.bdb
-rw------- 1 root other 24137728 Dec 2 23:51 description.bdb
-rw------- 1 root other 29724672 Dec 2 23:51 displayName.bdb
-rw------- 1 root other 54288384 Dec 2 23:54 dn2id.bdb
-rw------- 1 root other 729088 Dec 2 23:50 employeeNumber.bdb
-rw------- 1 root other 2076672 Dec 2 23:50 employeeType.bdb
-rw------- 1 root other 102400 Dec 2 23:50 facsimileTelephoneNumber.bdb
-rw------- 1 root other 675840 Dec 2 23:50 gidNumber.bdb
-rw------- 1 root other 11902976 Dec 2 23:54 givenName.bdb
-rw------- 1 root other 1003520 Nov 30 14:20 homePhone.bdb
-rw------- 1 root other 3551232 Nov 30 14:20 homePostalAddress.bdb
-rw------- 1 root other 285081600 Dec 2 23:55 id2entry.bdb
-rw------- 1 root other 1150976 Dec 2 23:50 initials.bdb
-rw------- 1 root other 319488 Nov 15 16:52 l.bdb
-rw------- 1 root other 24862720 Dec 2 23:53 mail.bdb
-rw------- 1 root other 43495424 Dec 2 23:51 memberNisNetgroup.bdb
-rw------- 1 root other 8192 Dec 2 23:51 memberUid.bdb
-rw------- 1 root other 24576 Dec 2 23:50 mobile.bdb
-rw------- 1 root other 36298752 Dec 2 23:51 ncsuAFSPath.bdb
-rw------- 1 root other 675840 Dec 2 23:51 ncsuAddable.bdb
-rw------- 1 root other 9412608 Dec 2 23:50 ncsuAffiliation.bdb
-rw------- 1 root other 30494720 Dec 2 23:50 ncsuAltDisplayName.bdb
-rw------- 1 root other 217088 Nov 15 16:52 ncsuBldgAbbrev.bdb
-rw------- 1 root other 45056 Nov 15 16:52 ncsuBldgNum.bdb
-rw------- 1 root other 1966080 Dec 2 23:50 ncsuCampusID.bdb
-rw------- 1 root other 1613824 Dec 2 23:50 ncsuClassCode.bdb
-rw------- 1 root other 3616768 Dec 2 23:54 ncsuCurriculumCode.bdb
-rw------- 1 root other 8192 Dec 3 00:06 ncsuIMAddress.bdb
-rw------- 1 root other 9375744 Dec 2 23:54 ncsuMiddleName.bdb
-rw------- 1 root other 21278720 Dec 2 23:51 ncsuMountPath.bdb
-rw------- 1 root other 40960 Dec 2 23:54 ncsuNickname.bdb
-rw------- 1 root other 12828672 Dec 2 23:52 ncsuPreferredDepartment.bdb
-rw------- 1 root other 2228224 Dec 2 23:54 ncsuPreferredGivenName.bdb
-rw------- 1 root other 720896 Dec 2 23:54 ncsuPreferredMiddleName.bdb
-rw------- 1 root other 2879488 Dec 2 23:52 ncsuPreferredName.bdb
-rw------- 1 root other 2433024 Dec 2 23:54 ncsuPreferredSurName.bdb
-rw------- 1 root other 8192 Dec 2 23:54 ncsuPreviousSurName.bdb
-rw------- 1 root other 8192 Dec 7 07:31 ncsuPrimaryEMail.bdb
-rw------- 1 root other 8192 Dec 3 00:06 ncsuPrimaryRole.bdb
-rw------- 1 root other 294912 Dec 2 23:51 ncsuPrivate.bdb
-rw------- 1 root other 892928 Dec 2 23:50 ncsuSpouse.bdb
-rw------- 1 root other 86016 Dec 2 23:50 ncsuWebSite.bdb
-rw------- 1 root other 675840 Dec 2 23:51 nisMapName.bdb
-rw------- 1 root other 4624384 Dec 2 23:54 objectClass.bdb
-rw------- 1 root other 31870976 Dec 2 23:54 ou.bdb
-rw------- 1 root other 8192 Nov 30 14:20 pager.bdb
-rw------- 1 root other 45920256 Dec 2 23:52 postalAddress.bdb
-rw------- 1 root other 229376 Nov 15 16:52 postalCode.bdb
-rw------- 1 root other 8192 Nov 30 14:20 printer-name.bdb
-rw------- 1 root other 45764608 Dec 2 23:52 registeredAddress.bdb
-rw------- 1 root other 13094912 Dec 2 23:54 sn.bdb
-rw------- 1 root other 397312 Nov 15 16:52 st.bdb
-rw------- 1 root other 729088 Nov 30 14:20 street.bdb
-rw------- 1 root other 12222464 Dec 2 23:52 telephoneNumber.bdb
-rw------- 1 root other 18006016 Dec 2 23:50 title.bdb
-rw------- 1 root other 3067904 Dec 2 23:54 uid.bdb
-rw------- 1 root other 2154496 Dec 2 23:54 uidNumber.bdb

Attachment: acls.conf
Description: Binary data

Attachment: DB_CONFIG
Description: Binary data

Attachment: indexes.conf
Description: Binary data

Attachment: slapd.conf
Description: Binary data

Attachment: vfstab
Description: Binary data


I've been running OpenLDAP on Solaris for nearly 2 years. It certainly isn't as fast as Linux, but it can be made plenty fast. I/O is an issue on Solaris if you don't tune things correctly.

That's definitely good to hear! =) I will certainly and humbly admit that performance tuning of an entire system is not my strong suit. Thanks much for any help you can provide!!

Daniel