Re: Scaling LDAP

[Hallvard B Furuseth <h.b.furuseth@usit.uio.no>]

How much write activity do you expect?  How big entries?
Will your entire DB fit in RAM, so disk read speed will mostly
be an issue during startup (if back-bdb is properly tuned)?


Current stats for our slapd machines, which are currently vastly
overpowered for their task:

- 3G database size from 0.5G LDIF data with 0.9 million entries.

- Average 60 connections and 280 requests per second the last
  two days, but just 6 per minute were write requests.

- 8G RAM - the entire databases can be cached in RAM.

- 4 2GHz cores in 2 processors, Linux x86_64.

- The 'unchecked' limit is set so that only the rootdn can trawl
  an entire DB with one search.  Others get adminSizeExceeded
  if the number of candidate result entries after consulting
  the database indexes exceed the unchecked limit.

- RAID disks not specially tuned for BDB.  Not recommended for
  speed nor database durability.  But it's fast enough anyway
  and we can regenerate the DBs from LDIF, so we're happy to leave
  the disks to the site admins and the site's standard disk setup.

- The traffic gives 100K syslog output/second from loglevel 'stats'.

Some years ago, the disks on our aging machines got too slow for all 
the
syslog output.  Many requests took more than one second.  So we had to
turn off slapd logging.  The issue went away when we put '-' in front 
of
the logfile name in /etc/syslog.conf and upgraded to more modern
hardware.

It would surely also have helped to play around more with the disks -
like logging to separate disks if we did not already.  Don't remember
how much we did with that.  We do like leaving such concerns to
others at the site.

--
Hallvard