Re: slapadd, 1 million entries, some numbers

[pll+ldap@lanminds.com]

>>>>> On Tue, 27 May 2003, "Andreas" == Andreas wrote:

  Andreas> Yesterday I tried do add about 1 million entries via
  Andreas> slapadd. Results are below. The machine is a P4 2.4GHz, 1Gb
  Andreas> RAM, cheap 20G udma IDE disk, P4PE motherboard.

[...snip...]

  Andreas> I noticed via top (and according to the time output above)
  Andreas> that slapd spends a considerable amount of time in the D
  Andreas> state, that is, waiting for a system call to complete if
  Andreas> I'm not mistaken.

Actually, the 'D' means the process is in a disk wait, i.e. the 
process can't do anything further until it can get access to
read from/write to the disk.

With an IDE drive, you're going to have pretty good throughput, and 
will be able to handle decent sized bursts of activity.  However, 
with long periods of high disk I/O, IDE breaks down.  SCSI will fare 
a lot better, since it's a parallel bus, and can prioritize requests, 
and use advanced features like tagged queueing.  IDE doesn't have 
any of that.

  Andreas> I suppose this is due to the heavy logging the BDB backend
  Andreas> uses/makes. I got about 2.7G worth of log files.

Well, the BDB backend will be some of that, but my guess is the 
amount of logging you have going on.  It might be worth testing this 
again and turn off logging completely just to see the performance 
difference.  For a production environment, you might consider having 
all the logging sent to a different syslog host which will 
dramatically reduce the disk I/O on the slapd server.  Connecting to 
the syslog host via a private LAN will help reduce the amount to 
network traffic you generate via syslog calls as well.
-- 

Seeya,
Paul
--
Key fingerprint = 1660 FECC 5D21 D286 F853  E808 BB07 9239 53F1 28EE

	It may look like I'm just sitting here doing nothing,
   but I'm really actively waiting for all my problems to go away.

	 If you're not having fun, you're not doing it right!