[Date Prev][Date Next] [Chronological] [Thread] [Top]

slapadd *very* slow: tuning advice?



Dear Folks,

I'm upgrading a cluster of OpenLDAP servers from 2.3.43-25.el5 to
2.4.32 with BDB 4.8.30 on CentOS 5, x86_64, on HP BL460cG6 blades with
two 4-core CPUs, and 12GB RAM.  These are slaves, and have eleven
trees on them.  I have dumped and restored six of the LDAP databases
in reasonable time, but the seventh is taking a long time.  Here are
the sizes of the slapcatted LDIF files:

320M
252M
80M
225K
246K
833K
2.6G
1.2G
24M
947M
522K

It's the 2.6G LDIF file that's taking the time to slapadd:
-############          63.34% eta 02h09m elapsed       03h43m30s spd   2.7 k/s 	

As you can see, it has slowed to a crawl.

# cat DB_CONFIG set_flags DB_LOG_AUTOREMOVE
set_cachesize 0 286162472 0

# egrep 'tool-threads|cachesize' /etc/openldap/slapd.conf
tool-threads 8
cachesize    100000
idlcachesize 100000
cachesize    100000
idlcachesize 100000
cachesize    100000
idlcachesize 100000
cachesize    100000
idlcachesize 100000
cachesize    100000
idlcachesize 100000
cachesize    100000
idlcachesize 100000
cachesize    100000
idlcachesize 100000
cachesize    100000
idlcachesize 100000
cachesize    100000
idlcachesize 100000
cachesize    100000
idlcachesize 100000
cachesize    100000
idlcachesize 100000

On another member of the cluster, not upgraded from 2.3.43 yet, we
have, for this big tree:
# sudo -u ldap slapd_db_stat -d id2entry.bdb
Tue Aug 21 15:15:38 2012	Local time
53162	Btree magic number
9	Btree version number
Little-endian	Byte order
	Flags
2	Minimum keys per-page
16384	Underlying database page size
3	Number of levels in the tree
4611665	Number of unique keys in the tree
4611665	Number of data items in the tree
418	Number of tree internal pages
36096	Number of bytes free in tree internal pages (99% ff)
308745	Number of tree leaf pages
871M	Number of bytes free in tree leaf pages (82% ff)
0	Number of tree duplicate pages
0	Number of bytes free in tree duplicate pages (0% ff)
0	Number of tree overflow pages
0	Number of bytes free in tree overflow pages (0% ff)
0	Number of empty pages
0	Number of pages on the free list
# sudo -u ldap slapd_db_stat -d dn2id.bdb
Tue Aug 21 15:34:30 2012	Local time
53162	Btree magic number
9	Btree version number
Little-endian	Byte order
duplicates, sorted duplicates	Flags
2	Minimum keys per-page
4096	Underlying database page size
4	Number of levels in the tree
9227811	Number of unique keys in the tree
9263210	Number of data items in the tree
2185	Number of tree internal pages
2781648	Number of bytes free in tree internal pages (68% ff)
219185	Number of tree leaf pages
287M	Number of bytes free in tree leaf pages (68% ff)
37	Number of tree duplicate pages
40214	Number of bytes free in tree duplicate pages (73% ff)
0	Number of tree overflow pages
0	Number of bytes free in tree overflow pages (0% ff)
0	Number of empty pages
0	Number of pages on the free list

QUESTIONS:
==========

Any suggestions on how to optimise this a little more towards slapadd?

Would mdb possibly be faster than bdb?
--
Nick Urbanik http://nicku.org 808-71011 nick.urbanik@optusnet.com.au
GPG: 7FFA CDC7 5A77 0558 DC7A 790A 16DF EC5B BB9D 2C24  ID: BB9D2C24
I disclaim, therefore I am.