[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: need suggestion on indexing big directory



--On Saturday, June 05, 2004 11:05 PM +0200 vadim tarassov <vadim.tarassov@swissonline.ch> wrote:

Hi Quanah,

thanx a lot for replying me. Could you please tell me what is difference
between slapindex and slapadd in terms of disk usage? Do they work so
differently? I just want to understand what is happening ...

slapindex will create indices in an existing database. The only time you generally would use it is if you change the indices in slapd.conf, and then want to update your database. Note, that in *repeated* tests I've done, it was always quicker to "slapcat" the entire database and then "slapadd" it back in, than to run slapindex. There was some work done at one point to fix this problem, I don't recall if it made it into 2.2 or not, IIRC there were some unintended side effects, and it was put off for now.

Generally, if you are building your DB from scratch (which it sounds like you are), you use "slapadd" to create the DB. It will both create the entries, and index them. ldapadd is useful for adding some new entries to a DB. Note that indexing is generally done when you use ldapadd, so there is no reason to do "ldapadd" followed by "slapindex".

You'll see in the notes I've written, you generally get better performance by having two disks, one for the transaction logs, one for the DB. This isn't so necessary now that you can turn the transaction logs off when doing slapadd operations. Also note that using a memory cache is vastly more efficient than a disk cache. Another reason I recommend 2.2 over 2.1.

Some comparisons on a 330,000 entry db:

running slapindex where I changed a single attribute to have "sub" as well as "eq": > 26 hours

running "slapcat" then "slapadd" for the same DB with memory cache: approx. 2 hours

running "slapcat" then "slapadd" for the same DB with disk cache: approx. 6 hours

--Quanah

--
Quanah Gibson-Mount
Principal Software Developer
ITSS/Shared Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html