[Date Prev][Date Next] [Chronological] [Thread] [Top]

RE: big directories (more than 1M entries)

> -----Original Message-----
> From: owner-openldap-software@OpenLDAP.org
> [mailto:owner-openldap-software@OpenLDAP.org]On Behalf Of Daniel Tiefnig

> Hi there,
>
> first i've a question on removing transaction logs..
> FAQ-O-Matic and Sleepycat Documentation say, someone should run
> db_archive to determine which Logs can be removed. But now i've added
> about 250k entries into my OpenLDAP 2.1.8 with BDB backend, and i
> noticed that i have to restart (or at least stop) slapd (same for
> slapadd, as i notice) before db_archive shows me unused logs.
> Is that intended behaviour?

You need to set a checkpoint value in slapd.conf. See slapd-bdb(5).

> Next i noticed that slapdindex grows quite large when
> indexing the whole
> backend, i.e. it grows larger than 256MB, which is the
> maximum size for
> processes on AIX 32 and 64 Bit systems if the binary has been
> compiled
> without any -bmaxdata switch. Maybe this should be change to
> default to
> somethin bigger in the Makefiles, or at least should be mentioned in
> the FAQs? Further information about that can be obtained from [0].

This shouldn't happen. There have been other reports of memory leaks on AIX
but we have never been able to track any of them down. If you have access to
any malloc debugging tools it might be helpful for us if you could trace
where all the memory has gone.
>
> Well, after that worked out (I patched slapindex using dd) I started
> slapd and issued a simple search on an indexed attribute, but
> i had to
> notice that slapd isn't using the index files. (Not even
> `fuser` shows
> a process attached to the index files.) After ~2 minutes
> slapd returns
> the queried entry.
> I generated the DB with slapadd and had the index stuff
> commented out in
> slapd.conf. Could that cause a problem? Running slapindex should have
> created all needed indexes, shouldn't it?

As long as you uncommented them again before running slapindex, yes.

> Another topic with respect to problems mentioned above..
> I thought of splitting that backend into several smaller parts and
> having them tied back together by slapd.
> I vaguely remember someone on the list mentioned a possibility to do
> this without referrals? With a glue backend or something like
> that?

See the "subordinate" keyword in slapd.conf(5) and in the archives of this
mailing list.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support