[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
RE: slapd memory and performance problems
Quanah,
Thanks for the information. I'm not familiar with GSSAPI accept to
understand that it's a generic security services API. I'm not inclined
to hack the Exim code to incorporate another API. I have heard that
performance improvements can be gained by using unix sockets, but I'm
not familiar with the use of those either and am looking for more info
on that topic. But your performance seems enviable to me. I've not
heard any complaints about the thread implementation in this version of
RedHat, but I'll check into that.
You say you created an eq index for the maildrop attribute that I assume
belongs to your main entry which is accessed by the uid filter. I was
wondering about creating an eq index on my mailquota attribute, but it
seemed unnecessary since the attribute is part of an entry already
accessed by uid. Can you comment on your reasoning for adding an index
for that attribute if you don't explicitly search for an entry by
filtering on that attribute? I may be incorrect in my thinking here as
I am not really a db person by vocation or by nature.
Thanks,
Mike
-----Original Message-----
From: Quanah Gibson-Mount [mailto:quanah@stanford.edu]
Sent: Tuesday, April 01, 2003 6:04 PM
To: Mike Denka; OpenLDAP-software@OpenLDAP.org
Subject: Re: slapd memory and performance problems
--On Tuesday, April 01, 2003 5:25 PM -0800 Mike Denka
<mdenk@whidbey.net>
wrote:
> After experimenting with threads (max number and idletimeout),
upgrading
> to v.2.1.16 and applying Howard's new cvs version of tpool.c, I am
still
> experiencing some performance issues with slapd. The most immediate
> problem I see is that the response time for a query under peak loads
is
> very slow. The most obvious oddity is that the slapd threads are
> growing in size over time.
>
> I am currently running four load balanced mail servers which query the
> ldap server once for each incoming message to extract the user's mail
> quota. The user's entry is indexed by uid and a user defined
attribute
> in the user's entry stores the mail quota. At peak times I am getting
a
> combined total of between 1 and 4 queries per second for these quotas
> from the mail servers. A search on this server during peak load times
> takes several seconds (10 to 20) to complete. If I try to add
> additional loads on this same server (e.g., radius authentication
making
> maybe one query every two to 4 seconds), queries start timing out and
> applications (like radius) making those queries start to fail.
>
> Upon starting slapd, each thread has a size of about 4800KB. This
> thread size seems to double after 8 hours or so and grows to about
15M.
> Also, the total memory available on the box seems to degrade over
time,
> not just from the increased thread size, but from what appears to be a
> memory leak somewhere since just restarting slapd does not increase
the
> available amount of system memory. Only a reboot recovers system RAM.
>
> I'm running RedHat 8.0 on a platform with an Intel 1G processor and
> 512MB of RAM. Processor idle time never dips below 90%, no swap space
> is ever used but memory, as I said, diminishes over time. However,
> performance stays constant and doesn't change as memory degrades
(until
> memory drops to below 5M or so, at which time the server must be
> rebooted).
>
> Each query for a mail quota does a simple bind via the ldap:// url
using
> the entire dn as the base and '(objectclass=*)' as the filter. This
> base and filter is composed internally by Exim (our MTA).
>
> I am considering using Unix sockets as was suggested in an earlier
> thread but I don't understand that approach enough to engage it yet.
>
> Could anyone comment on the performance of this box? Is 1 to 4
> queries/second the best I can expect on a system of this kind? Any
> ideas what might be causing the memory in the slapd threads to grow
over
> time? Slapd is the only user application running on this machine. I
> can't be sure that slapd is causing the memory leak, but I don't see
> this problem on any of our other similar machines. Are there any
known
> issues involving memory leaks in this latest version of slapd? Also,
> could someone comment on the efficacy of using unix sockets and point
me
> to some docs somewhere explaining them?
Mike,
I can't comment on the redhat portion of this, as we are running under
Solaris 8, but I can offer the following:
We began a production rollout of OpenLDAP today, using 2.1.16 with the
tpool.c patch. So far, only 1 mail router has been cut over to one
machine. It is doing about 10,000 queries/hour. So far, I have seen no
degradation of performance, and the memory footprint is remaining
constant.
A big difference between our setups, is that we have sendmail using
GSSAPI
to bind, as we hacked our sendmail to use GSSAPI authentication. In
load
testing of these systems, I consistently saw 66 queries/second as the
peak
for the particular hardware we are using, and that was leaving the test
running overnight, with over 1 million queries made in about 6 hours
against the machine (18 querying clients). The hardware is:
SunFire v120 with 650MHz CPU & 4GB of memory.
I will keep you posted if we encounter any problems with threads. Maybe
Redhat's thread implementation is buggy. I have had performance
problems
using Solaris 9, due to a change in Sun's thread implementation.
For indexing, we index the filter (essentially uid) and the attribute we
are searching for (maildrop) with eq as well. Our searches continue to
take less than a second to complete.
Software wise, we are running:
OpenLDAP 2.1.16 + tpool.c patch
Cyrus-SASL 2.1.12
Heimdal-0.5.2
BDB 4.1.25
OpenSSL 0.9.6i
--Quanah
--
Quanah Gibson-Mount
Senior Systems Administrator
ITSS/TSS/Computing Systems
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html