[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
RE: Memory and CPU usage
I see a lot of memory problems with back-bdb when a particular configuration
is set to allow too many threads. back-bdb uses a lot of memory in each
thread, much more than back-ldbm. In general though, you shouldn't be
configuring a large number of threads; a single CPU can only answer so many
requests before it gets bogged down in I/O waits anyway. All the threads in
the world won't improve your concurrency once your I/O bandwidth (disk and
network) is saturated. And using too many threads will push you to the
saturation point sooner, because the increased memory usage will trigger
excessive paging/swapping (thrashing).
Some may be good, but more is not always better. (And I still believe that a
good async I/O API obviates the need for threads. So whether the situation is
"good" to begin with is still debatable...) Threads don't come for free, and
today they're not even "cheap". As a sysadmin you need to understand the
limits of the machines you're working with. A particular CPU has only so much
on-chip Level 1 cache, some amount of Level 2 cache, page tables, VM
lookaside tables, etc... When you spawn more execution paths than your CPU's
hardware has cache/table space for, you destroy the CPU performance. When you
spawn so many threads that all of the free RAM in your address space is
consumed, slapd aborts because it cannot allocate any more dynamic memory for
its normal operation. This is a strong message to you, the administrator,
that your hardware is inadequate to support your configuration.
-- Howard Chu
Chief Architect, Symas Corp. Director, Highland Sun
http://www.symas.com http://highlandsun.com/hyc
Symas: Premier OpenSource Development and Support
> -----Original Message-----
> From: owner-openldap-software@OpenLDAP.org
> [mailto:owner-openldap-software@OpenLDAP.org]On Behalf Of Tony Earnshaw
> søn, 2003-02-16 kl. 10:17 skrev Timothy H Folks:
>
> > This problem went away when I switched from the bdb backend
> to the ldbm
> > backend (still using DB 4.1.25 though).
>
> Amazing. In such cases, including those that apply to my own
> installations, I always ask myself: "Is the rest of the world
> experiencing the same thing?" BDB is generally reckoned as being a
> "better" backend than ldbm, and that's my experience too - though on a
> far smaller scale than yours.
>
> In this case, I don't reckon the rest of the world is. Why?
>
> Are you still getting the "(?=undefined)" search reports in the log
> output?
>
> Best,
>
> Tony
>
> --
>
> Tony Earnshaw
>
> When you rob a person of his illusions,
> you are robbing him of his happiness
>
>
> e-post: tonni@billy.demon.nl
> www: http://www.billy.demon.nl
>
>
>