Re: Thread-local malloc discussion summary

[Ganesan R <rganesan-ldap@myrealbox.com>]

>>>>> "Howard" == Howard Chu <hyc@highlandsun.com> writes:

> So I take it you're in favor of exploring in this direction...

I guess that's another way of saying "talk is cheap, where's the data" :-).
While I don't have time to test OpenLDAP right now with these libraries, I
am not just handing out suggestions without testing. I've personally
experimented with many commercial and free versions of parallel allocators
with pretty good results.

The work was not on OpenLDAP but in a commercial LDAP version derived from
Umich LDAP. We found the biggest malloc overhead in the ber
encoding/decoding routines. I think that should still apply to OpenLDAP as
well. We eventually shipped with a commercial allocator but the following
free ones did give significant improvements.

I have already mentioned hoard which is being actively maintained and like I
said very easy to test because it doesn't require a recompile of the
server. The other alternative is ptmalloc (See http://www.malloc.de/en/). 
Linux glibc was based on ptmalloc for a long time. This gave significantly
better performance than the stock malloc on Solaris 2.6 and 7. There is a
new version ptmalloc2 which I've not tested.

All of these parallel allocators have a problem with sharply increased
memory usage. Hoard is designed to prevent that from happening, but still
it's not unusual to see memory usage go up tremendously. Reworking malloc
usage within OpenLDAP may be a better approach in the long run.

Ganesan

-- 
Ganesan R