RE: FW: profiling

["Howard Chu" <hyc@highlandsun.com>]

> -----Original Message-----
> From: Kurt D. Zeilenga [mailto:Kurt@OpenLDAP.org]

> At 05:05 PM 2001-09-20, Howard Chu wrote:
> >Has anyone spent any time profiling slapd to see where the CPU time goes?
>
> It's been awhile since I have done any... but as I recall, primary
> issues were in indexing and, in particular, management of IDLs.
> Of course, kind of depends on whether all entries/indices are cached
> or not and many other factors.

Yes, it looks like that is still definitely the case.

> >I've been doing a bit of this; I've long believed that slapd is too
> >malloc/free happy and needs some more efficient data structure
> approaches.
>
> Heap allocation is certainly an area which takes significant time.
> There are a number of areas where some improvements can be made.
> In back-bdb, I've done some work to use stack allocated IDLs.
> Unfornately, stack limitations in threading environments severely
> limits the ability to shift large structures from the heap to the
> stack with default thread settings.  I hadn't the time pursue this
> further.

I've been thinking of an approach to minimize heap calls, allocating and
deallocating entries in a single block. That is, instead of individual
malloc's for each entry structure and all of its individual attributes,
calculate the total amount of memory required all at once, do a single
malloc, and parse out the block. For Search results, this could be a big
win. Record the bounds of the block inside the Entry. You can still
replace individual attributes (for Modify purposes) but you don't need
to free an attribute unless its address lies outside of its Entry's
block bounds. entry_free can do a single free() if none of its attributes
have been modified, otherwise it just has to free the block and any
changed attributes.

Does this make sense?

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc