[Date Prev][Date Next]
[Chronological]
[Thread]
[Top]
RE: FW: profiling
BTW, I'm working with an IBM researcher on doing a detailed
performance analysis of OpenLDAP. It will be a few weeks
before we have any particular recommendations.
Kurt
At 08:16 AM 2001-09-21, Kurt D. Zeilenga wrote:
>At 07:14 AM 2001-09-21, Howard Chu wrote:
>
>>> -----Original Message-----
>>> From: Kurt D. Zeilenga [mailto:Kurt@OpenLDAP.org]
>>
>>> At 05:05 PM 2001-09-20, Howard Chu wrote:
>>> >Has anyone spent any time profiling slapd to see where the CPU time goes?
>>>
>>> It's been awhile since I have done any... but as I recall, primary
>>> issues were in indexing and, in particular, management of IDLs.
>>> Of course, kind of depends on whether all entries/indices are cached
>>> or not and many other factors.
>>
>>Yes, it looks like that is still definitely the case.
>>
>>> >I've been doing a bit of this; I've long believed that slapd is too
>>> >malloc/free happy and needs some more efficient data structure
>>> approaches.
>>>
>>> Heap allocation is certainly an area which takes significant time.
>>> There are a number of areas where some improvements can be made.
>>> In back-bdb, I've done some work to use stack allocated IDLs.
>>> Unfornately, stack limitations in threading environments severely
>>> limits the ability to shift large structures from the heap to the
>>> stack with default thread settings. I hadn't the time pursue this
>>> further.
>>
>>I've been thinking of an approach to minimize heap calls, allocating and
>>deallocating entries in a single block. That is, instead of individual
>>malloc's for each entry structure and all of its individual attributes,
>>calculate the total amount of memory required all at once, do a single
>>malloc, and parse out the block. For Search results, this could be a big
>>win. Record the bounds of the block inside the Entry. You can still
>>replace individual attributes (for Modify purposes) but you don't need
>>to free an attribute unless its address lies outside of its Entry's
>>block bounds. entry_free can do a single free() if none of its attributes
>>have been modified, otherwise it just has to free the block and any
>>changed attributes.
>>
>>Does this make sense?
>
>I think there are a number of other approaches which should be
>considered. One thing I note in back-bdb, is that entries are
>stored on disk BER encoded. This means optimizations to
>-llber will impact conversion to Entry structures. Then I note
>that one could rework BER decoding routines to avoid making
>duplicate copies of values contained within larger values. But
>it is clear that this would likely add complexity to entry_free().
>
>We should be leery of complexity in the name of performance.
>As I learned long ago: "It's easier to make a correct program
>run fast than a fast program run correctly."