RE: benchmarks &real world

["Howard Chu" <hyc@highlandsun.com>]

> -----Original Message-----
> From: owner-openldap-software@OpenLDAP.org
> [mailto:owner-openldap-software@OpenLDAP.org]On Behalf Of J.Smith

> Which probably means that it will be all that much more easy
> to 'compare' to
> other systems/vendors/products, if you use a somewhat
> 'standardized' test,
> like for example MindCraft's DirectoryMark. This test always
> uses the same
> amount of entry's, the same attributes, attributes length
> etc. etc and then
> reports this as 'Operations / per second'. Which still leaves
> the question
> of CPU/MEM/DISK etc, which is why it also reports a price/performance
> metric, which is expressed as the 'System Under Stress' price
> in dollars,
> divided by the 'operations per second'. So that actually
> makes it relatively
> fair and/or easy to compare different kinds of product/systems.

I've played with DirectoryMark a few times, after seeing it bandied about. In
fact, the data set that it uses is randomly generated, so there is no
canonical fixed data set. Nor is there a standardized job mix or load factor.
In articles I've seen that report DirectoryMark figures, these critical input
parameters are not included. As such, none of the published results I've ever
seen could be compared to any concrete reference, which makes the usage of
DirectoryMark rather meaningless.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support