benchmarks &real world

[denis.havlik@t-mobile.at]

"Kurt D. Zeilenga" wrote :

>> identity possible problems and to help verify that solutions
>> to those problems are reasonable.  I don't think we should
>> start chasing benchmark results.  Benchmarks results should
>> be taken with a large grain of salt.

>From an engineering viewpoint, I couldn't agree with you more here. However,
>outside of the community there is a whole other world that is effectively
>measuring the 'quality' of products based on the performance measured. For

Benchmarks have to be indeed taken with a grain of salt, but they are nevertheless needed. It's not even nessesary to have the fastest racing car in the street, but for the folks who are evaluating different products it would be very helpful to have an "official" page on openldap.org showing some simple benchmarks. For instance:

- adding a new user in a directory that already contains N entries (N=10.000, 100.000, 1M...) 
- dumping the DB
- filling the DB with initial data (10k, 100k, 1M.. entries)
- Finding an user from a directory with N user entries (some simple real-world search example)
- response times for some simple search depending on number of entries & server load (requests/sec)

Of course, this kind of tests will heavily depend on the DB backend, indices, hardware (CPU, RAM, HDs), but showing the results for a couple of reference systems is enough to get trough the primary evaluation phase. Keep in mind that folks who do this evaluation may have to choose 2-3 candidates out of all the LDAP servers available out there BEFORE even starting the internal tests...

regards
        Denis