[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: ldap_search_ext_s: maximum no of entries

Stefan Palme wrote: 
...
Your problem could be solved in many ways: using a band-aid, or
designing your client to get rid of self-imposed constraints.
...


There is a real use case for paged results (at least I had a similar
problem, too): As we had to write a piece of software for a BIG
company with thousands of accounts stored in an Active Directory
server, we had to fetch all accounts from the AD server. Starting
using asynchronous calls, all went fine, until we noticed, that
the SERVER side had a restriction to not give out more than
1000 results per single query.

We could not find a "nice" workaround for this restriction (I dont
know if the AD server CAN not give out more than X results per
query, or whether the admins did not WANT to remove this limit).

Just because Microsoft doesn't know how to implement the real LDAP protocol, and doesn't know how to write a server that can behave as a real LDAP server, doesn't make this a "real use"...

Note
 https://www.openldap.org/its/index.cgi/Historical?id=5092

especially followup #8 - it is *possible* to increase the sizelimit in AD, but it is strongly discouraged because increasing the limit will severely impair AD's performance. And of course, looking here
http://connexitor.com/blog/pivot/entry.php?id=185

AD's performance is already miserable to begin with, so when Microsoft themselves warns you that it will be impaired, one can only imagine how much worse it could possibly get.

By the way, the time to retrieve the DN's of all 1 million entries, using paged results, in OpenLDAP was only 18 seconds. Against AD it took 34 seconds. So even without the overhead of many threads and requests running at once, OpenLDAP is much faster.

OpenLDAP:
time ./ldapsearch -x -H ldap://sihu -D dc=example,dc=com -w "secret" -LLL -b ou=people,dc=example,dc=com -E pr=1000/noprompt 1.1 > dn1

real    0m17.766s
user    0m5.337s
sys     0m7.831s

ADAM:
time ./ldapsearch -x -H ldap://sihu -D cn=bench,cn=users,dc=example,dc=com -w "123foo#" -LLL -b ou=people,dc=example,dc=com -E pr=1000/noprompt 1.1 > dn1

real    0m33.875s
user    0m5.618s
sys     0m3.608s

(In both cases the same client was used, querying the exact same server machine, just with a different OS/directory server.) In fact OpenLDAP can scan its entire 1M entry DB in under 2 seconds; for AD it takes around 25 seconds. (That's running a search that returns no data, so there's no network turnaround overhead like there is in this pagedresult case.)

We solved this by splitting the "big" query in 26 single queries
(all accounts starting with "A", all accounts starting with "B"
and so on) - but this solution is ugly and does not scale very
well.


Naturally, since AD is ugly and does not scale very well.

So a "real" solution for paged results would have been nice
(until now I did not know RFC 2696).

For a BIG company with thousands of accounts, a real solution would use a real LDAP server, not Microsoft garbage.

--
  -- Howard Chu
  Chief Architect, Symas Corp.  http://www.symas.com
  Director, Highland Sun        http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP     http://www.openldap.org/project/