efficiency of queries returning many entries

["Mark Whitehouse" <markwhitehouse@home.com>]

Hi,

I have the following configuration:

   RH Linux 6.2
   openldap 2.0.7
   berkley db 3.2.9

I have a DB with approx 100K entries.  Most of my queries are fast.
However, whenever I submit a query which contains a large result set (i.e.
~10,000 entries) the query takes approx 80 secs.

The query in question is simple and all the attributes in the query are
indexed, so I assume the reason the query takes so long is the volume of
data being processed??

My question is....  How do I make queries which will return large numbers of
entries efficient?  How do search engines (Yahoo / Google) do this, I have
noticed that broad queries to search engine sites (which return many
thousands of queries) are very fast.  Do they use some sort of paging
mechanism where the results of the query are retrieved one part at a time?
Does any similar mechanism exist in OpenLDAP?

Any help would be appreciated.

Thanks,
Mark