RE: (ITS#7698) Multiple Paged search requests on one connection fail

[john.unsworth@cp.net]

>> There are no other directory servers in existence that can scale to the
level that OpenLDAP does and perform well at that scale.

How do you justify/prove that statement?

-----Original Message-----
From: Howard Chu [mailto:hyc@symas.com] 
Sent: 17 September 2013 20:52
To: john.unsworth@cp.net; openldap-its@openldap.org
Subject: Re: (ITS#7698) Multiple Paged search requests on one connection
fail

john.unsworth@cp.net wrote:
> Products other than browsers use LDAP servers. Our product is a meta 
> directory that watches for changes on multiple data sources (LDAP, 
> Databases, SAP, ...) and reconciles any changes across the complete 
> set of sources according to data filtering and transformation rules. 
> However sometimes it is necessary to read the whole data set - for 
> example when a new data server is added, or when changes may have been 
> missed for any reason, or when an 'audit' type operation is required 
> to ensure that all data sources are consistent. LDAP servers we 
> connect to typically contain thousands if not millions of entries.

OpenLDAP is regularly used in production by large telcos with billions of
entries. There are no other directory servers in existence that can scale to
the level that OpenLDAP does and perform well at that scale.

> Reading the lot without paging -
> although we can do it - is very inefficient both in terms of LDAP 
> resources and application resources.

That's utter nonsense. The amount of data is the same no matter how many
pieces you slice it into. It is *more* inefficient to slice it into multiple
requests.

> The directory is read using multiple parallel paged searches across 
> different branches on a single connection. Every directory we have 
> used except OpenLDAP can manage this effectively.

> In the case of OpenLDAP there is no LDAP changelog and so the only way 
> we have of discovering changes is to read the whole directory and 
> compare with what was read last time.

So you're saying you're connecting to OpenLDAP servers from version 2.1 or
older, from before syncrepl or the accesslog or changelog overlays existed.

> -----Original Message-----
> From: Michael Ströder [mailto:michael@stroeder.com]
> Sent: 17 September 2013 20:20
> To: john.unsworth@cp.net; openldap-its@openldap.org
> Subject: Re: (ITS#7698) Multiple Paged search requests on one 
> connection fail
>
> john.unsworth@cp.net wrote:
>> I would also be interested to understand why " paged results is 
>> inherently flawed".
>
> Although being the author of an interactive LDAP UI client I still 
> wonder why people want paged results (or tree browsing).
>
> If you have so many results you should narrow the search criteria.
> Browsing/paging is pretty inefficient working style.
>
> Ciao, Michael.
>
>
>
>
>


-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/