RE: server side sorting & paging?

["Howard Chu" <hyc@highlandsun.com>]

> -----Original Message-----
> From: owner-openldap-software@OpenLDAP.org
> [mailto:owner-openldap-software@OpenLDAP.org]On Behalf Of Jon Roberts

> Kurt D. Zeilenga wrote:
> > At 03:56 AM 4/22/2004, denis.havlik@t-mobile.at wrote:
> >>Thus we need:
> >>1) Server Side Sorting
> >
> > Feel free to submit a patch.
>
> "Patch" just doesn't seem like the right word here ;)

It's certainly not a gut/rewrite/overhaul task. (That's already been done.)

> >>2) Paging.
> >
> > 2.2 supports paged results.
>
> In my mind, paged results are of limited value

In my mind, you should have stopped right there. Clients should grab results
and use them or display them as they arrive, in realtime. I suppose that's
not great for HTTP clients, but HTML isn't fully interactive in the first
place.

> Server or client side, entry sorting appears a daunting task
> because of the various LDAP syntax types.

Server-side work like this doesn't scale well as the number of clients
increases. When you have 1 server and N clients making N queries with
sorting, it stands to reason that the N clients are more likely to have
sufficient memory and CPU to handle their own individual search than that the
1 server can do all of them simultaneously.

Generally a database server has far more data on disk than can fit in its
memory, so it can't even hold a single full copy of the data in memory. To
handle multiple sort requests would require the server to hold multiple
copies of the data, or keep multiple lists of references to the entries on
disk. If you use references, you can save on memory use, but the entries
themselves may get modified during the collect/sort processing, and so your
resulting list may still be out of order. You can lock the entries to ensure
the consistency of your sorted result list, but then you degrade concurrency.
You can use copies of the entries to avoid the locks and preserve
concurrency, but then you need enough memory to hold the result sets for N
queries simultaneously in memory. Whatever choice you make here, some aspect
of server performance will suffer badly.

Or you can choose to let the client deal with it. I like this approach,
because while server-side sorting results in many clients seeing poor
response, client-side sorting results in all clients seeing equally fast
server response.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support