Re: Objections to draft-ietf-ldapext-psearch-01.txt

[Tim Howes <howes@netscape.com>]

I think we are still not doing a good job of
explaining how psearch is intended to be used.
You (and Paul) seem to be thinking of it very
differently than we are, and re-reading our
document, I'm not surprised. Let me try to explain
a little better, both our view on things and
how we came to it.

We've been working with a lot of developers who
are using LDAP for various purposes, ranging
from an authentication database, to a configuration
store, to a means of providing location independence,
to many others. These developers are both inside
and outside of Netscape. Some are developing
end-user client software (like email clients or
browsers), but many are developing server software.
Psearch is focused on the needs of the server
community. To understand their need, it's necessary
to understand how a typical server, in our
experience, makes use of directory, and how a
typical server application + directory configuration,
in our experience, is deployed.

First, a typical server may make very heavy use of
the directory. A web server might need to access
the directory for each page served, to make an
authentication or access control check. A mail
server might need to perform many accesses to
deliver a single piece of mail (e.g., to a group).
Typically, this drives real deployments of these
applications to replicate a copy of the directory
for the near-exclusive use of the demanding application,
even when that application uses caching to reduce
the load on the directory server.

In this kind of situation, clearly psearch is a
much more efficient use of resources than a polling
approach. That's true both from the directory server's
point of view (less work to do, and only when it's
needed), and the application's point of view (more
timely notification).

You and Paul are focusing on the scenario where
there are hundreds, maybe thousands of clients with
mostly idle connections open to a central directory
server. White pages or email address books are good
examples of this situation. And I agree: Psearch is
not the best idea in this kind of environment. A more
appropriate approach would be client-side caching
based on TTLs, with polling if necessary. That's
what this draft is about:

 
http://info.internet.isi.edu:80/in-drafts/files/draft-ietf-asid-ldap-cache-01.txt

So to sum up:

- We expect psearch to be used in the real-world
situations I outlined above, where large numbers
of connections are not a problem.

- We expect other mechanisms like the ones described
in the above draft to be used in other real-world
situations in which large numbers of connections are
a problem.

- The intended use section, as it currently stands,
does not make this clear, and should be changed.
         -- Tim

Ed Reed wrote:
> 
> -------------------
> Ed Reed, Technologist
> Novell, Inc.
> +1 801 861 3320
> 
> >>> Tim Howes <howes@netscape.com> 08/14/1998 14:35:45 >>>
> 
> Psearch is very focused on solving a very limited
> problem space, best illustrated by the scenarios
> described in section 6, "Intended Use" of the document.
> The scaling concerns, which bear on the intended
> use, are discussed in section 7, "Implementation
> Considerations". If you think either of these sections
> need additional text to make the applicability of
> psearch more clear, we'd definitely be interested
> in incorporating your suggestions.
> 
> [eer] But the examples in section 6 are precisely the sort
> of uses which will drown LDAP servers supporting
> psearch, as presently designed, in dormant, open,
> stateful TCP connections.  There are likely to be
> LARGE numbers of clients caching entries in which
> they're interested.  Consider 411 or Bigfoot, if you
> like.  Or consider white pages clients in companies
> with more than a few dozen employees.
> 
> These are the scenarios you describe in section 6.1.
> 
> Or consider the number of local address books being
> maintained for email purposes by users of both
> desktop and portable machines - the scenario
> described in section 6.2.  Mail and directory servers
> will generally wind up with an open LDAP connection
> for open white pages applications which bind once
> and hold the connection until its needed, another
> for any persistent searches expressed to keep the
> local client cache up to date, etc.
> 
> Considerable industry experience with client-side
> caching of message stores and associated address
> book information suggests that for those kinds of
> human interface applications, a poll cycle on the
> order of minutes is adequate, and that with even
> reasonable design, random variations in the period
> of poll attempts by the clients will result in a smoothing
> of traffic and cpu loads on servers.  If the client
> already is holding an authenticated connection open
> to deal with queries, the poll can easily be inserted
> into the query queue without doubling the connections
> (and authentication operations) on the server, and
> without imposing serious state-ful management
> issues on the server tracking who wants what sent
> where.
> 
> Indeed, for the casual client, an opportunistic cache of
> the sort used by DNS (I'll use information 'til its ttl expires, or
> until the information seems bad, and then I'll refresh
> it with information I know at that time to be good) seems
> more than adequate.
> 
> For the portable client, a periodic poll for changes (the
> very search that would be performed by the psearch)
> leaves coordination of who needs what where it
> should be - with the clients who care, and not with
> the server.
> 
> Designing the psearch sort of server behavior into
> a system, and expecting that it will server more than
> a handfull of clients at a time from any given server
> seems ill advised.  I'd welcome information about
> deployments using the mechanism, or something
> similar to it, and the real benefits that clients see
> in comparison to the other approaches I've outlined
> here.
> 
> I've no objection to publication of the document as
> an experimental mechanism, or even as an
> informational RFC, but to put this on the standards
> track seems out of line.
> 
> Standards track features ought not, I think, be
> ones which are so narrowly focused in their intent.
> Being on the standards track, this feature will
> become something all of us are measured against,
> whether we do it or not, whether it interfers with
> other application service deployment metrics,
> whether it delivers any real value to the LDAP
> community (with its very narrow focus) or not.
> 
> I'd much rather see a UDP based registration
> callback mechanism established, as someone
> else on the list has suggested, if we're going
> to proceed with a standardization effort in this
> area.
> 
> Please reconsider putting this on a standards
> track without more deployment experience, and
> knock this back to experimental until we see if
> it has value enough, or if there are alternative
> approaches enough, to decide what to do with
> it.
> 
> Best regards,
> Ed Reed