Re: Objections to draft-ietf-ldapext-psearch-01.txt

[Tim Howes <howes@netscape.com>]

Ed Reed, Scott Isaacson, Dave Huntbach (Ed Reed) wrote:
> 
> Comments on  ftp://ftp.ietf.org/internet-drafts/draft-ietf-ldapext-psearch-01.txt
> 
> 1. Point to point registration and event distribution is not scalable; only channel-based
> approaches scale.  This approach of LDAP persistent queries represents a dramatic
> over-simplification of the implementation constraints and requirements for large scale
> (internet scale) event notification systems.  This persistent query approach is valid only for
> small-scale, synchronization systems.
> 
> 2. Both theory and empirical evidence shows that network bandwidth is the first resource to be
> exhausted by a large scale event notification system.  Without attribute-level filtering, the
> network will be clogged with useless notifications being sent just to be discarded at the LDAP
> client end.
> 
> 3. Each active Persistent Search registration requires an open TCP connection. Even a single 
> LDAP client with multiple active Persistent Searches will require as many open TCP connections
> as it has Persistent Searches. There are LDAP servers that have been tested with hundreds of
> connections; some may even support thousands of connections, but somewhere a practical number
> of simultaneous open connections "service ceiling" will be reached beyond which response times
> will be unacceptable. Dozens or hundreds of LDAP Persistent Search open TCP connections will
> adversely impact LDAP Server performance everywhere.

I mostly agree with these points, but they are not
really relevant to this draft. Psearch is not meant
to be a large-scale, general-purpose event notification
system. As you imply, such a system is a topic for
research. That problem is one that I doubt will be
solved to everybody's satisfaction any time soon.

Psearch is very focused on solving a very limited
problem space, best illustrated by the scenarios
described in section 6, "Intended Use" of the document.
The scaling concerns, which bear on the intended
use, are discussed in section 7, "Implementation
Considerations". If you think either of these sections
need additional text to make the applicability of
psearch more clear, we'd definitely be interested
in incorporating your suggestions.

Your second point seems to imply that the psearch
mechanism does not support filtering of events. In
fact, that's exactly what it does. If you are not
interested in some changes, construct a filter that
does not match those entries. This is one advantage
of psearch over the simpler, "send anything that has
changed" approach.

> 4. Rather than first defining an LDAP specific event notification mechanism, LDAP notification
> requirements should be fed into Event Notification BOF to be held at the Chicago IETF meeting.  
> The suggested agenda for the BOF does not propose one solution for all problems, however it
> does express a desire to classify each of the problems to see if there is any room for some
> common effort that can be leveraged by many applications, services and/or protocols.  Each
> applications group can then focus on defining the specific events and their semantics rather
> than reinventing notificaiton subsystems.  For example, in the LDAP case, the LDAP extensions
> group can focus on defining what is an "add" or a "delete" or a "modify" event is and what
> additional info is needed in the event reports for these events rather than defining new
> notification infrastructure.    At the WISEN conference (Workshop on Internet Scale Event
> Notification, July 13-14 1998,
> http://www.ics.uci.edu/IRUS/wisen/wisen.html) there seemed to be consensus that a common,
> scalable, infrastructure notification subsystem could be used for many applications, inlcuding
> directory events and notifications

This sounds to me like a good way to get nothing
done for a long time. When/if such a "common,
scalable, infrastructure notification subsystem"
exists, perhaps it will be able to replace psearch,
but I doubt it. The fact that psearch is specific
to LDAP makes it attractive to many applications,
so they can avoid getting, installing, maintaining
another big pile of general goo, just to solve a
limited problem that psearch already solves quite
well.

I think there are many applications that would benefit
from a more general event notification service, and
I think the work you mention above is a great start
at taking a look at this problem. I don't want my
comments above to be interpreted as disparaging either
the specific work you mention or the problem space
in general. But it would be a mistake to prevent
psearch, which solves an actual problem today, from
going forward because we think there might be a
more general solution to the problem developed at
some later date. Progress today is better than the
promise of "better" progress tomorrow.

> Based on the above reasons, we don't believe that the LDAP community should adopt or promote
> the mechanism described for common use, much less forward it for consideration on any standards
> track.  Instead, we believe work should go forward, as described in (4) above, with a more
> generally useful mechanism.

I would ask you to rethink your position, based on
my comments and explanations above. Is there some
stronger applicability statement, either part of
or separate from the existing intended use and
implementation considerations sections in the
existing document, that would help to allay your
concerns? Nobody is arguing that psearch should be
used for more than the narrow purposes for which
it was designed. Apparently, this is not as clear
as it should be in the current document. How can
we make it clearer?                   -- Tim