Re: ITS#3975: Lightweight Listener

["Kurt D. Zeilenga" <Kurt@OpenLDAP.org>]

First, I note, the patch was committed.  I had intended
to wait a bit, but wasn't as careful as I should have been
in committing some other changes... and once committed,
well, I didn't see much point in reversing the commit.

Second, in the following text, where I say "select",
I mean either select or epoll.

In reviewing the code, my primary concern is that the
listener thread is not as light as possible.  For instance,
some writes appears to be done in the listener.  I'd really
like the listener to simply pass each selected descriptor
to a pooled worker immediately with minimal information
(selected on read, on write, etc.) and then suspend further
selection until the worker says its okay.

But as implemented, I believe the code to solve the
primary problem it was designed to solve: reduce
serialization in the event loop.  In particular,
avoid serialization of connection acceptance and
to allow processing of new operations while a new
connection was being accepted.  Note that the
intent of this code is to reduce the worse case
scenario where one connection denies service to
other connections (intentionally or not).  While
every attempt seems to have been made at reducing
the overhead, that is, to maintain the same level
of latency/throughput in absence of such denials,
the current evidence shows the code's overhead
to result in minimal reduction of throughput.
I suspect with some additional tweaking we can
eliminate most if not all of the remaining overhead
(for instance, removing the writes from the listener
loop should improve throughput).

Enjoy!

Kurt

At 03:53 PM 10/11/2005, Kurt D. Zeilenga wrote:
>Sang Seok Lim (IBM) contributed a patch which modifies
>slapd(8) listener to have a lightweight listener.
>The patch includes a number of changes to intended
>to improve the behavior of the listener.  I am currently
>reviewing the patch.  At present, I find it generally
>suitable for HEAD and hence intend to commit it (behind
>LDAP_DEVEL) in the next few days.
>
>Comments?
>
>Kurt