Re: SLAP_LIGHTWEIGHT_LISTENER

["Kurt D. Zeilenga" <Kurt@OpenLDAP.org>]

At 10:31 AM 10/14/2005, Howard Chu wrote:
>So, the desired approach is:
>   1: the listener thread listens for reads on wake_sds, listeners, and data connection, and listens for writes on data connections, with a timeout to allow for idle and runqueue processing.

For clarity, listens for writes on data connections with pending
writes.

>   2: if the thread pool gets full, the listener stops listening for reads on listeners and data connections, but continues to listen for reads on wake_sds and writes on data connections.

with timeout as in 1.

>   3: (optional) if a single connection has many writers queued,

When would a connection have more than one writer queued?
If we're managing write events, there should never be more
than one write worker per connection (excepting the inconsequential
case when one writer has finished but not exited and another
worker on the connection writes).

Or do you mean that excess amounts of writes are pending for the
one writer to handle?

In that case, the worker queuing more writes can mark the
connection as write-bound and the listener seeing that
can avoid future reads.

>the listener should stop listening for reads on that connection. Currently we read the connection and put requests on the pending_ops queue. Maybe we should only stop listening for reads when the number of pending_ops hits a threshold, not the number of pending writes.

The problem could be that there are no pending ops but many
writes are pending, so I think that won't work well here.

I can see also avoiding executing a pending operation when
the connection is write-bound... but that seems a different
case to me.


>Dropping the readers from the event set (as in step 2) is very expensive using epoll() in its default mode, since it requires calling epoll_ctl() for each fd individually. (Once to remove it, and once again to reinsert it when we want to re-enable listening.) Using epoll() in Edge Triggered mode only partially mitigates this problem; it won't wake us up for old events but will keep on waking up for new events. The only practical alternative here is to use cascaded epoll sets, one dedicated to the listeners / data readers, and a main one for the wake_sd and the data writers. The listener epoll fd will itself be added or removed from the main epoll fd. When we want to mask off reads, we remove the listener epoll fd from the main fd set. When we want to monitor reads, we add it back in, and then we have to go through a second level loop to find out which readers are active. This is ugly and stupid, but that's life using epoll(). (And no, I never had time to write the equeue implemen
tation, though it would surely come in handy right now.)