Re: >1024 connections in slapd / select->poll

["Kurt D. Zeilenga" <Kurt@OpenLDAP.org>]

Personally, I'd like to see more experimentation with poll(2) before we widely
distribute slapd(8) which relies on it.  To get that experience, poll(2)
support in slapd(8) is needed.  So, I'm likely to incorporate poll(2) support
in some fashion (I haven't yet looked at the details of the patch, but hope
to soon).  I'd likely want to make poll(2) optional for starters.  Then after
we acquire some reasonable level of operational experience, consider whether
poll(2) should be the default.

Two areas concern me about the use of poll(2) are interaction with threads and
performance.  The former is basically fear of the unknown.  We might find that
poll(2) interacts just fine with threads (or at least not worse than select(2)).
Regarding the latter, I'd guess that poll(2) performance would be worse than
select(2) but difference could be in the noise.  But before I'd switch from
select(2) to poll(2), I'd want to see some benching to confirm that poll(2)
isn't going to kill performance.

epoll(2) and /dev/poll support doesn't help me as the kernel I use supports
neither.  (But kqueue support might help me.)  Like Howard, I am somewhat
concerned that we could easily get bogged down here.  As we found with
threading subsystems, its hard enough to make and maintain one well.
However, if there seems to be seems to be adequate community support for
developing and maintaining select(2)/poll(2) alternatives, I'm fine with
including such in slapd(8).

As far as remaining static sizing issue with select(2), I hope to statically
use a high number (like 8196) in 2.3.  We could switch to dynamically sized
fd_sets later (most OS don't care how the bitmask is allocated), but likely
not much point in that.

Kurt


At 04:13 AM 11/15/2004, Volker.Lendecke@SerNet.DE wrote:
>On Sun, Nov 14, 2004 at 09:04:04PM -0800, Howard Chu wrote:
>> Been there... I haven't been motivated to investigate poll() because it 
>> really doesn't offer any scaling benefits vs select(). 
>
>So can I take this as a 'no' for this patch?
>
>> As for /dev/poll and epoll() - they sound nice, but I don't want to get too
>> bogged down in OS-specific special cases. I guess a decent abstraction layer
>> above it would be OK, but I definitely don't want to see a lot of #ifdef
>> HAVE_DEVPOLL/HAVE_EPOLL junk littered all over daemon.c.
>
>The slapd_add/remove and set_write & friends already should roughly map to at
>least epoll() on Linux. I don't know about /dev/poll. It would only be
>necessary to re-write the loops over readers and writers. Maybe with a
>first_reader/next_reader style interface? This might work for select/poll as
>well as epoll.
>
>Volker