Re: >1024 connections in slapd / select->poll

[Howard Chu <hyc@symas.com>]

David Boreham wrote:

> > only present in the children. You can spawn as many of these child 
> > processes as necessary to keep the total number of active descriptors 
> > in each process down to a manageable limit, using shared memory to 
> > coordinate the rest of the program.
>
>
> What's your goal with the multiple processes ? To work around an OS 
> limit on the max number of
> descriptors per process ? Or some other thing ?
>
> (Modern OS versions allow large numbers of descriptors
> per process, BTW, and they have threads which allow
> exploitation of SMP hardware in a single process).

Not to work around the OS limit - to work around the unwieldiness of 
large fdset's. This is just an application of the same 
divide-and-conquer strategy that applies to binary search, trees, and 
other such log(N) algorithms. Just because an OS allows a single process 
to use thousands of file descriptors does not mean it's a good idea to 
do so. At the kernel level it's going to be more efficient to deal with 
a cluster of average sized processes vs a single process with a huge 
context. Eventually something is going to come along that causes the 
entire process to block, and all those threads are going to suspend, 
doing nothing besides occupy address space, whereas a cluster of 
processes will provide more scheduling opportunities for useful work to 
get done; even if one process completely blocks, other processes may 
continue.

--
 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support