controls and dynamic support

[Howard Chu <hyc@symas.com>]

I was thinking about how both pagedresults and server-side sorting could 
be accomplished using overlays, and the idea is simple enough. But one 
of the more awkward aspects of adding new controls is plugging them into 
the Operation structure so they can be easily referenced by other code. 
I recall how OpenSSL handles various OID-based entities - it has its own 
internal registry and assigns small integers (NIDs) for each one. I 
think we need a similar mechanism for slapd.

Specifically, the register_supported_control function should return the 
index of the control in slapd's internal table. The Operation structure 
should have an array
   void **o_controls
that is preallocated to a size equal to the number of currently 
registered controls. Each element of the array will be set by the 
control's parser to point to a control-specific structure to carry 
whatever state that control needs. (Only controls that were actually 
received will have non-NULL entries in the array.)

Certain controls that we decide to leave inside the core can have 
well-known variables carrying their assigned control IDs (CIDs):
   slap_control.sc_id_manageDSAIT
   slap_control.sc_id_noop

Others should be migrated out to private files specific to their 
implementation. (I of course want to migrate all the syncrepl state info 
out as well.)

So a pagedResults overlay would register its control and save the 
returned index. Its parser function would allocate a private structure 
and assign its address to op->o_controls[pagedresults_cid] on an 
incoming operation, and all of the overlay code that needs to reference 
the control parameters can find it there.

We can also export a find_ctrl function that returns the CID for a given 
OID, etc.

As for the actual controls...

For server-side sorting all you need is an overlay that intercepts 
search responses and builds a sorted list before actually returning 
results to a client. The biggest issue here is configuring how much 
memory it's allowed to consume per search.

For paged results you could be really gross and actually suspend the 
current thread at the end of a page. (Of course, this means tying up 
threads for some amount of time. Not so friendly.) That is, the 
overlay's search response callback just waits on a condition variable 
when it gets to the page limit. The overlay's search handler checks for 
any currently suspended operations when it receives a continuing paged 
results request, and signals the condition if it finds the match. There 
can be an idletimeout runtask that abandons operations that have been 
suspended for too long. It's a rather coarse-grained approach to saving 
search state, but it would work for all backends without needing to 
redesign them all.

For that matter, the noop control could be entirely processed as an 
overlay. It can call the underlying backend to retrieve entry info, 
perform ACL checks, and then return a result without performing any 
write operations on the actual backend.

For the syncrepl provider there's bits scattered across the Operation 
and BackendDB structures that I plan to migrate into private structures. 
Currently I'm testing with back-ldbm; when I get through all of the 
existing syncrepl test scripts I will start deleting the old definitions 
from the various places.

--
 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support