front, back, middle...

["Howard Chu" <hyc@highlandsun.com>]

Trying to decide what to do for configuring a middle layer... Currently
backglue is the only "middle" piece and it exists as its own entity. It would
take fairly simple mods to back-ldap to make it handle chaining for any other
backend. Mainly config changes to allow one instance to handle multiple
destination URIs, something which back-meta already does.

My first notion for extending back-ldap was to set all of the target info in
one statement, like replica does:

database bdb
suffix "ou=folks,o=my company"
...

database ldap
overlay "ou=folks,o=my company"
chain uri=<uri> [tls=yes|critical] bindmethod=simple|sasl
   bindid=<simple DN|SASL ID> credentials=<password>
chain uri=<uri2> ...

This would set back-ldap to insert itself over the backend that handles
suffix "ou=folks,o=my company". It would pass through all requests to the
underlying database, with a callback to intercept the responses. For any
responses that include referrals (or search references), the reference would
be parsed, and if it specified a URI that was configured as a chaining
destination, then back-ldap would proxy the request out to that destination.
If there was no matching URI configuration, the referral would be returned to
the client as normal. The notion of chaining implies server-to-server
communication; in my mind this necessarily means that the bind credentials
specified in a "chain" clause are privileged/authorized to submit operations
with proxyAuth controls. In a true X.500 system the credentials would just be
the X.509 cert of the DSA itself; the same could apply here using TLS and
SASL/EXTERNAL. However it's done, the point is that chaining requires
cooperation between all the DSAs; they all have to have trust agreements in
place beforehand for it to work.

One obvious place to use this would be on a slave server; instead of
returning update referrals to a client it could simply chain the update
request on behalf of the client.

But in this case, back-ldap is no longer a "backend", it's an "intermediate"
of some sort. And moving the rewrite engine from back-meta off on its own
becomes another intermediate. Insert? (My thesaurus isn't handy at the
moment; I'd like a good word to place here.)

Really backglue is an intermediate as well, and "glue" was the most obvious
choice at the time. How about "overlay" ?

overlay chain  (aka back-ldap)
target <suffix>
target <suffix>
 ... various configuration info ...
(one problem with this, it doesn't handle the global default referral. but I
guess you can just specify a default backend instead, using back-ldap if you
want it.)

overlay rewrite  (aka back-meta)
target <suffix>
...

overlay glue   (aka backglue)
target <suffix>
target <suffix>
target <suffix>

I think there should be a generic mechanism to configure all of these, and
obviously I have a few more middle-layer pieces in mind. We need some method
for registering native operation plugins as well, e.g.:

moduleload myplugin  <-- myplugin's init registers all of its handlers

overlay plugin
target <suffix>
plugin add myplugin.add	 <-- use the named handler for the specified
operation
plugin delete myplugin.del

Back to the slave/master example - you might set up a set of overlays/plugins
such that: all mods are chained to the master; if they succeed the operation
falls thru to the underlying database so that the slave updates itself
immediately. It's not quite multimaster (because you still must have realtime
access to the master) but it does let your clients immediately see their
changes on the slave that accepted the change. If your slaves are distributed
geographically this might be an advantage; if the slaves are all sitting
behind a load-balancer then there'd be no benefit.

Not sure how cumbersome this all may get, but it's a starting point.
Comments, suggestions?

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support