Re: (ITS#4261) [enhancement] Factor replication code out of slapd

[hyc@symas.com]

ando@sys-net.it wrote:
> This patch, for HEAD as of December 15th, 2005, factors replication code out of
> slapd.  It's actually composed of two parts, which are tightly coupled and thus
> it's submitted as just one.
>
> It provides two overlays:
>
> "slurprov" takes care of logging modifications to the replog, and of those
> config statements that are actually used by slurpd: replica-{args,pid}file,
> replica, replicationinterval.  It should be instantiated inside any master
> database.
>
> "consumer" is a generic consumer interface.  It handles generic consumer issues,
> like rejecting non-replication modifications (unless set up as so-called
> "multimaster") and, for syncrepl, it accounts for the configuration.
>
> Both overlays support back-config.
>
> Current issues:
>   

There's a lot of misc cleanup in the diff that appears unrelated to the 
current feature, perhaps you should commit those bits in advance. Also, 
don't include the diff for configure since that script will just get 
regenerated from configure.in.

> - some existing overlays that are replog-aware are not handled yet.  As such,
> slapo-refint(5) and slapo-ppolicy(5) cannot be compiled along with
> slapo-slurprov(5);
>   

I've looked at refint and ppolicy; the reason they invoke replog_cb 
directly is due to the order of processing between that callback and the 
overlay callbacks. With slurprov moved into an overlay, the references 
to slap_replog_cb can be removed from these overlays. Then it will just 
be a matter of configuring the overlays in the proper order to insure 
that refint and ppolicy's own updates are propagated as desired. (In 
this case, slurprov must be below refint/ppolicy in the overlay stack.) 
That takes care of one issue; the other is that these overlays may need 
to know whether the current database is a shadow or not, and alter their 
behavior accordingly.

> - the overlays need to be explicitly instantiated; the rest of the configuration
> is fully compatible with the current.  It would be nice to have them
> automatically instantiated when the appropriate replication statements occur in
> slapd.conf(5), I don't think it's going to be a big deal;
> - slapo-consumer(5) must be global; thus, a single slapo-consumer(5) takes care
> of all consumer database in the DSA.  As a consequence, the related
> configuration is held in subordinate entries of the global overlay rather than
> inside (or subordinate to) the related database.  This makes configuration via
> back-config little intuitive
> - slapo-consumer(5) doesn't yet support run-time configuration via back-config;
> only EMIT is available.
>
> The patch is provided for initial discussion, development and testing.  I plan
> to commit it when it's a bit more mature, i.e. when the above issues are closed.
>  Note though that (at least on my laptop ;) it passes all tests, and
> significantly all replication-related ones.
-- 
  -- Howard Chu
  Chief Architect, Symas Corp.  http://www.symas.com
  Director, Highland Sun        http://highlandsun.com/hyc
  OpenLDAP Core Team            http://www.openldap.org/project/