Re: commit: ldap/servers/slapd syncrepl.c

[Howard Chu <hyc@symas.com>]

hyc@OpenLDAP.org wrote:
> Update of /repo/OpenLDAP/pkg/ldap/servers/slapd
>
> Modified Files:
> 	syncrepl.c  1.208 -> 1.209
>
> Log Message:
> Add syncdata keyword, to select different sync data formats. Defaults
> to "default", can be set to "accesslog" or "changelog". changelog is
> not fully implemented yet.
>   
This patch and a corresponding patch to syncprov.c are an initial stab 
at hacking up syncrepl to propagate deltas instead of complete entries. 
This is not an ideal implementation, but it works without altering the 
existing syncrepl protocol specification.

The basic idea is this - on the provider, set up an accesslog DB for 
logging changes, in addition to the main database. Set up the syncprov 
overlay on the logging DB, not the main DB. Thus syncrepl will be 
propagating full entries as usual, but the content of those entries will 
actually be deltas, not actual entries... A sample config looks like this:

database bdb
suffix cn=accesslog
directory db.1.b
rootdn cn=accesslog
rootpw secret
...
overlay syncprov
syncprov-nopresent TRUE

database bdb
suffix dc=example,dc=com
directory db.1.a
...
overlay accesslog
logdb cn=accesslog
logops writes
logsuccess TRUE   # by default, accesslog logs failed requests too. we 
just want successful writes.


On the consumer, the configuration looks like:

database bdb
suffix dc=example,dc=com
syncrepl    rid=1
       provider=ldap://localhost:9011/
       binddn="cn=accesslog"
       bindmethod=simple
       credentials=secret
       searchbase="cn=accesslog"
       scope=sub
       type=refreshOnly
       interval=00:00:00:10
       syncdata=accesslog


This separation of log info from syncprovider is rather troublesome; it 
won't fallback properly to a full refresh if the consumer is too far out 
of date wrt the log database.

A better implementation would be to leave the syncprovider overlay on 
top of the main database, and add a config keyword telling it where it 
can find a log database to use. Then both syncprov and accesslog 
overlays would sit on the main database. And ideally, there would be a 
new flag in the syncrepl control requesting that changes be propagated 
in commit order, as deltas instead of full entries.

--
 -- Howard Chu
 Chief Architect, Symas Corp.  http://www.symas.com
 Director, Highland Sun        http://highlandsun.com/hyc
 OpenLDAP Core Team            http://www.openldap.org/project/