Re: syncrepl simplification

[Howard Chu <hyc@symas.com>]

OK, these changes are complete and working in HEAD:
   syncrepl consumer only uses be_rootdn
   only one consumer per database allowed (as it turns out, config.c 
already returned an error if you tried to specify more, so apparently no 
one was ever using multiple contexts anyway.)
   consumer context is stored in suffix contextCSN attribute, same as 
the provider - no more syncrepl subentries.
   slapadd options -p, -r, and -i removed.
      -w option causes the suffix contextCSN to be updated with the 
greater of:
          old value, if any
          greatest entryCSN seen in the current Add session
    slapcat options -k and -m removed.
    provider uses a single sessionlog for the entire context, it is 
used automatically for all sync operations. No more sid in sync cookie.

The slapadd -w option is really just a convenience now. Since the 
syncprov overlay will scan the database on startup, it will work fine if 
you load a provider without it. (But using -w in this case will of 
course allow syncprov to startup quicker.) Also since the -w option 
compares the old max with the current max, you can run incremental 
slapadd's and still get the right result. (It's important not to just 
override the old contextCSN with the greatest entryCSN in the current 
database; if the last operation on a database was a delete then the 
contextCSN will be greater than all the entryCSNs in the database and 
just using the greatest entryCSN would therefore cause a rollback.)

So now any provider database can be used as a consumer and vice versa, 
without any tweaking in between.

There was a bug where deletes from a sessionlog didn't propagate in a 
cascaded setup, but that all works now. We may need to extend the 
syncrepl test scripts to test with and without a sessionlog, although it 
looks like test019 does some of this already.

Howard Chu wrote:

> Kurt D. Zeilenga wrote:
>
> > At 09:19 PM 1/12/2005, Howard Chu wrote:
> >  
> >
> > > Kurt D. Zeilenga wrote:
> > >
> > >   
> > >
> > > > The rationale, I believe, for multiple consumer contexts
> > > > within a database context was that administrators could state
> > > > different consumer policies for different areas of the DIT.
> > > >
> > > >     
> > >
> > > I'll have to think about that some more.
> > >   
> >
> >
> > I've been, too.  In particular, I think the above rationale more
> > applies to provider contexts, where one might want to have different
> > server handling of different requests.  For instance, one might
> > want to have one area covered by a session log and one area not.
> >  
> That actually makes sense, although the current code doesn't allow for 
> this. It would require the sessionlog config directive to take a 
> search specifier (baseDN of a subtree at least) to differentiate logs. 
> Again this is problematic when you're dealing with ModDN requests; if 
> an entry moves across a sessionlog boundary then one or both logs 
> becomes useless.
>
> At the moment all updates in a context are recorded in all the active 
> sessionlogs, so the only difference between the multiple logs is their 
> size. This also seems unnecessary, it would be best to have a single 
> log of the largest practical size. I have a feeling this sessionlog 
> stuff was not thoroughly thought out; the fact that there is no config 
> keyword for the sid on the consumer implies to me that it was never 
> even tested before.
>
> Given all of this, I think it would be best to have only one 
> sessionlog per provider, and we can forget about the sid parameter 
> entirely.
>
> > On the consumer side, we just just require additional
> > database contexts where the admin desires different consumer
> > policies.
> >
> >  
> >
> > > To me, it still sounds like a recipe for disaster. If the separate 
> > > consumer contexts are talking to different providers, then you have 
> > > the cascade problem I mentioned before. If the contexts are all 
> > > pulling from the same provider, then I guess it works. I still don't 
> > > see any rationale for the updatedn on the consumer; surely any data 
> > > that the provider returns must satisfy the consumer criteria and 
> > > therefore must be reflected into the consumer DB.
> > >   
> >
> >
> > I concur with regards to the updatedn.  Also, we need to revisit
> > consumer ACLs.  Seems odd to me that syncrepl consumer needs
> > permission to write into its own database.
> >  
> Yes. OK, I'm going to delete the updatedn stuff then.


--
 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support