Re: mmr pair stops replicating: "consumer state is newer than provider"

[btb <btb@bitrate.net>]

On 6/27/17 4:55 PM, Quanah Gibson-Mount wrote:
> --On Tuesday, June 27, 2017 5:35 PM -0400 btb <btb@bitrate.net> wrote:
>
> > On 6/27/17 10:27 AM, Quanah Gibson-Mount wrote:
> > > --On Tuesday, June 27, 2017 10:37 AM -0400 btb <btb@bitrate.net> wrote:
> > > > i'm using 2.4.44 on freebsd, built from ports.  i can provide any 
> > > > config
> > > > details etc - i just didn't want to inundate the post with guesses on
> > > > detail that might not be relevant.
> > >
> > > What is your accesslog purge setting?  Do you have long periods of time
> > > with no write activity?
> >
> > there are some extended periods of time with no write activity, yes.
>
> That's one form of a known issue then with using accesslog (ITS#8100). 
> I've made a suggestion on how it could be fixed, and Howard agreed that 
> would be the correct solution.  Just need the fix. :)
>
> In the meantime, you can set up a cronjob that does a modify once a day 
> on some object that doesn't really do anything, like if you had:
>
> dn: cn=someobject,dc...
> objectClass: ...
> cn: someobject
> description: blah
>
> you could have a job that does:
> dn: cn=someobject,dc...
> changetype: modify
> replace: description
> description: blah
>
> So it in effect does nothing, but it keeps an active change in the 
> accesslog alive.
>
> Basically what happens with the accesslog empty is that it'll end up 
> generating its own new contextCSN that differs for that server than the 
> one stored in the rootDB, and will be /newer/ than it as well, which is 
> why you get that message.  You can also fix the problem simply by doing 
> a modify on each master, and it'll reset the contextCSNs and things will 
> flow (i.e., no need to do a dump/reload, etc).

i see, thanks.  i tested this, and did a modify on each, but didn't see 
replication resume.  emulating the syncrepl connection with a manual 
search against each master, there do seem to be accesslog entries now, 
on both masters:

ldapsearch -ZZxWLLLH 'ldap://dsa1.example.org/' -D 
'uid=dsa2_slapd-repl-content,ou=dsa2.example.org,ou=services,ou=accounts,cd=example,dc=org' 
-b 'cn=accesslog' '(&(objectClass=auditWriteObject)(reqResult=0))' reqDN 
reqType reqMod reqNewRDN reqDeleteOldRDN reqNewSuperior entryCSN
Enter LDAP Password:
dn: reqStart=20170628033755.000000Z,cn=accesslog
reqType: add
reqDN: cn=project_management_users,ou=general,ou=groups,cd=example,dc=org
reqMod: member:+ uid=jdoe,ou=People,cd=example,dc=org
reqMod: member:+ uid=mkline,ou=People,cd=example,dc=org
reqMod: member:+ uid=astavins,ou=People,cd=example,dc=org
reqMod: cn:+ project_management_users
reqMod: objectClass:+ groupOfNames
reqMod: objectClass:+ top
reqMod: structuralObjectClass:+ groupOfNames
reqMod: entryUUID:+ ea9a8246-effe-1036-966a-9d166400a934
reqMod: creatorsName:+ 
uid=dit_admin,ou=role_accounts,ou=accounts,dc=example,dc=or
 g
reqMod: createTimestamp:+ 20170628033755Z
reqMod: entryCSN:+ 20170628033755.410123Z#000000#001#000000
reqMod: modifiersName:+ 
uid=dit_admin,ou=role_accounts,ou=accounts,dc=example,dc=o
 rg
reqMod: modifyTimestamp:+ 20170628033755Z
entryCSN: 20170628033755.410123Z#000000#001#000000

dn: reqStart=20170628033931.000000Z,cn=accesslog
reqType: modify
reqDN: uid=jdoe,ou=People,cd=example,dc=org
reqMod: givenName:+ john
reqMod: entryCSN:= 20170628033931.892952Z#000000#001#000000
reqMod: modifiersName:= 
uid=dit_admin,ou=role_accounts,ou=accounts,dc=example,dc=o
 rg
reqMod: modifyTimestamp:= 20170628033931Z
entryCSN: 20170628033931.892952Z#000000#001#000000

dn: reqStart=20170628034017.000000Z,cn=accesslog
reqType: modify
reqDN: uid=jdoe,ou=People,cd=example,dc=org
reqMod: sn:= doe
reqMod: entryCSN:= 20170628034017.532972Z#000000#001#000000
reqMod: modifiersName:= 
uid=dit_admin,ou=role_accounts,ou=accounts,dc=example,dc=o
 rg
reqMod: modifyTimestamp:= 20170628034017Z
entryCSN: 20170628034017.532972Z#000000#001#000000

dn: reqStart=20170628034119.000000Z,cn=accesslog
reqType: modify
reqDN: uid=mkline,ou=People,cd=example,dc=org
reqMod: givenName:+ michael
reqMod: entryCSN:= 20170628034119.327974Z#000000#001#000000
reqMod: modifiersName:= 
uid=dit_admin,ou=role_accounts,ou=accounts,dc=example,dc=o
 rg
reqMod: modifyTimestamp:= 20170628034119Z
entryCSN: 20170628034119.327974Z#000000#001#000000

but the same log messages persist.  what can i look at next?  are these 
the relevant contextcsn values?

dsa1 cn=accesslog:
20161019002438.652359Z#000000#000#000000
20170521175113.974560Z#000000#002#000000
20170530214415.204052Z#000000#001#000000

dsa1 dc=example,dc=org:
20170520031415.276678Z#000000#000#000000
20170530214231.171959Z#000000#002#000000
20170530214415.204052Z#000000#001#000000

dsa2 cn=accesslog:
20170520031415.276678Z#000000#000#000000
20170521175113.974560Z#000000#002#000000
20170628034119.327974Z#000000#001#000000

dsa2 dc=example,dc=org:
20170520031415.276678Z#000000#000#000000
20170619014933.531051Z#000000#002#000000
20170628034119.327974Z#000000#001#000000

if it might be of value, here are the olcsyncrepl attributes from each 
master:

dsa1:
{0}rid=000
provider=ldap://dsa2.example.org/
starttls=critical
tls_cacert=/usr/local/etc/pki/trusted_root_authorities/root_ca-cert.pem
bindmethod=simple
binddn="uid=dsa1_slapd-repl-content,ou=dsa1.example.org,ou=services,ou=accounts,cd=example,dc=org"
credentials="xxxxxxxxxxxxxxxxxxxxxx"
searchbase="dc=example,dc=org"
logbase="cn=accesslog"
logfilter="(&(objectClass=auditWriteObject)(reqResult=0))"
schemachecking=on
type=refreshAndPersist
retry="15 +"
syncdata=accesslog

dsa2:
{0}rid=000
provider=ldap://dsa1.example.org/
starttls=critical
tls_cacert=/usr/local/etc/pki/trusted_root_authorities/root_ca-cert.pem
bindmethod=simple
binddn="uid=dsa2_slapd-repl-content,ou=dsa2.example.org,ou=services,ou=accounts,cd=example,dc=org"
credentials="xxxxxxxxxxxxxxxxxxxxxx"
searchbase="dc=example,dc=org"
logbase="cn=accesslog"
logfilter="(&(objectClass=auditWriteObject)(reqResult=0))"
schemachecking=on
type=refreshAndPersist
retry="15 +"
syncdata=accesslog

thanks
-ben