Syncrepl issues in multi-versions environment

[Thibault Le Meur <Thibault.LeMeur@supelec.fr>]

Hi openldap Gurus,

I'm sorry for the long post, but I try to give as much details as possible.

We are having serious issues with syncrepl replication and our 3 
openldap servers. These issues first occurred when one of the servers 
was upgraded to openldap 2.4.11 while the two others were (and still 
are) openldap-servers-2.3.39 and openldap-servers-2.3.30.

My main goal in writing this email is to make sure our 3-servers 
architecture is supported and to check if my setup is correct.

Just to let you know, the issues we've found so far include:
* having the openldap 2.3.x servers unable to restart (with a crash 
report) [seems to be due to several contextCSN on the openlda2.4 server 
database]
* sometimes the subtree obtained from the openldap2.4 by syncrepl 
processes in openldap2.3 servers disapears (by reading the log, with 
loglevel 16384,it seems that the NON PRESENT phase of the LdapSync 
protocol decides that entries are no more in the openldap2.4 provider 
[though they are still present]).

Since I have a doubt about our replication structure which uses syncrepl 
and subordinate databases, I wonder if it is really supported by these 
versions of openldap.
I would really appreciate if someone could have a look at this and say 
if the following scenario is supported:

------------------------

AllServers have the following DIT:

dc=mycompany,dc=edu
   |
   |-dc=A,dc=mycompany,dc=edu
   |
   |-dc=B,dc=mycompany,dc=com
   |
   |-dc=B,dc=mycompany,dc=com

Each server replicates 2 subtrees from {dc=A, dc=B, dc=C} and is the 
provider for the remaining one.


Server A has the following setup:
=================================
* a section used to glue the subordinate "dc=B,dc=mycompany,dc=com" 
subtree, with content replicated from the ldap.B.supelec.fr
* section used to glue the subordinate "dc=C,dc=mycompany,dc=com" 
subtree, with content replicated from the ldap.C.supelec.fr
* section having the root suffix "dc=mycompany,dc=com" and managing its 
own subtree dc=A,dc=mycompany,dc=com

The configuration file looks like:

* First section is used to get data from the B server, and set it in the 
dc=B,dc=mycompany,dc=com glued subtree (using subordinate keyword)
database        bdb
suffix          "dc=B,dc=mycompany,dc=com"
rootdn          "cn=mgr,dc=mycompany,dc=com"
subordinate

directory       /openldap/openldap-data-B

syncrepl rid=125
       provider=ldaps://ldap.B.supelec.fr
       type=refreshOnly
       interval=00:00:03:00
       retry="60 10 300 +"
       searchbase="dc=B,dc=mycompany,dc=com"
       filter="(objectClass=*)"
       scope=sub
       schemachecking=off
       bindmethod=simple
       binddn="cn=replicator,dc=mycompany,dc=com"
       credentials="password"

* Second section is used to get data from the C server, and set it in 
the dc=C,dc=mycompany,dc=com glued subtree (using subordinate keyword)
database        bdb
suffix          "dc=C,dc=mycompany,dc=com"
rootdn          "cn=mgr,dc=mycompany,dc=com"
subordinate

directory       /openldap/openldap-data-C

syncrepl rid=120
       provider=ldaps://ldap.C.supelec.fr
       type=refreshOnly
       interval=00:00:03:00
       retry="60 10 300 +"
       searchbase="dc=C,dc=mycompany,dc=com"
       filter="(objectClass=*)"
       scope=sub
       schemachecking=off
       bindmethod=simple
       binddn="cn=replicator,dc=mycompany,dc=com"
       credentials="password"

* Third section is used to define data from the A server, it has a root 
suffix of "dc=mycompany,dc=com" and owns his own subtree 
dc=A,dc=mycompany,dc=com
database        bdb
suffix          "dc=mycompany,dc=com"
rootdn          "cn=mgr,dc=mycompany,dc=com"
rootpw          THEPASS

directory       /openldap/openldap-data-A
overlay syncprov
syncprov-checkpoint 100 10


Server B has the symetric setup:
=================================
* section used to glue the subordinate "dc=C,dc=mycompany,dc=com" 
subtree, with content replicated from the ldap.C.supelec.fr
* section used to glue the subordinate "dc=A,dc=mycompany,dc=com" 
subtree, with content replicated from the ldap.A.supelec.fr
* section having the root suffix "dc=mycompany,dc=com" and managing its 
own subtree dc=B,dc=mycompany,dc=com

Server C has the symetric setup:
=================================
* section used to glue the subordinate "dc=B,dc=mycompany,dc=com" 
subtree, with content replicated from the ldap.B.supelec.fr
* section used to glue the subordinate "dc=A,dc=mycompany,dc=com" 
subtree, with content replicated from the ldap.A.supelec.fr
* section having the root suffix "dc=mycompany,dc=com" and managing its 
own subtree dc=C,dc=mycompany,dc=com

------------------------

Do you think this structure is supported by these versions of openldap?

Though not explicitely described in the "Upgrading from 2.3.x" section 
of the Administrator guide, I've read in the slapd.conf manpage 
(openldap2.4) that the "serverID" new parameter is required when using 
"separate masters contributing to a glued set of databases." which, I 
think, is a correct description of our structure. Thus I added a 
"serverID 20" parameter to the openldap2.4 server's configuration.

Am I right in using serverID for our setup?

Adding this parameter had a side effect though: we end up having two 
contextCSN cookies on the openldap2.4 database suffix entry. We managed 
to get rid of these by:
* extracting the database content with "slapcat -g",
* deleting contextCSN and entryCSN lines form the file,
* then purging the bdb files and importing with 'slapadd -g -l'.

Do you think this is a correct procedure to restart from scratch our 
openldap2.4 server ?

I thank you in advance for any information and advice you could give me 
in order to get back to a working configuration.

Regards,
Thibault