Re: Data is not getting replicated correctly in multimaster configuratio

Re: Data is not getting replicated correctly in multimaster configuration

To: Gurjot Kaur <gurjot.kaur@aricent.com>

Subject: Re: Data is not getting replicated correctly in multimaster configuration

From: PRATIK SINGAL <pratik.singal13@gmail.com>

Date: Fri, 8 May 2015 15:04:20 +0530

Cc: "openldap-technical@openldap.org" <openldap-technical@openldap.org>

Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=+C6oznhZh+2O6eT1rBuJnTJKbCADCE3HOonhrQirEYQ=; b=jJWr9qq0cvMX4ZWXcRPgvgRgzCxy7BIFOBQ/Ugurnemslq+iACFxrouUban9WIMLU5 gHp6/TLN/pR3WWQ+gOFafhMJqxrmc9RrOs1s+p3ggdcCejqq9Q+rkkREh6bFwOP7+81+ qjqksRDNAFPi/2pOyIBE9wxo/rkYDR1Yolu1POHhJgSfma6RPY6PvKOm4AOf2GfQLgsP QjSEbupJW989gDPu4BnBkBd59FVR6ofmIjjt4mPc+NyPO1WiQAZ4m4X6bZS0ouqWQ0+X vG6vPwnQvxBgSN22GLX/fGUw/7IjOBBejVAf3AQwEvAZyihL6xp0lM9WwsPe7l9+iF+e 1XaQ==

In-reply-to: <605a436002484aea9caf7cee9605df40@GURMBXV02.AD.ARICENT.COM>

References: <5f6faabd29084fbc87049a9a7d1929b2@GURMBXV02.AD.ARICENT.COM> <20150408112248.19016a58@pink.avci.de> <e65ee56a625b4bf08d29ff48e965ea33@GURMBXV02.AD.ARICENT.COM> <552D2BD1.70709@stroeder.com> <605a436002484aea9caf7cee9605df40@GURMBXV02.AD.ARICENT.COM>

Hello Gurjot

Can you please let me know what were the steps followed for the replication.

As initially i was using Rhel 4 with ldap 2.4.23 and replication was working but when one system used to go down and come up the dtaa was not able to syn for that time stamp.

Attaching the issues which were caused earlier.

Regards,

Pratik

On Fri, May 8, 2015 at 2:30 PM, Gurjot Kaur <gurjot.kaur@aricent.com> wrote:

I have upgraded LDAP to 2.4.40. Everything is working fine.
Multimaster replication is accurate.
Thanks for your help.

Gurjot Kaur

"DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus."

I have installed openldap on two machines and tried to setup multi-way replication.I am able to perform the multi-way replication (Add/Update/delete) when both the machines are up and ldap is running. But when my one machine goes down(server2) and records are added,deleted,modified on (server1) are not getting replicated on server2 when its is up and ldap service is running. **Below are the machine on which ldap is installed:** [root@localhost openldap]# cat /etc/*-release LSB_VERSION=base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch Red Hat Enterprise Linux Server release 6.4 (Santiago) Red Hat Enterprise Linux Server release 6.4 (Santiago) **Openldap version:** [root@localhost openldap]# slapd -V @(#) $OpenLDAP: slapd 2.4.23 (Oct 31 2012 08:14:14) $ mockbuild@x86-022.build.eng.bos.redhat.com:/builddir/build/BUILD /openldap-2.4.23/openldap-2.4.23/build-servers/servers/slapd **Below are the slapd.conf files from both the servers:** **1. Server1:** include /etc/openldap/schema/corba.schema include /etc/openldap/schema/core.schema include /etc/openldap/schema/cosine.schema include /etc/openldap/schema/duaconf.schema include /etc/openldap/schema/dyngroup.schema include /etc/openldap/schema/inetorgperson.schema include /etc/openldap/schema/java.schema include /etc/openldap/schema/misc.schema include /etc/openldap/schema/nis.schema include /etc/openldap/schema/openldap.schema include /etc/openldap/schema/ppolicy.schema include /etc/openldap/schema/collective.schema include /etc/openldap/schema/sim-data-attributes.schema include /etc/openldap/schema/eps-pdn-attributes.schema include /etc/openldap/schema/SIMSubscription.schema include /etc/openldap/schema/EPSSubscription.schema include /etc/openldap/schema/PDNSubscriptionContexts.schema allow bind_v2 pidfile /var/run/openldap/slapd.pid argsfile /var/run/openldap/slapd.args modulepath /usr/lib64/openldap moduleload syncprov.la serverID 1 database config access to * by dn.exact="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" manage by * none database monitor access to * by dn.exact="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" read by dn.exact="cn=Manager,dc=example,dc=com" read by * none database bdb suffix "dc=example,dc=com" checkpoint 1024 15 rootdn "cn=Manager,dc=example,dc=com" rootpw ******redacted****** sizelimit unlimited directory /var/lib/ldap index objectClass eq,pres index ou,cn,mail,surname,givenname eq,pres,sub index uidNumber,gidNumber,loginShell eq,pres index uid,memberUid eq,pres,sub index nisMapName,nisMapEntry eq,pres,sub overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 100 syncrepl rid=100 provider=ldap://172.16.101.60:389 type=refreshAndPersist retry="60 +" searchbase="dc=example,dc=com" scope=sub schemachecking=on bindmethod=simple binddn="cn=Manager,dc=example,dc=com" credentials=secret mirrormode on loglevel 16777 logfile /var/log/ldap.log **2. Server2:** [root@localhost openldap]# cat slapd.conf include /etc/openldap/schema/corba.schema include /etc/openldap/schema/core.schema include /etc/openldap/schema/cosine.schema include /etc/openldap/schema/duaconf.schema include /etc/openldap/schema/dyngroup.schema include /etc/openldap/schema/inetorgperson.schema include /etc/openldap/schema/java.schema include /etc/openldap/schema/misc.schema include /etc/openldap/schema/nis.schema include /etc/openldap/schema/openldap.schema include /etc/openldap/schema/ppolicy.schema include /etc/openldap/schema/collective.schema include /etc/openldap/schema/sim-data-attributes.schema include /etc/openldap/schema/eps-pdn-attributes.schema include /etc/openldap/schema/SIMSubscription.schema include /etc/openldap/schema/EPSSubscription.schema include /etc/openldap/schema/PDNSubscriptionContexts.schema allow bind_v2 pidfile /var/run/openldap/slapd.pid argsfile /var/run/openldap/slapd.args modulepath /usr/lib64/openldap moduleload syncprov.la serverID 2 database config access to * by dn.exact="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" manage by * none database monitor access to * by dn.exact="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth" read by dn.exact="cn=Manager,dc=example,dc=com" read by * none database bdb suffix "dc=example,dc=com" checkpoint 1024 15 rootdn "cn=Manager,dc=example,dc=com" rootpw *****redacted***** sizelimit unlimited directory /var/lib/ldap index objectClass eq,pres index ou,cn,mail,surname,givenname eq,pres,sub index uidNumber,gidNumber,loginShell eq,pres index uid,memberUid eq,pres,sub index nisMapName,nisMapEntry eq,pres,sub overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 100 syncrepl rid=100 provider=ldap://172.16.101.36:389 type=refreshAndPersist retry="60 +" searchbase="dc=example,dc=com" scope=sub schemachecking=on bindmethod=simple binddn="cn=Manager,dc=example,dc=com" credentials=secret mirrormode on loglevel 393 logfile /var/log/ldap.log **On both the nodes the Syn log are enabled:** # Logging # - trace function calls (1) # - connection management (8) # - ACL processing (128) # - stats log connections/operations/results (256) # - LDAPSync replication (16384) # (1 + 8 + 128 + 256+ 16384)=16777 loglevel 16777 logfile /var/log/ldap.log **Below are the logs from both the servers** **Server2:** Apr 30 00:39:29 localhost slapd[5891]: daemon: removing 15 Apr 30 00:39:29 localhost slapd[5891]: conn=1001 fd=15 closed (slapd shutdown) Apr 30 00:39:29 localhost slapd[5891]: slapd shutdown: waiting for 0 operations/tasks to finish Apr 30 00:39:29 localhost slapd[5891]: slapd shutdown: initiated Apr 30 00:39:29 localhost slapd[5891]: ====> bdb_cache_release_all Apr 30 00:39:29 localhost slapd[5891]: slapd destroy: freeing system resources. Apr 30 00:39:29 localhost slapd[5891]: syncinfo_free: rid=100 Apr 30 00:39:29 localhost slapd[5891]: connection_get(13): got connid=0 Apr 30 00:39:29 localhost slapd[5891]: daemon: removing 13r Apr 30 00:39:29 localhost slapd[5891]: slapd stopped. **Server1:Trying to conncet to Server2 but since its stopped so failed** Apr 29 19:10:27 localhost slapd[28124]: =>do_syncrepl rid=100 Apr 29 19:10:27 localhost slapd[28124]: slap_client_connect: URI=ldap://172.16.101.60:389 DN="cn=manager,dc=example,dc=com" ldap_sasl_bind_s failed (-1) Apr 29 19:10:27 localhost slapd[28124]: do_syncrepl: rid=100 rc -1 retrying Apr 29 19:10:27 localhost slapd[28124]: daemon: activity on 1 descriptor Apr 29 19:10:27 localhost slapd[28124]: daemon: activity on: **Server1:Adding new entry** Apr 29 19:12:11 localhost slapd[28124]: op tag 0x68, time 1430314931 Apr 29 19:12:11 localhost slapd[28124]: conn=1001 op=15 do_add Apr 29 19:12:11 localhost slapd[28124]: => get_ctrls Apr 29 19:12:11 localhost slapd[28124]: => get_ctrls: oid="2.16.840.1.113730.3.4.2" (noncritical) Apr 29 19:12:11 localhost slapd[28124]: <= get_ctrls: n=1 rc=0 err="" Apr 29 19:12:11 localhost slapd[28124]: >>> dnPrettyNormal: <IMSI=123,dc=example,dc=com> Apr 29 19:12:11 localhost slapd[28124]: <<< dnPrettyNormal: <IMSI=123,dc=example,dc=com>, <IMSI=123,dc=example,dc=com> Apr 29 19:12:11 localhost slapd[28124]: conn=1001 op=15 ADD dn="IMSI=123,dc=example,dc=com" Apr 29 19:12:11 localhost slapd[28124]: oc_check_required entry (IMSI=123,dc=example,dc=com), objectClass "SIMSubscription" Apr 29 19:12:11 localhost slapd[28124]: oc_check_allowed type "IMSI" Apr 29 19:12:11 localhost slapd[28124]: oc_check_allowed type "objectClass" Apr 29 19:12:11 localhost slapd[28124]: oc_check_allowed type "structuralObjectClass" Apr 29 19:12:11 localhost slapd[28124]: slap_queue_csn: queing 0x7fc1c7ffe030 20150429134211.927786Z#000000#001#000000 Apr 29 19:12:11 localhost slapd[28124]: bdb_dn2entry("IMSI=123,dc=example,dc=com") Apr 29 19:12:11 localhost slapd[28124]: => bdb_dn2id("IMSI=123,dc=example,dc=com") Apr 29 19:12:11 localhost slapd[28124]: <= bdb_dn2id: get failed: DB_NOTFOUND: No matching key/data pair found (-30988) Apr 29 19:12:11 localhost slapd[28124]: => access_allowed: add access to "dc=example,dc=com" "children" requested Apr 29 19:12:11 localhost slapd[28124]: <= root access granted Apr 29 19:12:11 localhost slapd[28124]: => access_allowed: add access granted by manage(=mwrscxd) Apr 29 19:12:11 localhost slapd[28124]: => access_allowed: add access to "IMSI=123,dc=example,dc=com" "entry" requested Apr 29 19:12:11 localhost slapd[28124]: <= root access granted Apr 29 19:12:11 localhost slapd[28124]: => access_allowed: add access granted by manage(=mwrscxd) Apr 29 19:12:11 localhost slapd[28124]: => bdb_dn2id_add 0x5a: "IMSI=123,dc=example,dc=com" Apr 29 19:12:11 localhost slapd[28124]: <= bdb_dn2id_add 0x5a: 0 Apr 29 19:12:11 localhost slapd[28124]: => index_entry_add( 90, "IMSI=123,dc=example,dc=com" ) Apr 29 19:12:11 localhost slapd[28124]: => key_change(ADD,5a) Apr 29 19:12:11 localhost slapd[28124]: <= key_change 0 Apr 29 19:12:11 localhost slapd[28124]: => key_change(ADD,5a) Apr 29 19:12:11 localhost slapd[28124]: <= key_change 0 Apr 29 19:12:11 localhost slapd[28124]: => key_change(ADD,5a) Apr 29 19:12:11 localhost slapd[28124]: <= key_change 0 Apr 29 19:12:11 localhost slapd[28124]: <= index_entry_add( 90, "IMSI=123,dc=example,dc=com" ) success Apr 29 19:12:11 localhost slapd[28124]: daemon: activity on 1 descriptor Apr 29 19:12:11 localhost slapd[28124]: daemon: activity on: Apr 29 19:12:11 localhost slapd[28124]: Apr 29 19:12:11 localhost slapd[28124]: daemon: epoll: listen=7 active_threads=0 tvp=zero Apr 29 19:12:11 localhost slapd[28124]: daemon: epoll: listen=8 active_threads=0 tvp=zero Apr 29 19:12:11 localhost slapd[28124]: daemon: epoll: listen=9 active_threads=0 tvp=zero Apr 29 19:12:11 localhost slapd[28124]: => entry_encode(0x0000005a): IMSI=123,dc=example,dc=com Apr 29 19:12:11 localhost slapd[28124]: <= entry_encode(0x0000005a): IMSI=123,dc=example,dc=com **Server1:After starting the server 2 Server 1 was able to communicate with server2** Apr 29 19:12:52 localhost slapd[28124]: daemon: added 14r (active) listener=(nil) Apr 29 19:12:52 localhost slapd[28124]: conn=1002 fd=14 ACCEPT from IP=172.16.101.60:42695 (IP=0.0.0.0:389) Apr 29 19:12:52 localhost slapd[28124]: daemon: activity on 2 descriptors Apr 29 19:12:52 localhost slapd[28124]: daemon: activity on: Apr 29 19:12:52 localhost slapd[28124]: 14r Apr 29 19:12:52 localhost slapd[28124]: Apr 29 19:12:52 localhost slapd[28124]: daemon: read active on 14 Apr 29 19:12:52 localhost slapd[28124]: daemon: epoll: listen=7 active_threads=0 tvp=zero Apr 29 19:12:52 localhost slapd[28124]: daemon: epoll: listen=8 active_threads=0 tvp=zero Apr 29 19:12:52 localhost slapd[28124]: daemon: epoll: listen=9 active_threads=0 tvp=zero Apr 29 19:12:52 localhost slapd[28124]: connection_get(14): got connid=1002 Apr 29 19:12:52 localhost slapd[28124]: connection_read(14): checking for input on id=1002 Apr 29 19:12:52 localhost slapd[28124]: op tag 0x60, time 1430314972 Apr 29 19:12:52 localhost slapd[28124]: conn=1002 op=0 do_bind Apr 29 19:12:52 localhost slapd[28124]: >>> dnPrettyNormal: <cn=manager,dc=example,dc=com> Apr 29 19:12:52 localhost slapd[28124]: <<< dnPrettyNormal: <cn=manager,dc=example,dc=com>, <cn=manager,dc=example,dc=com> Apr 29 19:12:52 localhost slapd[28124]: conn=1002 op=0 BIND dn="cn=manager,dc=example,dc=com" method=128 Apr 29 19:12:52 localhost slapd[28124]: do_bind: version=3 dn="cn=manager,dc=example,dc=com" method=128 Apr 29 19:12:52 localhost slapd[28124]: conn=1002 op=0 BIND dn="cn=manager,dc=example,dc=com" mech=SIMPLE ssf=0 Apr 29 19:12:52 localhost slapd[28124]: do_bind: v3 bind: "cn=manager,dc=example,dc=com" to "cn=manager,dc=example,dc=com" Apr 29 19:12:52 localhost slapd[28124]: send_ldap_result: conn=1002 op=0 p=3 Apr 29 19:12:52 localhost slapd[28124]: send_ldap_response: msgid=1 tag=97 err=0 Apr 29 19:12:52 localhost slapd[28124]: conn=1002 op=0 RESULT tag=97 err=0 text= Apr 29 19:12:52 localhost slapd[28124]: daemon: activity on 2 descriptors Apr 29 19:12:52 localhost slapd[28124]: daemon: activity on: Apr 29 19:12:52 localhost slapd[28124]: 14r Apr 29 19:12:52 localhost slapd[28124]: Apr 29 19:12:52 localhost slapd[28124]: daemon: read active on 14 Apr 29 19:12:52 localhost slapd[28124]: daemon: epoll: listen=7 active_threads=0 tvp=zero Apr 29 19:12:52 localhost slapd[28124]: daemon: epoll: listen=8 active_threads=0 tvp=zero Apr 29 19:12:52 localhost slapd[28124]: daemon: epoll: listen=9 active_threads=0 tvp=zero Apr 29 19:12:52 localhost slapd[28124]: connection_get(14): got connid=1002 Apr 29 19:12:52 localhost slapd[28124]: connection_read(14): checking for input on id=1002 Apr 29 19:12:52 localhost slapd[28124]: op tag 0x63, time 1430314972 Apr 29 19:12:52 localhost slapd[28124]: conn=1002 op=1 do_search Apr 29 19:12:52 localhost slapd[28124]: >>> dnPrettyNormal: <dc=example,dc=com> Apr 29 19:12:52 localhost slapd[28124]: <<< dnPrettyNormal: <dc=example,dc=com>, <dc=example,dc=com> Apr 29 19:12:52 localhost slapd[28124]: => get_ctrls Apr 29 19:12:52 localhost slapd[28124]: => get_ctrls: oid="1.3.6.1.4.1.4203.1.9.1.1" (noncritical) Apr 29 19:12:52 localhost slapd[28124]: => get_ctrls: oid="2.16.840.1.113730.3.4.2" (critical) Apr 29 19:12:52 localhost slapd[28124]: <= get_ctrls: n=2 rc=0 err="" **Server2:After server 2 was started it was also able to communicate with server1 but the replication did not happened** Apr 30 00:43:30 localhost slapd[6070]: >>> slap_listener(ldap:///) Apr 30 00:43:30 localhost slapd[6070]: daemon: listen=7, new connection on 14 Apr 30 00:43:30 localhost slapd[6070]: daemon: added 14r (active) listener=(nil) Apr 30 00:43:30 localhost slapd[6070]: conn=1000 fd=14 ACCEPT from IP=172.16.101.36:46102 (IP=0.0.0.0:389) Apr 30 00:43:30 localhost slapd[6070]: daemon: activity on 2 descriptors Apr 30 00:43:30 localhost slapd[6070]: daemon: activity on: Apr 30 00:43:30 localhost slapd[6070]: 14r Apr 30 00:43:30 localhost slapd[6070]: Apr 30 00:43:30 localhost slapd[6070]: daemon: read active on 14 Apr 30 00:43:30 localhost slapd[6070]: daemon: epoll: listen=7 active_threads=0 tvp=zero Apr 30 00:43:30 localhost slapd[6070]: daemon: epoll: listen=8 active_threads=0 tvp=zero Apr 30 00:43:30 localhost slapd[6070]: daemon: epoll: listen=9 active_threads=0 tvp=zero Apr 30 00:43:30 localhost slapd[6070]: connection_get(14): got connid=1000 Apr 30 00:43:30 localhost slapd[6070]: connection_read(14): checking for input on id=1000 Apr 30 00:43:30 localhost slapd[6070]: op tag 0x60, time 1430334810 Apr 30 00:43:30 localhost slapd[6070]: conn=1000 op=0 do_bind Apr 30 00:43:30 localhost slapd[6070]: >>> dnPrettyNormal: <cn=manager,dc=example,dc=com> Apr 30 00:43:30 localhost slapd[6070]: <<< dnPrettyNormal: <cn=manager,dc=example,dc=com>, <cn=manager,dc=example,dc=com> Apr 30 00:43:30 localhost slapd[6070]: conn=1000 op=0 BIND dn="cn=manager,dc=example,dc=com" method=128 Apr 30 00:43:30 localhost slapd[6070]: do_bind: version=3 dn="cn=manager,dc=example,dc=com" method=128 Apr 30 00:43:30 localhost slapd[6070]: conn=1000 op=0 BIND dn="cn=manager,dc=example,dc=com" mech=SIMPLE ssf=0 Apr 30 00:43:30 localhost slapd[6070]: do_bind: v3 bind: "cn=manager,dc=example,dc=com" to "cn=manager,dc=example,dc=com" Apr 30 00:43:30 localhost slapd[6070]: send_ldap_result: conn=1000 op=0 p=3 Apr 30 00:43:30 localhost slapd[6070]: send_ldap_response: msgid=1 tag=97 err=0 Apr 30 00:43:30 localhost slapd[6070]: conn=1000 op=0 RESULT tag=97 err=0 text= Apr 30 00:43:30 localhost slapd[6070]: daemon: activity on 1 descriptor Apr 30 00:43:30 localhost slapd[6070]: daemon: activity on: Apr 30 00:43:30 localhost slapd[6070]: Apr 30 00:43:30 localhost slapd[6070]: daemon: epoll: listen=7 active_threads=0 tvp=zero Apr 30 00:43:30 localhost slapd[6070]: daemon: epoll: listen=8 active_threads=0 tvp=zero Apr 30 00:43:30 localhost slapd[6070]: daemon: epoll: listen=9 active_threads=0 tvp=zero Apr 30 00:43:30 localhost slapd[6070]: daemon: activity on 1 descriptor Apr 30 00:43:30 localhost slapd[6070]: daemon: activity on: Apr 30 00:43:30 localhost slapd[6070]: 14r Apr 30 00:43:30 localhost slapd[6070]: Apr 30 00:43:30 localhost slapd[6070]: daemon: read active on 14 Apr 30 00:43:30 localhost slapd[6070]: daemon: epoll: listen=7 active_threads=0 tvp=zero Apr 30 00:43:30 localhost slapd[6070]: daemon: epoll: listen=8 active_threads=0 tvp=zero Apr 30 00:43:30 localhost slapd[6070]: daemon: epoll: listen=9 active_threads=0 tvp=zero Apr 30 00:43:30 localhost slapd[6070]: connection_get(14): got connid=1000 Apr 30 00:43:30 localhost slapd[6070]: connection_read(14): checking for input on id=1000 Apr 30 00:43:30 localhost slapd[6070]: op tag 0x63, time 1430334810 Apr 30 00:43:30 localhost slapd[6070]: conn=1000 op=1 do_search