[Date Prev][Date Next] [Chronological] [Thread] [Top]
Re: Sync replication failure during startup.

To: openldap-bugs@openldap.org
Subject: Re: Sync replication failure during startup.
From: Stelios Grigoriadis <stelios.xx.grigoriadis@ericsson.com>
Date: Fri, 05 Oct 2007 15:56:53 +0200
Cc: Howard Chu <hyc@symas.com>
In-reply-to: <47060B8D.8080403@symas.com>
References: <1190879783.5150.15.camel@ebcl206.hf.sw.ericsson.se> <1190985618.5150.52.camel@ebcl206.hf.sw.ericsson.se> <46FD9602.8000202@symas.com> <1191230857.23473.16.camel@ebcl206.hf.sw.ericsson.se> <47060B8D.8080403@symas.com>
User-agent: Thunderbird 1.5 (X11/20060317)
Howard Chu wrote:
> Stelios Grigoriadis wrote:
>> The setup to reproduce the error is as follows: 1 master, 3 replicas.
>
> I've been traveling and haven't yet been able to run this test. In the
> meantime, can you run slapd with debug level sync, and upload those
> logs somewhere for this test?
>
>> 1. Start the replicas.
>> 2. Start the program that adds persons (parallell_stress_simple.sh).
>>    Actually a script that starts a number of processes (add_person.c)
>>    on different machines that add persons.
>> 3. Start the master.
>> 4. When the script completes, compare the number of added entries in
>>    the master and replicas.
>>
>> To Quanah Gibson-Mount: The slapd.conf i also provided.
>>
>> /Stelios
>
>
The output from one run. I haven't had the time to look at it yet.

ldapt1:~/openldap-2.3.38/servers/slapd # ./slapd -d 16384
@(#) $OpenLDAP: slapd 2.3.38 (Oct  5 2007 14:08:48) $
        root@ldapt1:/root/openldap-2.3.38/servers/slapd
/etc/openldap/slapd.conf: line 101: rootdn is always granted unlimited
privileges.
/etc/openldap/slapd.conf: line 135: rootdn is always granted unlimited
privileges.
slapd starting
=> bdb_idl_insert_key: c_put id failed: DB_LOCK_DEADLOCK: Locker killed
to resolve a deadlock (-30995)
=> bdb_dn2id_add: parent (dc=example,dc=com) insert failed: -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_idl_insert_key: c_put id failed: DB_LOCK_DEADLOCK: Locker killed
to resolve a deadlock (-30995)
=> bdb_dn2id_add: parent (dc=example,dc=com) insert failed: -30995
=> bdb_idl_insert_key: c_put id failed: DB_LOCK_DEADLOCK: Locker killed
to resolve a deadlock (-30995)
=> bdb_dn2id_add: parent (dc=example,dc=com) insert failed: -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_idl_insert_key: c_put id failed: DB_LOCK_DEADLOCK: Locker killed
to resolve a deadlock (-30995)
=> bdb_dn2id_add: parent (dc=example,dc=com) insert failed: -30995
=> bdb_dn2id_add: subtree (cn=dlt6107-0,dc=example,dc=com) put failed:
-30995
=> bdb_dn2id_add: subtree (cn=dlt6172-0,dc=example,dc=com) put failed:
-30995
=> bdb_dn2id_add: subtree (cn=dlt6139-0,dc=example,dc=com) put failed:
-30995
=> bdb_dn2id_add: subtree (cn=dlt6123-0,dc=example,dc=com) put failed:
-30995
=> bdb_dn2id_add: subtree (cn=dlt6080-0,dc=example,dc=com) put failed:
-30995
=> bdb_idl_insert_key: c_put id failed: DB_LOCK_DEADLOCK: Locker killed
to resolve a deadlock (-30995)
=> bdb_dn2id_add: parent (dc=example,dc=com) insert failed: -30995
=> bdb_idl_insert_key: c_put id failed: DB_LOCK_DEADLOCK: Locker killed
to resolve a deadlock (-30995)
=> bdb_dn2id_add: parent (dc=example,dc=com) insert failed: -30995
=> bdb_dn2id_add: subtree (cn=dlt30715-1,dc=example,dc=com) put failed:
-30995
=> bdb_idl_insert_key: c_put id failed: DB_LOCK_DEADLOCK: Locker killed
to resolve a deadlock (-30995)
=> bdb_dn2id_add: parent (dc=example,dc=com) insert failed: -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_idl_insert_key: c_put id failed: DB_LOCK_DEADLOCK: Locker killed
to resolve a deadlock (-30995)
=> bdb_dn2id_add: parent (dc=example,dc=com) insert failed: -30995
=> bdb_dn2id_add: subtree (cn=dlt6172-0,dc=example,dc=com) put failed:
-30995
=> bdb_dn2id_add: subtree (cn=dlt6080-0,dc=example,dc=com) put failed:
-30995
=> bdb_dn2id_add: subtree (cn=dlt6107-0,dc=example,dc=com) put failed:
-30995
=> bdb_dn2id_add: subtree (cn=dlt6123-0,dc=example,dc=com) put failed:
-30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: subtree (cn=dlt6139-0,dc=example,dc=com) put failed:
-30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: subtree (cn=dlt6123-0,dc=example,dc=com) put failed:
-30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_idl_insert_key: c_put id failed: DB_LOCK_DEADLOCK: Locker killed
to resolve a deadlock (-30995)
=> bdb_dn2id_add: parent (dc=example,dc=com) insert failed: -30995
=> bdb_dn2id_add: subtree (cn=dlt6107-0,dc=example,dc=com) put failed:
-30995
=> bdb_dn2id_add: subtree (cn=dlt6080-0,dc=example,dc=com) put failed:
-30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_idl_insert_key: c_put id failed: DB_LOCK_DEADLOCK: Locker killed
to resolve a deadlock (-30995)
=> bdb_dn2id_add: parent (dc=example,dc=com) insert failed: -30995
=> bdb_dn2id_add: subtree (cn=dlt6139-0,dc=example,dc=com) put failed:
-30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_idl_insert_key: c_put id failed: DB_LOCK_DEADLOCK: Locker killed
to resolve a deadlock (-30995)
=> bdb_dn2id_add: parent (dc=example,dc=com) insert failed: -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: subtree (cn=dlt6123-0,dc=example,dc=com) put failed:
-30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
=> bdb_idl_insert_key: c_put id failed: DB_LOCK_DEADLOCK: Locker killed
to resolve a deadlock (-30995)
=> bdb_dn2id_add: parent (dc=example,dc=com) insert failed: -30995
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
Entry cn=dlt30604-0,dc=example,dc=com CSN
20071005135356Z#00000e#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21127-0,dc=example,dc=com CSN
20071005135356Z#00001d#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt6107-0,dc=example,dc=com CSN
20071005135356Z#00001f#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt6172-0,dc=example,dc=com CSN
20071005135356Z#000020#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30678-0,dc=example,dc=com CSN
20071005135356Z#000021#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt6080-0,dc=example,dc=com CSN
20071005135356Z#000024#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21126-0,dc=example,dc=com CSN
20071005135356Z#000025#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21125-0,dc=example,dc=com CSN
20071005135356Z#000026#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9984-0,dc=example,dc=com CSN
20071005135356Z#000027#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30715-1,dc=example,dc=com CSN
20071005135356Z#000028#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30604-1,dc=example,dc=com CSN
20071005135357Z#000000#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9934-0,dc=example,dc=com CSN
20071005135357Z#000001#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9952-1,dc=example,dc=com CSN
20071005135357Z#000002#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21127-1,dc=example,dc=com CSN
20071005135357Z#000003#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9984-1,dc=example,dc=com CSN
20071005135357Z#000004#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30678-1,dc=example,dc=com CSN
20071005135357Z#000005#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9923-1,dc=example,dc=com CSN
20071005135357Z#000006#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21125-1,dc=example,dc=com CSN
20071005135358Z#000000#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt6172-1,dc=example,dc=com CSN
20071005135358Z#000001#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30616-1,dc=example,dc=com CSN
20071005135358Z#000002#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt6107-1,dc=example,dc=com CSN
20071005135358Z#000003#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9934-1,dc=example,dc=com CSN
20071005135358Z#000004#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30604-0,dc=example,dc=com CSN
20071005135356Z#00000e#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21127-0,dc=example,dc=com CSN
20071005135356Z#00001d#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt6107-0,dc=example,dc=com CSN
20071005135356Z#00001f#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt6172-0,dc=example,dc=com CSN
20071005135356Z#000020#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30678-0,dc=example,dc=com CSN
20071005135356Z#000021#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt6080-0,dc=example,dc=com CSN
20071005135356Z#000024#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21126-0,dc=example,dc=com CSN
20071005135356Z#000025#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21125-0,dc=example,dc=com CSN
20071005135356Z#000026#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9984-0,dc=example,dc=com CSN
20071005135356Z#000027#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30715-1,dc=example,dc=com CSN
20071005135356Z#000028#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30604-1,dc=example,dc=com CSN
20071005135357Z#000000#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9934-0,dc=example,dc=com CSN
20071005135357Z#000001#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9952-1,dc=example,dc=com CSN
20071005135357Z#000002#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21127-1,dc=example,dc=com CSN
20071005135357Z#000003#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9984-1,dc=example,dc=com CSN
20071005135357Z#000004#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30678-1,dc=example,dc=com CSN
20071005135357Z#000005#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9923-1,dc=example,dc=com CSN
20071005135357Z#000006#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21125-1,dc=example,dc=com CSN
20071005135358Z#000000#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt6172-1,dc=example,dc=com CSN
20071005135358Z#000001#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30616-1,dc=example,dc=com CSN
20071005135358Z#000002#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt6107-1,dc=example,dc=com CSN
20071005135358Z#000003#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9934-1,dc=example,dc=com CSN
20071005135358Z#000004#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
=> bdb_dn2id_add: put failed: DB_LOCK_DEADLOCK: Locker killed to resolve
a deadlock -30995
Entry cn=dlt30604-0,dc=example,dc=com CSN
20071005135356Z#00000e#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21127-0,dc=example,dc=com CSN
20071005135356Z#00001d#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt6107-0,dc=example,dc=com CSN
20071005135356Z#00001f#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt6172-0,dc=example,dc=com CSN
20071005135356Z#000020#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30678-0,dc=example,dc=com CSN
20071005135356Z#000021#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt6080-0,dc=example,dc=com CSN
20071005135356Z#000024#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21126-0,dc=example,dc=com CSN
20071005135356Z#000025#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21125-0,dc=example,dc=com CSN
20071005135356Z#000026#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9984-0,dc=example,dc=com CSN
20071005135356Z#000027#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30715-1,dc=example,dc=com CSN
20071005135356Z#000028#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30604-1,dc=example,dc=com CSN
20071005135357Z#000000#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9934-0,dc=example,dc=com CSN
20071005135357Z#000001#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9952-1,dc=example,dc=com CSN
20071005135357Z#000002#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21127-1,dc=example,dc=com CSN
20071005135357Z#000003#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9984-1,dc=example,dc=com CSN
20071005135357Z#000004#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30678-1,dc=example,dc=com CSN
20071005135357Z#000005#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9923-1,dc=example,dc=com CSN
20071005135357Z#000006#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21125-1,dc=example,dc=com CSN
20071005135358Z#000000#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt6172-1,dc=example,dc=com CSN
20071005135358Z#000001#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt30616-1,dc=example,dc=com CSN
20071005135358Z#000002#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt6107-1,dc=example,dc=com CSN
20071005135358Z#000003#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt9934-1,dc=example,dc=com CSN
20071005135358Z#000004#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Entry cn=dlt21126-1,dc=example,dc=com CSN
20071005135358Z#000005#00#000000 greater than snapshot
20071005135356Z#00000d#00#000000
Follow-Ups:
- Re: Sync replication failure during startup.
  - From: Howard Chu <hyc@symas.com>
References:
- Re: Sync replication failure during startup.
  - From: Stelios Grigoriadis <stelios.xx.grigoriadis@ericsson.com>
- Re: Sync replication failure during startup.
  - From: Howard Chu <hyc@symas.com>
Prev by Date: Re: No replication after power failure
Next by Date: Re: (ITS#5169) Test0001 - segmentation fault - slapd
Index(es):
- Chronological
- Thread