[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Antw: Cannot add to mdb

On 27/11/2014 01:39, Ulrich Windl wrote:

If gdb is attached to the process ("attach <PID>"), you can examine the CPU registers and the memory. Specifically the call stack. If a specific routine uses all the CPU, it's very likely that gdb will find that active routine and where it was called from. Together with local variables (info locals), a developer could find out what's wrong. Maybe you'll have to combine "-g" with "-O0" for more reliable results.

You'll get a coredump by something like kill -BUS <pid> (check ulimit -c also before starting the process), and you can detach gdb from the process after inspection.

Apologies for the delay - had some technical difficulties to take care of.

So here's what I have so far:

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions. 
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...(no debugging symbols found)... 
Attaching to program: /usr/local/libexec/slapd, process 1326
Reading symbols from /usr/local/lib/libldap_r-2.4.so.2...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/libldap_r-2.4.so.2
Reading symbols from /usr/local/lib/liblber-2.4.so.2...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/liblber-2.4.so.2
Reading symbols from /usr/local/lib/libltdl.so.7...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/libltdl.so.7
Reading symbols from /usr/local/lib/libdb-5.3.so.0...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/libdb-5.3.so.0
Reading symbols from /usr/local/lib/libsasl2.so.3...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/libsasl2.so.3
Reading symbols from /lib/libcrypt.so.5...(no debugging symbols found)...done. 
Loaded symbols for /lib/libcrypt.so.5
Reading symbols from /usr/lib/libwrap.so.6...(no debugging symbols found)...done. 
Loaded symbols for /usr/lib/libwrap.so.6
Reading symbols from /usr/local/lib/libssl.so.8...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/libssl.so.8
Reading symbols from /usr/local/lib/libcrypto.so.8...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/libcrypto.so.8
Reading symbols from /lib/libthr.so.3...(no debugging symbols found)...done.
[New Thread 802ce9800 (LWP 100233/slapd)]
[New Thread 802ce9400 (LWP 100232/slapd)]
[New Thread 802ce9000 (LWP 100231/slapd)]
[New Thread 802ce8c00 (LWP 100230/slapd)]
[New Thread 802ce8800 (LWP 100229/slapd)]
[New Thread 802ce8400 (LWP 100228/slapd)]
[New Thread 802ce8000 (LWP 100227/slapd)]
[New Thread 802ce7c00 (LWP 100226/slapd)]
[New Thread 8093a3400 (LWP 100210/slapd)]
[New Thread 802ce7800 (LWP 100209/slapd)]
[New Thread 802ce7400 (LWP 100197/slapd)]
[New Thread 802ce7000 (LWP 100196/slapd)]
[New Thread 802ce6c00 (LWP 100194/slapd)]
[New Thread 802ce6800 (LWP 100090/slapd)]
[New Thread 802ce6400 (LWP 100089/slapd)]
[New Thread 802c0dc00 (LWP 100088/slapd)]
[New Thread 802c07400 (LWP 100081/slapd)]
Loaded symbols for /lib/libthr.so.3
Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.7
Reading symbols from /usr/lib/libssl.so.6...(no debugging symbols found)...done. 
Loaded symbols for /usr/lib/libssl.so.6
Reading symbols from /lib/libcrypto.so.6...(no debugging symbols found)...done. 
Loaded symbols for /lib/libcrypto.so.6
Reading symbols from /usr/local/lib/sasl2/libanonymous.so.3...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/sasl2/libanonymous.so.3
Reading symbols from /usr/local/lib/sasl2/libcrammd5.so.3...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/sasl2/libcrammd5.so.3
Reading symbols from /usr/local/lib/sasl2/libdigestmd5.so.3...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/sasl2/libdigestmd5.so.3
Reading symbols from /usr/local/lib/sasl2/liblogin.so.3...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/sasl2/liblogin.so.3
Reading symbols from /usr/local/lib/sasl2/libscram.so.3...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/sasl2/libscram.so.3
Reading symbols from /usr/local/lib/sasl2/libntlm.so.3...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/sasl2/libntlm.so.3
Reading symbols from /usr/local/lib/sasl2/libotp.so.3...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/sasl2/libotp.so.3
Reading symbols from /usr/lib/libopie.so.7...(no debugging symbols found)...done. 
Loaded symbols for /usr/lib/libopie.so.7
Reading symbols from /lib/libmd.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib/libmd.so.5
Reading symbols from /usr/local/lib/sasl2/libplain.so.3...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/sasl2/libplain.so.3
Reading symbols from /usr/local/lib/sasl2/libsasldb.so.3...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/sasl2/libsasldb.so.3
Reading symbols from /usr/local/lib/sasl2/libgssapiv2.so...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/sasl2/libgssapiv2.so
Reading symbols from /usr/lib/libgssapi.so.10...(no debugging symbols found)...done. 
Loaded symbols for /usr/lib/libgssapi.so.10
Reading symbols from /usr/lib/libheimntlm.so.10...(no debugging symbols found)...done. 
Loaded symbols for /usr/lib/libheimntlm.so.10
Reading symbols from /usr/local/lib/libkrb5.so...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/libkrb5.so
Reading symbols from /usr/lib/libhx509.so.10...(no debugging symbols found)...done. 
Loaded symbols for /usr/lib/libhx509.so.10
Reading symbols from /usr/local/lib/libcom_err.so...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/libcom_err.so
Reading symbols from /usr/lib/libasn1.so.10...(no debugging symbols found)...done. 
Loaded symbols for /usr/lib/libasn1.so.10
Reading symbols from /usr/lib/libroken.so.10...(no debugging symbols found)...done. 
Loaded symbols for /usr/lib/libroken.so.10
Reading symbols from /usr/lib/libkrb5.so.10...(no debugging symbols found)...done. 
Loaded symbols for /usr/lib/libkrb5.so.10
Reading symbols from /usr/local/lib/libk5crypto.so...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/libk5crypto.so
Reading symbols from /usr/local/lib/libkrb5support.so...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/libkrb5support.so
Reading symbols from /usr/local/lib/libintl.so.9...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/libintl.so.9
Reading symbols from /usr/lib/libcom_err.so.5...(no debugging symbols found)...done. 
Loaded symbols for /usr/lib/libcom_err.so.5
Reading symbols from /usr/local/lib/libiconv.so.3...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/lib/libiconv.so.3
Reading symbols from /usr/local/libexec/openldap/back_mdb-2.4.so.2...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/libexec/openldap/back_mdb-2.4.so.2
Reading symbols from /usr/local/libexec/openldap/back_bdb-2.4.so.2...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/libexec/openldap/back_bdb-2.4.so.2
Reading symbols from /usr/local/libexec/openldap/back_hdb-2.4.so.2...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/libexec/openldap/back_hdb-2.4.so.2
Reading symbols from /usr/local/libexec/openldap/back_ldap-2.4.so.2...(no debugging symbols found)...done. 
Loaded symbols for /usr/local/libexec/openldap/back_ldap-2.4.so.2
Reading symbols from /usr/lib/libz.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libz.so
Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done. 
Loaded symbols for /libexec/ld-elf.so.1
[Switching to Thread 802ce9800 (LWP 100233/slapd)]
0x000000080200187c in pthread_kill () from /lib/libthr.so.3
(gdb) bt
#0  0x000000080200187c in pthread_kill () from /lib/libthr.so.3
#1  0x0000000801ffbb25 in pthread_getschedparam () from /lib/libthr.so.3
#2  0x0000000802003c8d in pthread_cond_signal () from /lib/libthr.so.3
#3  0x000000080097c868 in ldap_pvt_thread_pool_submit ()
   from /usr/local/lib/libldap_r-2.4.so.2
#4  0x0000000801ffa0a4 in pthread_getprio () from /lib/libthr.so.3
#5  0x0000000000000000 in ?? ()
Error accessing memory address 0x7ffff73ed000: Bad address.

That occurs before and after adding the new data.
I'm assuming now that I have to rebuild with debug, but I'm not sure exactly how I do that with ports yet and that will take more time. I also can't seem to remember how to access threads... or at least determine which one to follow. 

At any rate, is any of this looking familiar at all?


Da Rock <openldap-technical@herveybayaustralia.com.au> schrieb am 26.11.2014 um

10:41 in Nachricht <5475A053.7060209@herveybayaustralia.com.au>:

On 26/11/2014 18:23, Ulrich Windl wrote:

What you could try is (assuming you have debug symbols in your binary)

attaching gdb to the running slapd to see where it is spending CPU time. I
can't remember the commands precisely, but "bt (backtrace)" and "info
threads" should point you in the right direction.

Note that slapd will likely stop responding while gdb is attached, so you

could also try to force a core dump for offline examination.
I'm not sure I follow here. How would this work? I attach gdb to the
running slapd I get, but if it stops how does that help me? I've only
had a little bit of experience with gdb...

How would I get a core dump, as well? That sounds like it might be more
useful.

Maybe the gurus get a clue what may be wrong then...

Regards,
Ulrich


Da Rock <openldap-technical@herveybayaustralia.com.au> schrieb am 26.11.2014 um

01:31 in Nachricht <54751F73.3080902@herveybayaustralia.com.au>:

I'm trying to get openldap to play nice with mdb given that it is the
"recommended" database backend for it now- although the conf wasn't an
issue excepting I'm playing with the new cn=config setup we're expected
to use now as well (even though it is mainly broken).

My issue is that it seems to not respond like the older bdb/hdb
databases. And when I say respond, I mean it hangs the ldapadd and makes
slapd go into conniptions. I see slapd go to 100% WCPU and not come down
as well as going into a uwait state. I've left it going for 10 minutes
or more with no change, and I'm only adding 1 small entry of less the 10
lines. Strangely, I can still view other entries in the specific db as
well access the rest of the server, which I won't complain about (aren't
threads a wonderful invention? ).

So coming to the experts - got a fix at all? Or should I just go back to
ye olde db backends? At this point I have a db I can't add anything to.

And before anyone asks, there is practically nothing in the logs that I
can see; and I set the logging to everything (-1). I see recognition of
the user in the acl and then nothing. The only possible curious entry is
some blank lines and a number (that changes each time), so nothing
informative.

I set it up using the cn=config (and I'm still not entirely convinced
that I will keep cn=config, but apparently it could be gone next version
according to the grapevine, so the consensus is to suck it up and get
used to it or your panties will get in a bunch and around your ankles
when the upgrade comes along), and I've got only olcDBMaxSize.
olcSizeLimit (not sure exactly which of these 2 can go just yet),
olcDBMode, olcDBDirectory, and olcDatabase and the obviously root attrs.
My max size I've set larger than 50M (so 7 digits) which is larger than
what I have in another db so far, and I figure I can add more if needed
- currently it is sitting at 64k.

I'm using FreeBSD 9.1, ports Openldap version is 2.4.40_1 with bdb/hdb
and mdb set in config. But I notice lmdb is not installed as a
dependency - is this right?

I've been on this for near a week now with no further advancement so any
help would be very welcome at this point. No googling seems to find
anything remotely similar either.

TIA