openldap on AIX 4

["Howard Chu" <hyc@highlandsun.com>]

AIX 4.2 has some dynamic linking issues... AIX dynamic libraries must be
fully resolved at build time, and they store both the library/symbolname pair
in their linkage table. This is different from Linux, Solaris, and other
SVR4-derived systems that store a list of library names and a separate list
of symbol names. With the SVR4 approach, since symbols aren't bound to
library names, the runtime linker will happily resolve a symbol from the
first object file that it sees that provides the right symbol name.

AIX's DLL scheme causes problems because their pthreads implementation relies
on a different C library (-lc_r) from the non-threaded library (-lc). This
causes trouble when mixing non-threaded libraries with threaded executables.
E.g., liblber.so is built without thread support, and is linked with -lc.
slapd is threaded, linked with libldap_r and liblber. Because all of slapd
uses ber_memalloc, and liblber is linked with -lc, that means all of slapd
uses the non-threaded version of malloc. This situation rapidly causes
crashes in malloc/free. The obvious fix is to link liblber with -lc_r, but
then single-threaded apps that use libldap (as opposed to libldap_r) will get
the wrong thing.

I guess we took a stab at working around this in the past; the configure
script on AIX always looks for cc_r (the reentrant C compiler) and uses it to
compile the whole source tree. This should have been good enough to insure
that all of our object code was linked with -lc_r automatically, because
that's cc_r's default behavior. Unfortunately, libtool gets in the way and
adds explicit "-lc" arguments when building (AIX?) shared libraries. (libtool
should not be doing this at all, it should let the chosen $(CC) command take
care of everything; that's its job...)

A final glitch turns out to be in the Berkeley DB library; even though we
tell BDB to use our malloc routines, there are still several instances in the
BDB library that call libc's malloc directly. So, if you didn't use cc_r to
build BDB, you get the wrong malloc. And, since BDB is also linked by
libtool, and libtool inserts -lc when it shouldn't, even if you build BDB
with cc_r you'll still get the wrong malloc.

You can outmaneuver libtool by adding "-lc_r" to LDFLAGS. Since this flag
will appear very early in the link command, it will be in front of libtool's
own "-lc" and the linker will resolve all its symbols against -lc_r and never
try to use -lc. Just using static libraries would avoid the problem as well.

I hear that newer pthreads implementations no longer depend on the existence
of -lc_r so this may not be an issue on AIX 4.3 or 5.x. I don't know offhand.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support