[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: Openldap 2.4.15, TLS, SASL EXTERNAL and authzregexp = segfaults

This appears to be due to an incorrect patch for ITS#5849. A fix is now in HEAD. (libldap/tls_o.c)

Mathieu MILLET wrote: 
Hi everyone,

We have a configuration with 2 Openldap in Multimaster Replication mode,
using TLS, client certificate and SASL EXTERNAL to secure the replication.
(Two sets of certificate are used to differentiate the replication of
cn=config and the data backend)

It is working in 2.4.13 (on Red Hat Entreprise Linux 4.5 and Debian 5),
compiled from sources, with openssl libs (not gnutls).

Being affected by ITS#5906 (slapo-rwm with back-config) and ITS#5843 (slapd
syncrepl MMR with deleted entries), I decided to try on a (test)
environment this new version.

With 2.4.15 (and also reproduced in 2.4.14), our configuration segfaults on
one of the two nodes at a short period of time after the 1st replication.
When restarting the segfaulted node, the other segfaults and so on.

The segfault happens when just adding the syncrepl configuration for the
cn=config backend, but some times they are alive long enough to enable
syncrepl options for the databackend, but then again, segfaults always
happen.

During some segfaults, I got some backtraces that follow :
*** glibc detected *** /usr/local/libexec/slapd: realloc(): invalid
pointer: 0xb6db9260 ***
======= Backtrace: =========
/lib/i686/cmov/libc.so.6[0xb6ccf624]
/lib/i686/cmov/libc.so.6(realloc+0x242)[0xb6cd3c82]
/usr/lib/i686/cmov/libcrypto.so.0.9.8[0xb6e224c5]
/usr/lib/i686/cmov/libcrypto.so.0.9.8(CRYPTO_realloc+0xab)[0xb6e22c0b]
/usr/lib/i686/cmov/libcrypto.so.0.9.8(BUF_MEM_grow+0x75)[0xb6e83415]
/usr/lib/i686/cmov/libcrypto.so.0.9.8[0xb6ea95a4]
/usr/lib/i686/cmov/libcrypto.so.0.9.8(ASN1_item_ex_i2d+0x14d)Abandon

or
*** glibc detected *** /usr/local/libexec/slapd: realloc(): invalid
pointer: 0xb6de4260 ***
======= Backtrace: =========
/lib/i686/cmov/libc.so.6[0xb6cfa624]
/lib/i686/cmov/libc.so.6(realloc+0x242)[0xb6cfec82]
/usr/lib/i686/cmov/libcrypto.so.0.9.8[0xb6e4d4c5]
/usr/lib/i686/cmov/libcrypto.so.0.9.8(CRYPTO_realloc+0xab)[0xb6e4dc0b]
/usr/lib/i686/cmov/libcrypto.so.0.9.8(BUF_MEM_grow+0x75)[0xb6eae415]
/usr/lib/i686/cmov/libcrypto.so.0.9.8[0xb6ed45a4]
/usr/lib/i686/cmov/libcrypto.so.0.9.8(ASN1_item_ex_i2d+0x14d)[0xb6edbfbd]
/usr/lib/i686/cmov/libcrypto.so.0.9.8[0xb6edc5b5]
/usr/lib/i686/cmov/libcrypto.so.0.9.8(ASN1_item_ex_i2d+0x2f3)[0xb6edc163]
/usr/lib/i686/cmov/libcrypto.so.0.9.8[0xb6edc5b5]
/usr/lib/i686/cmov/libcrypto.so.0.9.8(ASN1_item_ex_i2d+0x2f3)[0xb6edc163]
/usr/lib/i686/cmov/libcrypto.so.0.9.8(ASN1_item_i2d+0x53)[0xb6edc923]
/usr/lib/i686/cmov/libcrypto.so.0.9.8(i2d_X509+0x2e)[0xb6ed506e]
/usr/lib/i686/cmov/libssl.so.0.9.8(ssl3_output_cert_chain+0x3d4)[0xb6f7b824]
/usr/lib/i686/cmov/libssl.so.0.9.8(ssl3_send_client_certificate+0x142)[0xb6f721b2]
/usr/lib/i686/cmov/libssl.so.0.9.8(ssl3_connect+0xb3)[0xb6f759d3]
/usr/lib/i686/cmov/libssl.so.0.9.8(SSL_connect+0x2a)[0xb6f89c1a]
/usAbandon

It definitely has something to do with TLS stuff.

After more testing, the ldap* clients also segfault when performing TLS and
SASL External with Client Certificate.

Has anybody encounter this behaviour ?

Thanks in advance for any help,
Sincerely yours, Mathieu MILLET.


******************* Startup config (of one node) **************
----------------
slapd.d/cn=config/olcDatabase={-1}frontend.ldif
----------------
dn: olcDatabase={-1}frontend
objectClass: olcDatabaseConfig
objectClass: olcFrontendConfig
olcDatabase: {-1}frontend
olcAccess: {0}to dn.base=""  by * read
olcAccess: {1}to dn.base="cn=subschema"  by * read
olcAccess: {2}to *  by self write  by users read  by anonymous auth
olcAddContentAcl: FALSE
olcLastMod: TRUE
olcMaxDerefDepth: 0
olcReadOnly: FALSE
olcSchemaDN: cn=Subschema
olcMonitoring: FALSE
structuralObjectClass: olcDatabaseConfig
entryUUID: 10002a99-3485-4805-a247-9e4ee777135d
creatorsName: cn=config
createTimestamp: 20090224192423Z
entryCSN: 20090224192423.202231Z#000000#000#000000
modifiersName: cn=config
modifyTimestamp: 20090224192423Z

----------------
slapd.d/cn=config/olcDatabase={0}config.ldif
----------------
dn: olcDatabase={0}config
objectClass: olcDatabaseConfig
olcDatabase: {0}config
olcAccess: {0}to *  by * none
olcAddContentAcl: TRUE
olcLastMod: TRUE
olcMaxDerefDepth: 15
olcReadOnly: FALSE
olcRootDN: cn=config
olcRootPW:: c2VjcmV0
olcMonitoring: FALSE
structuralObjectClass: olcDatabaseConfig
entryUUID: fc35a505-ba8f-4bbf-828e-b061bb3aabba
creatorsName: cn=config
createTimestamp: 20090224192423Z
entryCSN: 20090224192423.202231Z#000000#000#000000
modifiersName: cn=config
modifyTimestamp: 20090224192423Z

----------------
slapd.d/cn=config/olcDatabase={2}hdb/olcOverlay={0}ppolicy.ldif
----------------
dn: olcOverlay={0}ppolicy
objectClass: olcConfig
objectClass: olcOverlayConfig
objectClass: olcPPolicyConfig
olcOverlay: {0}ppolicy
olcPPolicyDefault: cn=default,ou=policies,dc=htam,dc=net
olcPPolicyHashCleartext: FALSE
olcPPolicyUseLockout: FALSE
structuralObjectClass: olcPPolicyConfig
entryUUID: 8078dd1d-369e-4c62-9fdc-1ce6820482d8
creatorsName: cn=config
createTimestamp: 20090302125140Z
entryCSN: 20090302125140.681319Z#000000#001#000000
modifiersName: cn=config
modifyTimestamp: 20090302125140Z

----------------
slapd.d/cn=config/olcDatabase={2}hdb/olcOverlay={1}memberof.ldif
----------------
dn: olcOverlay={1}memberof
objectClass: olcConfig
objectClass: olcOverlayConfig
objectClass: olcMemberOf
olcOverlay: {1}memberof
olcMemberOfDangling: ignore
olcMemberOfRefInt: TRUE
olcMemberOfGroupOC: groupOfUniqueNames
olcMemberOfMemberAD: uniqueMember
olcMemberOfMemberOfAD: memberOf
structuralObjectClass: olcMemberOf
entryUUID: b0a0abdd-77ef-47f6-a1e1-52637e30ebcc
creatorsName: cn=config
createTimestamp: 20090302125140Z
entryCSN: 20090302125140.683800Z#000000#001#000000
modifiersName: cn=config
modifyTimestamp: 20090302125140Z

----------------
slapd.d/cn=config/olcDatabase={2}hdb/olcOverlay={2}refint.ldif
----------------
dn: olcOverlay={2}refint
objectClass: olcConfig
objectClass: olcOverlayConfig
objectClass: olcRefintConfig
olcOverlay: {2}refint
olcRefintAttribute: uniqueMember
olcRefintNothing: cn=Manager,dc=htam,dc=net
structuralObjectClass: olcRefintConfig
entryUUID: 13d0a0a0-8284-447c-9d49-426e37692f57
creatorsName: cn=config
createTimestamp: 20090302125140Z
entryCSN: 20090302125140.685440Z#000000#001#000000
modifiersName: cn=config
modifyTimestamp: 20090302125140Z

----------------
slapd.d/cn=config/cn=module{0}.ldif
----------------
dn: cn=module{0}
objectClass: olcModuleList
cn: module{0}
olcModulePath: /usr/local/libexec/openldap
olcModuleLoad: {0}memberof.la
olcModuleLoad: {1}ppolicy.la
olcModuleLoad: {2}refint.la
olcModuleLoad: {3}retcode.la
olcModuleLoad: {4}rwm.la
olcModuleLoad: {5}syncprov.la
olcModuleLoad: {6}unique.la
olcModuleLoad: {7}back_monitor.la
olcModuleLoad: {8}back_hdb.la
olcModuleLoad: {9}back_relay.la
structuralObjectClass: olcModuleList
entryUUID: 353f4a38-3a12-446f-9176-570021c59341
creatorsName: cn=config
createTimestamp: 20090224192423Z
entryCSN: 20090224192423.202231Z#000000#001#000000
modifiersName: cn=config
modifyTimestamp: 20090224192423Z

----------------
slapd.d/cn=config/olcDatabase={2}hdb.ldif
----------------
dn: olcDatabase={2}hdb
objectClass: olcDatabaseConfig
objectClass: olcHdbConfig
olcDatabase: {2}hdb
olcDbDirectory: /usr/local/var/openldap-data/
olcSuffix: dc=htam,dc=net
olcAccess: {0}to attrs=userPassword by
group/groupOfUniqueNames/uniqueMember="
  cn=ldapadmins,ou=groups,dc=htam,dc=net" write by
dn.subtree="ou=replicators,d
  c=htam,dc=net" read by dn.subtree="ou=computers,dc=htam,dc=net" auth by
self
  =xwd by anonymous auth
olcAccess: {1}to
attrs=entry,objectClass,uid,uidNumber,gidNumber,loginShell,cn
  ,gecos,description,homeDirectory by
group/groupOfUniqueNames/uniqueMember="cn
  =ldapadmins,ou=groups,dc=htam,dc=net" write by
dn.subtree="ou=replicators,dc=
  htam,dc=net" read by dn.subtree="ou=computers,dc=htam,dc=net" read by self
re
  ad
olcAccess::
ezJ9dG8gYXR0cnM9dW5pcXVlTWVtYmVyIGJ5IGdyb3VwL2dyb3VwT2ZVbmlxdWVOYW
  1lcy91bmlxdWVNZW1iZXI9ImNuPWxkYXBhZG1pbnMsb3U9Z3JvdXBzLGRjPWh0YW0sZGM9bmV0IiB
  3cml0ZSBieSBkbi5zdWJ0cmVlPSJvdT1yZXBsaWNhdG9ycyxkYz1odGFtLGRjPW5ldCIgcmVhZCBi
  eSBkbi5zdWJ0cmVlPSJvdT1jb21wdXRlcnMsZGM9aHRhbSxkYz1uZXQiIHJlYWQg
olcAccess: {3}to * by
group/groupOfUniqueNames/uniqueMember="cn=ldapadmins,ou=
  groups,dc=htam,dc=net" write by dn.subtree="ou=replicators,dc=htam,dc=net"
re
  ad by self read
olcLastMod: TRUE
olcMaxDerefDepth: 15
olcReadOnly: FALSE
olcRootDN: cn=Manager,dc=htam,dc=net
olcRootPW:: c2VjcmV0
olcMonitoring: TRUE
olcDbCacheSize: 1000
olcDbConfig: {0}set_cachesize 0 268435456 1
olcDbConfig: {1}set_lg_regionmax 262144
olcDbConfig: {2}set_lg_bsize 2097152
olcDbConfig: {3}set_flags DB_LOG_AUTOREMOVE
olcDbIndex: objectClass eq
olcDbIndex: uid pres,eq
olcDbIndex: cn pres,eq,sub
olcDbIndex: sn pres,eq,sub
olcDbIndex: givenname pres,eq,sub
olcDbIndex: uniqueMember pres,eq
olcDbIndex: memberUid pres,eq
olcDbIndex: entryCSN eq
olcDbIndex: entryUUID eq
olcDbIndex: ipServicePort eq
olcDbIndex: ipServiceProtocol eq
olcDbIndex: oncRpcNumber eq
olcDbIndex: ipProtocolNumber eq
structuralObjectClass: olcHdbConfig
entryUUID: 9f1eb1ca-a001-46db-aa58-4fc7897c64cc
creatorsName: cn=config
createTimestamp: 20090302125140Z
entryCSN: 20090302125140.183122Z#000000#001#000000
modifiersName: cn=config
modifyTimestamp: 20090302125140Z

----------------
slapd.d/cn=config/olcDatabase={1}monitor.ldif
----------------
dn: olcDatabase={1}monitor
objectClass: olcDatabaseConfig
olcDatabase: {1}monitor
olcAccess: {0}to *  by dn.base="cn=Manager,dc=htam,dc=net" read  by * none
olcLastMod: TRUE
olcMaxDerefDepth: 15
olcReadOnly: FALSE
olcMonitoring: FALSE
structuralObjectClass: olcDatabaseConfig
entryUUID: 6d366d19-e3ce-417b-a0b6-fd41bc690d83
creatorsName: cn=config
createTimestamp: 20090302125140Z
entryCSN: 20090302125140.118423Z#000000#001#000000
modifiersName: cn=config
modifyTimestamp: 20090302125140Z

----------------
slapd.d/cn=config.ldif
----------------
dn: cn=config
objectClass: olcGlobal
cn: config
olcConfigFile: slapd.conf.start
olcConfigDir: slapd.d.start
olcArgsFile: /usr/local/var/run/slapd.args
olcAttributeOptions: lang-
olcAuthzPolicy: none
olcConcurrency: 0
olcConnMaxPending: 100
olcConnMaxPendingAuth: 1000
olcGentleHUP: FALSE
olcIdleTimeout: 0
olcIndexSubstrIfMaxLen: 4
olcIndexSubstrIfMinLen: 2
olcIndexSubstrAnyLen: 4
olcIndexSubstrAnyStep: 2
olcIndexIntLen: 4
olcLocalSSF: 71
olcLogLevel: Packets
olcLogLevel: Config
olcLogLevel: Stats
olcLogLevel: Sync
olcPidFile: /usr/local/var/run/slapd.pid
olcReadOnly: FALSE
olcSaslSecProps: noplain,noanonymous
olcServerID: 1 ldap://vmlinux01/
olcSockbufMaxIncoming: 262143
olcSockbufMaxIncomingAuth: 16777215
olcThreads: 16
olcTLSCACertificateFile: /usr/local/etc/openldap/cacerts/cacert.pem
olcTLSCertificateFile: /usr/local/etc/openldap/slapd.cert.pem
olcTLSCertificateKeyFile: /usr/local/etc/openldap/slapd.key.pem
olcTLSCipherSuite: HIGH:MEDIUM
olcTLSCRLCheck: none
olcTLSVerifyClient: try
olcToolThreads: 1
structuralObjectClass: olcGlobal
entryUUID: 67b85bb6-58a2-4c6e-abd5-2bf7ce077d69
creatorsName: cn=config
createTimestamp: 20090224192423Z
entryCSN: 20090302142216.165509Z#000000#001#000000
modifiersName: cn=config
modifyTimestamp: 20090302142216Z


******************* LDIF for activating syncrepl on cn=config
**************
dn: cn=config
changetype: modify
replace: olcServerID
olcServerID: 1 "ldap://vmlinux01";
olcServerID: 2 "ldap://vmlinux02";
-
add: olcAuthzRegexp
olcAuthzRegexp: "cn=.*_repl_config,o=Htam.net Inc.,c=fr" "cn=config"

dn: olcOverlay=syncprov,olcDatabase={0}config,cn=config
changetype: add
objectClass: olcOverlayConfig
objectClass: olcSyncProvConfig
olcOverlay: syncprov

dn: olcDatabase={0}config,cn=config
changetype: modify
replace: olcSyncRepl
olcSyncRepl: rid=001
  provider="ldap://vmlinux01";
  bindmethod=sasl
  saslmech="EXTERNAL"
  searchbase="cn=config"
  type=refreshAndPersist
  starttls=critical
  retry="5 5 60 +"
  timeout=1
  tls_cacert=/usr/local/etc/openldap/cacerts/cacert.pem
  tls_cert=/usr/local/etc/openldap/slapd_repl_config.cert.pem
  tls_key=/usr/local/etc/openldap/slapd_repl_config.key.pem
olcSyncRepl: rid=002
  provider="ldap://vmlinux02";
  bindmethod=sasl
  saslmech="EXTERNAL"
  searchbase="cn=config"
  type=refreshAndPersist
  starttls=critical
  retry="5 5 60 +"
  timeout=1
  tls_cacert=/usr/local/etc/openldap/cacerts/cacert.pem
  tls_cert=/usr/local/etc/openldap/slapd_repl_config.cert.pem
  tls_key=/usr/local/etc/openldap/slapd_repl_config.key.pem
-
add: olcMirrorMode
olcMirrorMode: TRUE
-
add: olcLimits
olcLimits: dn="cn=config" size=unlimited time=unlimited

******************* LDIF for activating syncrepl on data backend
**************
dn: cn=config
changetype: modify
add: olcAuthzRegexp
olcAuthzRegexp: "cn=.*_replicator,o=Htam.net Inc.,c=FR"
cn=Replicator,ou=replicators,dc=htam,dc=net

dn: olcOverlay=syncprov,olcDatabase={2}hdb,cn=config
changetype: add
objectClass: olcOverlayConfig
objectClass: olcSyncProvConfig
olcOverlay: syncprov

dn: olcDatabase={2}hdb,cn=config
changetype: modify
add: olcLimits
olcLimits: dn.subtree="ou=replicators,dc=htam,dc=net" size=unlimited
time=unlimited
-
add: olcSyncRepl
olcSyncRepl: rid=201
  provider="ldap://vmlinux01";
  bindmethod=sasl
  saslmech="EXTERNAL"
  searchbase="dc=htam,dc=net"
  type=refreshOnly
  interval=00:00:00:10
  retry="5 5 300 +"
  timeout=1
  starttls=critical
  tls_cacert=/usr/local/etc/openldap/cacerts/cacert.pem
  tls_cert=/usr/local/etc/openldap/slapd_replicator.cert
  tls_key=/usr/local/etc/openldap/slapd_replicator.key
olcSyncRepl: rid=202
  provider="ldap://vmlinux02";
  bindmethod=sasl
  saslmech="EXTERNAL"
  searchbase="dc=htam,dc=net"
  type=refreshOnly
  interval=00:00:00:10
  retry="5 5 300 +"
  timeout=1
  starttls=critical
  tls_cacert=/usr/local/etc/openldap/cacerts/cacert.pem
  tls_cert=/usr/local/etc/openldap/slapd_replicator.cert
  tls_key=/usr/local/etc/openldap/slapd_replicator.key
-
add: olcMirrorMode
olcMirrorMode: TRUE


--
Mathieu MILLET
mailto:ldap@htam.net
----



--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/