[Date Prev][Date Next] [Chronological] [Thread] [Top]
Unstable slapd-meta connection with MS AD LDAP

To: openldap-technical@openldap.org
Subject: Unstable slapd-meta connection with MS AD LDAP
From: Nick Anzalone <nanzalone@achievementnetwork.org>
Date: Thu, 29 Aug 2013 10:48:59 -0400
Hello,

I'm trying to use slapd-meta (openldap version 2.4.28) as a proxy
between a web application LDAP authentication request and a MS AD
server. The idea being that if we have multiple LDAP authentication
clients we can add another target with suffix massage to redirect
requests to the appropriate LDAP server. We were forced to use a
dedicated proxy because we were told the MS server's firewall requires
an actual IP, not a FQDN, and we don't have a stable ip (we're using a
load balancer service which only provides a FQDN).

In general, it looked like this solution would work well. My problem
is that the connection between the slapd-meta proxy and the MS server
does not seem very robust. I would be extremely grateful if someone
could help me understand where I might start looking to diagnose what
is wrong.

####################
Problem Summary
#####################
The problem I'm seeing was reported as an "Operations Error" from the
command line. Initially, I did not have idle-timeout set at all, and
this error would occur reliably after 15 minutes of inactivity, and
was not recoverable (I needed to restart the slapd server to correct
the problem). I suspected that the cause was that the connection was
remaining idle too long and getting invalidated by the MS LDAP server
(which I don't have access to).

To address this problem, I set idle-timeout to 10m (MS AD apparently
has a default timeout of 15m). I have set up a cron job to "ping" the
MS server every minute in an attempt to keep the connection open
(although I'm not sure if it actually uses the same connection, so
this may not be doing much). I also set the value of the nretries
configuration directive to 20.

These adjustments seemed to help significantly. With constant activity
(authentication through the web application), everything appears to
work great. However I still occasionally get the "operations error,"
which causes my web application to reject the initial authentication
even with correct credentials. At this point it is most reliably
reproduced by inducing more than 15 minutes of inactivity on the
slapd-meta proxy, and then attempting a login. After an initial failed
login attempt, the problem seems to correct for itself (i.e. a second
login attempt through the web application with the same credentials
will succeed).

I am at all times able to connect directly to the LDAP server with no
trouble. It is only connections through the slapd-meta proxy I have
set up that (sometimes) fail. It may also be worth noting that I don't
see any of these problems with an openldap test server that I set up.
Thanks so much in advance for any help or suggestions.

##################################
Problem Details
#####################################
Below is an excerpt from my syslog from a failed authentication
request initiated by my web application:

-------------------- log output from syslog
------------------------------------------
Aug 28 20:33:26 my-proxy-server slapd[10330]: daemon: epoll: listen=8
active_threads=0 tvp=NULL
Aug 28 20:33:26 my-proxy-server slapd[10330]: daemon: epoll: listen=9
active_threads=0 tvp=NULL
Aug 28 20:33:26 my-proxy-server slapd[10330]: daemon: epoll: listen=10
active_threads=0 tvp=NULL
Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 do_search
Aug 28 20:33:26 my-proxy-server slapd[10330]: >>> dnPrettyNormal:
<dc=AAA,dc=YYY,dc=ZZZ>
Aug 28 20:33:26 my-proxy-server slapd[10330]: <<< dnPrettyNormal:
<dc=AAA,dc=YYY,dc=ZZZ>, <dc=AAA,dc=YYY,dc=ZZZ>
Aug 28 20:33:26 my-proxy-server slapd[10330]: SRCH "dc=AAA,dc=YYY,dc=ZZZ" 2 3
Aug 28 20:33:26 my-proxy-server slapd[10330]:     0 0 0
Aug 28 20:33:26 my-proxy-server slapd[10330]: begin get_filter
Aug 28 20:33:26 my-proxy-server slapd[10330]: EQUALITY
Aug 28 20:33:26 my-proxy-server slapd[10330]: end get_filter 0
Aug 28 20:33:26 my-proxy-server slapd[10330]:     filter: (?SAMACCOUNTNAME=xxx)
Aug 28 20:33:26 my-proxy-server slapd[10330]: => get_ctrls
Aug 28 20:33:26 my-proxy-server slapd[10330]: => get_ctrls:
oid="2.16.840.1.113730.3.4.2" (noncritical)
Aug 28 20:33:26 my-proxy-server slapd[10330]: <= get_ctrls: n=1 rc=0 err=""
Aug 28 20:33:26 my-proxy-server slapd[10330]:     attrs:
Aug 28 20:33:26 my-proxy-server slapd[10330]:
Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 SRCH
base="dc=AAA,dc=YYY,dc=ZZZ" scope=2 deref=3
filter="(?SAMACCOUNTNAME=xxx)"
Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1
meta_back_getconn: mc=0x7fafbc004500 conn=ROOTDN expired (tainted).
Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1:
meta_back_getconn[1]
Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1
meta_back_getconn: candidates=1 conn=ROOTDN fetched
Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 >>>
meta_back_search_start[1]
Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 >>>
meta_search_dobind_init[1]
Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 <<<
meta_search_dobind_init[1]=1
Aug 28 20:33:26 my-proxy-server slapd[10330]: ==>
rewrite_context_apply [depth=1] string='dc=AAA,dc=YYY,dc=ZZZ'
Aug 28 20:33:26 my-proxy-server slapd[10330]: ==> rewrite_rule_apply
rule='((.+),)?dc=AAA,[ ]?dc=YYY,[ ]?dc=ZZZ$'
string='dc=AAA,dc=YYY,dc=ZZZ' [1 pass(es)]
Aug 28 20:33:26 my-proxy-server slapd[10330]: ==>
rewrite_context_apply [depth=1] res={0,'ou=Org
Unit,dc=AAA,dc=BBB,dc=CCC,dc=DDD'}
Aug 28 20:33:26 my-proxy-server slapd[10330]: [rw] searchBase:
"dc=AAA,dc=YYY,dc=ZZZ" -> "ou=Org Unit,dc=AAA,dc=BBB,dc=CCC,dc=DDD"
Aug 28 20:33:26 my-proxy-server slapd[10330]: ==>
rewrite_context_apply [depth=1] string='(SAMACCOUNTNAME=xxx)'
Aug 28 20:33:26 my-proxy-server slapd[10330]: ==>
rewrite_context_apply [depth=1] res={0,'NULL'}
Aug 28 20:33:26 my-proxy-server slapd[10330]: [rw] searchFilter:
"(SAMACCOUNTNAME=xxx)" -> "(SAMACCOUNTNAME=xxx)"
Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1
meta_back_retry[1]: retrying
URI="ldap://ms-ldap-server.AAA.BBB.CCC.DDD"; DN="cn=bind-manager,ou=Org
Unit,dc=AAA,dc=BBB,dc=CCC,dc=DDD".
Aug 28 20:33:26 my-proxy-server slapd[10330]: >>> conn=1094 op=1
meta_back_bind_op_result[1]
Aug 28 20:33:26 my-proxy-server slapd[10330]: <<< conn=1094 op=1
meta_back_bind_op_result[1] err=0
Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1
meta_back_retry[1]: meta_back_single_dobind=0
Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 <<<
meta_back_search_start[1]=1
Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1
meta_back_search: ncandidates=1 cnd=" *"
Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1
meta_back_search[1] match="" err=1 (Operations error).
Aug 28 20:33:26 my-proxy-server slapd[10330]: send_ldap_result:
conn=1094 op=1 p=3
Aug 28 20:33:26 my-proxy-server slapd[10330]: send_ldap_result: err=1
matched="" text=""
Aug 28 20:33:26 my-proxy-server slapd[10330]: send_ldap_response:
msgid=2 tag=101 err=1
Aug 28 20:33:26 my-proxy-server slapd[10330]: conn=1094 op=1 SEARCH
RESULT tag=101 err=1 nentries=0 text=


#########################
slapd.conf
########################
# This is the main slapd configuration file. See slapd.conf(5) for more
# info on the configuration options.

#######################################################################
# Global Directives:

# Features to permit
#allow bind_v2

# Schema and objectClass definitions
include         /etc/ldap/schema/core.schema
include         /etc/ldap/schema/cosine.schema
include         /etc/ldap/schema/nis.schema
include         /etc/ldap/schema/inetorgperson.schema

# Where the pid file is put. The init.d script
# will not stop the server if you change this.
pidfile         /var/run/slapd/slapd.pid

# List of arguments that were passed to the server
argsfile        /var/run/slapd/slapd.args

# Read slapd.conf(5) for possible values
# should reduce this once we're live
loglevel        any

# Where the dynamically loaded modules are stored
modulepath      /usr/lib/ldap
moduleload back_meta.la
moduleload back_ldap.la

# The maximum number of entries that is returned for a search operation
sizelimit 500

# The tool-threads parameter sets the actual amount of cpu's that is used
# for indexing.
tool-threads 1

# experiment trying to figure out error below:
# ber_get_next on fd 12 failed errno=34 (Numerical result out of range)
#sockbuf_max_incoming_auth 5000000000

# Specific Backend Directives for 'other':
# Backend specific directives apply to this backend until another
# 'backend' directive occurs
backend meta

# this is broken - causes slapd to terminate
#TLSCipherSuite ALL:!NULL
#TLSCipherSuite ECDHE-RSA-AES256-SHA
#TLSCipherSuite SECURE256

# TLS Security settings
TLSCACertificateFile /etc/ldap/cert/private/ca.my-proxy-server.YYY.ZZZ.pem
TLSCertificateFile /etc/ldap/cert/private/ca.my-proxy-server.YYY.ZZZ.pem
TLSCertificateKeyFile /etc/ldap/cert/private/private.my-proxy-server.YYY.ZZZ.pem
#TLSVerifyClient try
# require TLS (secure) connection
#security tls=1

#########################################################
# CUSTOM META SLAPD
#################################################

database        meta
suffix          "dc=YYY,dc=ZZZ"
rootdn          "cn=XXX,dc=YYY,dc=ZZZ"
rootpw          *****

# The following parameters were added to trouble-shoot the unstable
connection w/ MS LDAP server
idle-timeout    10m
nretries           20

#######################################################################
# Test LDAP server on staging
# Each LDAP server will get its own entry like this
# Please document each server
#######################################################

# The LDAP server URL. Note the dc=test in the suffix, which allows us
to uniquely identify the request as
# one that should be passed to our test server. We might use one like
dc=AAA for an entry that would pass
# requests to the AAA LDAP server
uri           "ldap://test.YYY.ZZZ/dc=test,dc=YYY,dc=ZZZ";

# Tells the client to use a secure connection with the server.
tls           start

# More connection specifics: specifies the filename/location of the
client certificate if the server requires one
# we also specify the DN used to bind, and credentials (password) for
the binding DN.
idassert-bind tls_cacert=/etc/ldap/cert/test.YYY.ZZZ.crt
binddn="cn=admin,dc=YYY,dc=ZZZ" credentials="*****"
# NB: There is a parameter that needs to "allow" the server to require
a client certificate (tls_reqcert=allow), however it doesn't seem to
# work. So we were forced to set this in the ldap.conf (client-side)
config file. slapd uses this file to set default values for many
# parameters, including this one.

# This removes the "dc=test" specifier before delegating the LDAP
request to the test server.
suffixmassage "dc=test,dc=YYY,dc=ZZZ" "dc=YYY,dc=ZZZ"

##########################################################################
# AAA LDAP Server Entry
# Here is the AAA-provided "user context":
# OU=Org Unit, DC=AAA, DC=BBB, DC=CCC, DC=DDD
#
##########################################################

uri           "ldap://ms-ldap-server.AAA.BBB.CCC.DDD/dc=AAA,dc=YYY,dc=ZZZ";
tls           start
idassert-bind tls_cacert=/etc/ldap/cert/AAA.BBB.CCC.DDD.crt
binddn="cn=bind-manager,ou=Org Unit,dc=AAA,dc=BBB,dc=CCC,dc=DDD"
credentials="******"
suffixmassage "dc=AAA,dc=YYY,dc=ZZZ" "ou=Org Unit,dc=AAA,dc=BBB,dc=CCC,dc=DDD"
default-target
Follow-Ups:
- Re: Unstable slapd-meta connection with MS AD LDAP
  - From: Quanah Gibson-Mount <quanah@zimbra.com>
Prev by Date: Re: Antw: Re: Object not found
Next by Date: RE: ldapadd "ldap_bind: Invalid credentials (49)"
Index(es):
- Chronological
- Thread