[Date Prev][Date Next] [Chronological] [Thread] [Top]
RE: openldap-2.2alpha - abysmal performance with hdb

To: "'Quanah Gibson-Mount'" <quanah@stanford.edu>, <openldap-software@OpenLDAP.org>
Subject: RE: openldap-2.2alpha - abysmal performance with hdb
From: "Howard Chu" <hyc@highlandsun.com>
Date: Fri, 4 Jul 2003 23:16:34 -0700
Importance: Normal
In-reply-to: <148739726.1057324849@cadabra-dsl.stanford.edu>
I was afraid of that. Try adding
	idlcachesize 10000
(or some large number) to the hdb configuration in slapd.conf and then try
the search twice. The 2nd one should be reasonable.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support

> -----Original Message-----
> From: owner-openldap-software@OpenLDAP.org
> [mailto:owner-openldap-software@OpenLDAP.org]On Behalf Of Quanah
> Gibson-Mount

> Testing 2.2 with the same dataset as I have running in 2.1, using hdb
> instead of bdb, I find I have abysmal performance running
> queries -- It
> takes 17+ seconds to return for a single search.  Same
> indices, etc, as
> before.  The search that is running is essentially:
>
> ldapsearch -h <host> suseassunetid=<identifier> sumaildrop
>
> Yes, suseassunetid is indexed.
>
>
> It starts this part of the connection, and then hangs:
>
> connection_get(11)
> connection_get(11): got connid=3
> connection_read(11): checking for input on id=3
> ber_get_next
> ber_get_next: tag 0x30 len 523 contents:
> ber_get_next
> ber_get_next on fd 11 failed errno=11 (Resource temporarily
> unavailable)
> do_bind
> ber_scanf fmt ({imt) ber:
> ber_scanf fmt ({m) ber:
> ber_scanf fmt (m) ber:
> ber_scanf fmt (}}) ber:
> >>> dnPrettyNormal: <>
> <<< dnPrettyNormal: <>, <>
> do_sasl_bind: dn () mech GSSAPI
> ==> sasl_bind: dn="" mech=GSSAPI datalen=495
> send_ldap_sasl: err=14 len=98
> send_ldap_response: msgid=1 tag=97 err=14
> ber_flush: 114 bytes to sd 11
> <== slap_sasl_bind: rc=14
> connection_get(11)
> connection_get(11): got connid=3
> connection_read(11): checking for input on id=3
> ber_get_next
> ber_get_next: tag 0x30 len 20 contents:
> ber_get_next
> do_bind
> ber_get_next on fd 11 failed errno=11 (Resource temporarily
> unavailable)
> ber_scanf fmt ({imt) ber:
> ber_scanf fmt ({m) ber:
> ber_scanf fmt (}}) ber:
> >>> dnPrettyNormal: <>
> <<< dnPrettyNormal: <>, <>
> do_sasl_bind: dn () mech GSSAPI
> ==> sasl_bind: dn="" mech=<continuing> datalen=0
> send_ldap_sasl: err=14 len=53
> send_ldap_response: msgid=2 tag=97 err=14
> ber_flush: 69 bytes to sd 11
> <== slap_sasl_bind: rc=14
> connection_get(11)
> connection_get(11): got connid=3
> connection_read(11): checking for input on id=3
> ber_get_next
> ber_get_next: tag 0x30 len 75 contents:
> ber_get_next
> do_bind
> ber_get_next on fd 11 failed errno=11 (Resource temporarily
> unavailable)
> ber_scanf fmt ({imt) ber:
> ber_scanf fmt ({m) ber:
> ber_scanf fmt (m) ber:
> ber_scanf fmt (}}) ber:
> >>> dnPrettyNormal: <>
> <<< dnPrettyNormal: <>, <>
> do_sasl_bind: dn () mech GSSAPI
> ==> sasl_bind: dn="" mech=<continuing> datalen=53
> SASL Canonicalize [conn=3]: authcid="quanah"
> slap_sasl_getdn: id=quanah [len=6]
> getdn: u:id converted to uid=quanah,cn=stanford.edu,cn=GSSAPI,cn=auth
> >>> dnNormalize: <uid=quanah,cn=stanford.edu,cn=GSSAPI,cn=auth>
> => ldap_bv2dn(uid=quanah,cn=stanford.edu,cn=GSSAPI,cn=auth,0)
> <= ldap_bv2dn(uid=quanah,cn=stanford.edu,cn=GSSAPI,cn=auth,0)=0
> => ldap_dn2bv(272)
> <= ldap_dn2bv(uid=quanah,cn=stanford.edu,cn=gssapi,cn=auth,272)=0
> <<< dnNormalize: <uid=quanah,cn=stanford.edu,cn=gssapi,cn=auth>
> ==>slap_sasl2dn: converting SASL name
> uid=quanah,cn=stanford.edu,cn=gssapi,cn=auth to a DN
> slap_sasl_regexp: converting SASL name
> uid=quanah,cn=stanford.edu,cn=gssapi,cn=auth
> slap_sasl_regexp: converted SASL name to
> ldaps:///uid=quanah,cn=Accounts,dc=stanford,dc=edu??sub?suSeas
> Status=active
> slap_parseURI: parsing
> ldaps:///uid=quanah,cn=Accounts,dc=stanford,dc=edu??sub?suSeas
> Status=active
> ldap_url_parse_ext(ldaps:///uid=quanah,cn=Accounts,dc=stanford
> ,dc=edu??sub?
> suSeasStatus=active)
> put_filter: "suSeasStatus=active"
> put_filter: default
> put_simple_filter: "suSeasStatus=active"
> ber_scanf fmt ({mm}) ber:
> >>> dnNormalize: <uid=quanah,cn=Accounts,dc=stanford,dc=edu>
> => ldap_bv2dn(uid=quanah,cn=Accounts,dc=stanford,dc=edu,0)
> <= ldap_bv2dn(uid=quanah,cn=Accounts,dc=stanford,dc=edu,0)=0
> => ldap_dn2bv(272)
> <= ldap_dn2bv(uid=quanah,cn=accounts,dc=stanford,dc=edu,272)=0
> <<< dnNormalize: <uid=quanah,cn=accounts,dc=stanford,dc=edu>
> slap_sasl2dn: performing internal search
> (base=uid=quanah,cn=accounts,dc=stanford,dc=edu, scope=2)
> => bdb_back_search
> bdb_dn2entry("uid=quanah,cn=accounts,dc=stanford,dc=edu")
> search_candidates: base="uid=quanah,cn=accounts,dc=stanford,dc=edu"
> (0x0000d4fe) scope=2
> => hdb_dn2idl( "uid=quanah,cn=accounts,dc=stanford,dc=edu" )
> bdb_search_candidates: failed (rc=-30991)
> send_ldap_result: conn=3 op=0 p=3
> send_ldap_result: err=0 matched="" text=""
> <==slap_sasl2dn: Converted SASL name to
> uid=quanah,cn=accounts,dc=stanford,dc=edu
> getdn: dn:id converted to uid=quanah,cn=accounts,dc=stanford,dc=edu
> SASL Canonicalize [conn=3]:
> authcDN="uid=quanah,cn=accounts,dc=stanford,dc=edu"
> SASL Authorize [conn=3]: authcid="quanah@stanford.edu"
> authzid="quanah@stanford.edu"
> SASL Authorize [conn=3]:  authorization allowed
> send_ldap_sasl: err=0 len=-1
> send_ldap_response: msgid=3 tag=97 err=0
> ber_flush: 14 bytes to sd 11
> <== slap_sasl_bind: rc=0
> do_bind: SASL/GSSAPI bind:
> dn="uid=quanah,cn=accounts,dc=stanford,dc=edu"
> ssf=56
> connection_get(11)
> connection_get(11): got connid=3
> connection_read(11): checking for input on id=3
> ldap_pvt_sasl_install
> ber_get_next
> ber_get_next: tag 0x30 len 92 contents:
> ber_get_next
> do_search
> ber_scanf fmt ({miiiib) ber:
> ber_get_next on fd 11 failed errno=11 (Resource temporarily
> unavailable)
> >>> dnPrettyNormal: <cn=accounts,dc=stanford,dc=edu>
> => ldap_bv2dn(cn=accounts,dc=stanford,dc=edu,0)
> <= ldap_bv2dn(cn=accounts,dc=stanford,dc=edu,0)=0
> => ldap_dn2bv(272)
> <= ldap_dn2bv(cn=accounts,dc=stanford,dc=edu,272)=0
> => ldap_dn2bv(272)
> <= ldap_dn2bv(cn=accounts,dc=stanford,dc=edu,272)=0
> <<< dnPrettyNormal: <cn=accounts,dc=stanford,dc=edu>,
> <cn=accounts,dc=stanford,dc=edu>
> SRCH "cn=accounts,dc=stanford,dc=edu" 2 0    0 0 -1
> ber_scanf fmt ({mm}) ber:
>     filter: (suSeasSunetID=quanah)
> ber_scanf fmt ({M}}) ber:
>     attrs: sumaildrop
> => bdb_back_search
> bdb_dn2entry("cn=accounts,dc=stanford,dc=edu")
> search_candidates: base="cn=accounts,dc=stanford,dc=edu" (0x00000024)
> scope=2
> => hdb_dn2idl( "cn=accounts,dc=stanford,dc=edu" )
>
>
> In this part, it hangs for 15-17 seconds... then continues:
>
> => bdb_equality_candidates (objectClass)
> => key_read
> bdb_idl_fetch_key: [b49d1940]
> <= bdb_index_read: failed (-30991)
> <= bdb_equality_candidates: id=0, first=0, last=0
> => bdb_equality_candidates (suSeasSunetID)
> => key_read
> bdb_idl_fetch_key: [5d71ce6b]
> <= bdb_index_read 1 candidates
> <= bdb_equality_candidates: id=1, first=70231, last=70231
> bdb_search_candidates: id=1 first=70231 last=70231
> entry_decode: ""
> <= entry_decode()
> => bdb_entry_get: ndn:
> "cn=supervisor,cn=applications,dc=stanford,dc=edu"
> => bdb_entry_get: oc: "groupOfNames", at: "member"
> bdb_dn2entry("cn=supervisor,cn=applications,dc=stanford,dc=edu")
> bdb_entry_get: rc=0
> dnMatch 1
>         "uid=chandau,cn=accounts,dc=stanford,dc=edu"
>         "uid=quanah,cn=accounts,dc=stanford,dc=edu"
> dnMatch 0
>         "uid=quanah,cn=accounts,dc=stanford,dc=edu"
>         "uid=quanah,cn=accounts,dc=stanford,dc=edu"
> => send_search_entry: dn="uid=quanah,cn=Accounts,dc=stanford,dc=edu"
> (attrsOnly)
> ber_flush: 69 bytes to sd 11
> <= send_search_entry
> send_ldap_result: conn=3 op=3 p=3
> send_ldap_result: err=0 matched="" text=""
> send_ldap_response: msgid=4 tag=101 err=0
> ber_flush: 14 bytes to sd 11
> connection_get(11)
> connection_get(11): got connid=3
> connection_read(11): checking for input on id=3
> ber_get_next
> ber_get_next on fd 11 failed errno=0 (Error 0)
> connection_read(11): input error=-2 id=3, closing.
> connection_closing: readying conn=3 sd=11 for close
> connection_close: conn=3 sd=11
>
>
>
> --
> Quanah Gibson-Mount
> Senior Systems Administrator
> ITSS/TSS/Computing Systems
> Stanford University
> GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
>
>
Follow-Ups:
- RE: openldap-2.2alpha - abysmal performance with hdb
  - From: Quanah Gibson-Mount <quanah@stanford.edu>
References:
- openldap-2.2alpha - abysmal performance with hdb
  - From: Quanah Gibson-Mount <quanah@stanford.edu>
Prev by Date: openldap-2.1.22 core dumped on create new entries with UTF/base64 dn
Next by Date: Re: rc file for openldap command line commands
Index(es):
- Chronological
- Thread