[Date Prev][Date Next] [Chronological] [Thread] [Top]

address look-up performance and a filterindex.c patch



The DirectoryMark address look-up scenario mimics a situation where
multiple
clients ask general address look-up queries. Each client thread sends a
sequence of
requests of the following mix :

- exact match on emailAddress :     28%
- CN wildcard search :        24%
- exact match on given name :       16%
- exact match on SN :          8%
- exact match on CN :         16%
- not found :                  8%
- bind :    as anonymous, every 5 requests

In the experimental run, the throughput of back-bdb happened to be
siginificanlty lower than that of back-ldbm.

# of Clients : 8, # of Threads / Clients : 2
Run Time : 5 minutes

DirectoryMark Results
- back-ldbm
      Number of Transactions  : 170,812
      Min Time (ms)           :       0
      Max Time (ms)           :   8,607
      Average Time (ms) :      27
      Standard Dev. (ms)      :      14
      DirectoryMark           :     561.9 operations / second
- back-bdb
      Number of Transactions  :     511
      Min Time (ms)           :       1
      Max Time (ms)           : 153,428
      Average Time (ms) :  13,137
      Standard Dev. (ms)      :  13,207
      DirectoryMark           :       1.2 operations / second

When examining back-bdb traces, I found that search_candidates() returns
all entries when searching for non-existent entries.
Because slapd should loop through all entries to vainly find matching
entries,
8% of the searches seem to result in the drastic performance degradation
as shown above.

The following is a possible patch to make ids empty in such cases (please
examine) :

*** filterindex.c.org         Wed Jan  2 17:14:24 2002
--- filterindex.c.jhchoi      Wed Jan  2 17:24:29 2002
*************** presence_candidates(
*** 229,239 ****
            return 0;
      }

      rc = bdb_key_read( be, db, NULL, &prefix, ids );

      if( rc == DB_NOTFOUND ) {
            rc = 0;
-
      } else if( rc != LDAP_SUCCESS ) {
            Debug( LDAP_DEBUG_TRACE,
                  "<= bdb_presense_candidates: key read failed (%d)\n",
--- 235,247 ----
            return 0;
      }

+
      rc = bdb_key_read( be, db, NULL, &prefix, ids );

+
      if( rc == DB_NOTFOUND ) {
+           BDB_IDL_ZERO( ids );
            rc = 0;
      } else if( rc != LDAP_SUCCESS ) {
            Debug( LDAP_DEBUG_TRACE,
                  "<= bdb_presense_candidates: key read failed (%d)\n",
*************** equality_candidates(
*** 316,325 ****
            return 0;
      }

      for ( i= 0; keys[i].bv_val != NULL; i++ ) {
            rc = bdb_key_read( be, db, NULL, &keys[i], tmp );

!           if( rc != LDAP_SUCCESS ) {
                  Debug( LDAP_DEBUG_TRACE,
                        "<= bdb_equality_candidates key read failed (%d)
\n",
                        rc, 0, 0 );
--- 324,339 ----
            return 0;
      }

+
      for ( i= 0; keys[i].bv_val != NULL; i++ ) {
            rc = bdb_key_read( be, db, NULL, &keys[i], tmp );

!           if (rc == DB_NOTFOUND) {
!                 BDB_IDL_ZERO( ids );
!                 rc = 0;
!                 break;
!           }
!           else if( rc != LDAP_SUCCESS ) {
                  Debug( LDAP_DEBUG_TRACE,
                        "<= bdb_equality_candidates key read failed (%d)
\n",
                        rc, 0, 0 );
*************** approx_candidates(
*** 424,430 ****
      for ( i= 0; keys[i].bv_val != NULL; i++ ) {
            rc = bdb_key_read( be, db, NULL, &keys[i], tmp );

!           if( rc != LDAP_SUCCESS ) {
                  Debug( LDAP_DEBUG_TRACE, "<= bdb_approx_candidates key
read failed (%d)\n",
                        rc, 0, 0 );
                  break;
--- 440,450 ----
      for ( i= 0; keys[i].bv_val != NULL; i++ ) {
            rc = bdb_key_read( be, db, NULL, &keys[i], tmp );

!           if( rc == DB_NOTFOUND ) {
!                 BDB_IDL_ZERO( ids );
!                 rc = 0;
!                 break;
!           } else if( rc != LDAP_SUCCESS ) {
                  Debug( LDAP_DEBUG_TRACE, "<= bdb_approx_candidates key
read failed (%d)\n",
                        rc, 0, 0 );
                  break;
*************** substring_candidates(
*** 523,529 ****
      for ( i= 0; keys[i].bv_val != NULL; i++ ) {
            rc = bdb_key_read( be, db, NULL, &keys[i], tmp );

!           if( rc != LDAP_SUCCESS ) {
                  Debug( LDAP_DEBUG_TRACE, "<= bdb_substring_candidates key
read failed (%d)\n",
                        rc, 0, 0 );
                  break;
--- 543,553 ----
      for ( i= 0; keys[i].bv_val != NULL; i++ ) {
            rc = bdb_key_read( be, db, NULL, &keys[i], tmp );

!           if( rc == DB_NOTFOUND ) {
!                 BDB_IDL_ZERO( ids );
!                 rc = 0;
!                 break;
!           } else if( rc != LDAP_SUCCESS ) {
                  Debug( LDAP_DEBUG_TRACE, "<= bdb_substring_candidates key
read failed (%d)\n",
                        rc, 0, 0 );
                  break;



The DirectoryMark Results with the patch is :
-back-bdb
      Number of Transactions  :  57,298
      Min Time (ms)           :       0
      Max Time (ms)           :  13,985
      Average Time (ms) :      83
      Standard Dev. (ms)      :      46
      DirectoryMark           :     189.7 operations / second


------------------------
Jong Hyuk Choi
IBM Thomas J. Watson Research Center - Enterprise Linux Group
P. O. Box 218, Yorktown Heights, NY 10598
email: jongchoi@us.ibm.com
(phone) 914-945-3979    (fax) 914-945-4425   TL: 862-3979