FW: ITS#2588 subinitial and subfinal indexes not working

["Howard Chu" <hyc@symas.com>]

I thought about patching this so that for long subinitial and subfinal
strings we did a regular susbtring hash on the remainder of the string. I
think this would help in this case, but it would be the equivalent of just
using e.g. "head*verylongtail*" in the filter instead of
"headverylongtail*" - is it worth the trouble? (I had a patch implemented,
but then I dropped it 'cause it seemed a bit redundant. But it might make
things more convenient; certainly it can't produce more false positives than
the current situation.)

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support

-----Original Message-----
From: Howard Chu [mailto:hyc@highlandsun.com]

>slapd.conf, index I'm using:
>index   cn,sn,mail      eq,subinitial,subany,subfinal

>I also tried with just "eq,sub", same results.

>searches being performed: string, *string*, *string and string*.
>"string" and "*string*" are very fast (<1s), while "*string" and "string*"
>are very slow (>10s).

The subinitial (and subfinal) index code only helps if the indexed strings
are unique within their first (or last) 4 characters. For the data you're
using, this condition doesn't apply; the majority of your entries all fall
into the same index slot and so they must be checked individually.

This same substring indexing logic has been in place for a long time, which
is why the same behavior is present in 2.0, 2.1, and probably even 2.2, in
all the  backend types that support indexing (back-ldbm, back-bdb, back-hdb).

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support