RE: IDL management in idl.c

["Howard Chu" <hyc@highlandsun.com>]

> -----Original Message-----
> From: owner-openldap-devel@OpenLDAP.org
> [mailto:owner-openldap-devel@OpenLDAP.org]On Behalf Of Jonghyuk Choi

> ITS#1651 patch I posted yesterday tries to use alloca() first and then use
> malloc()
> if alloca() fails. We allocate BDB_IDL_DB_SIZE memory either in
> stack or heap, depending on its size (BDB_IDL_DB_SIZE).
> As a result, we can use large value of BDB_IDL_DB_SIZE without worrying
> about stack size limit.

The announcement for ITS#1651 just now showed up, after this email, so sorry
for any late responses.

> An improvement to the current patch would be
> to use multiple moderate size IDL buffers by managing overflows in the
> DB_MULTIPLE_NEXT loop.
> For the cases that IDLs are of moderate size, we can avoid using malloc().
> Only when we can't allocate from stack, then we pay for the price
> of malloc
> ().
> Because we don't know the amount of memory in advance as in
> Howard's recent
> filterindex.c patch,
> we don't seem to be able to resort to that solution in this idl.c case.
> Comments are welcome ...

alloca() is nice in general, but as I recall it's only reliable in GCC. If
we can rely on its behavior, then it would be good to use in the filterindex
patch.

I don't see any need to change the current behavior of the bdb_idl functions
since they don't nest recursively. At most, bdb_idl_insert_key() calls
bdb_idl_fetch_key() once to retrieve an existing key. So there are at most
two
BDB_IDL_DB_SIZE arrays on the stack. With the current sizes, this is never a
problem. If the objective here is to increase the sizes, that's another
story.
Otherwise, the code in filterindex.c is the only place that presents the
danger of stack blowout, which is why I focused on that.

On a sidenote, I see that in bdb_idl_delete_key we always assume that
deleting from a range is a no-op. Perhaps we should shrink the range if the
deleted item is either the lo or hi bound of the range? Hm, I guess there's
a bug here as well in the BDB_IDL_MULTI case, since it always uses a cursor
to delete the given item. If you delete the lo or hi bound of a range, the
range item will be "broken". Amazing the things you see after being away
from the code for a while.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support