RE: IDL management in idl.c

["Howard Chu" <hyc@highlandsun.com>]

> -----Original Message-----
> From: Kervin Pierre [mailto:kpierre@fit.edu]

> Howard Chu wrote:
> ...
> > alloca() is nice in general, but as I recall it's only reliable
> in GCC. If
> > we can rely on its behavior, then it would be good to use in
> the filterindex
> > patch.

> We might need a HAVE_ALLOCA macro and fallback to (a) the old way or (b)
> use malloc().

> as far as win32 it looks like alloca still throws an exception on
> failure.  Although using MS specific C exceptions, that can be handled,
> but leaves the issue of performance loss due to the use of exceptions
> (?) and additional compiler checks.

This sounds ugly... If there was a way to turn off the exception behavior
and just return NULL like a normal allocator, that'd be OK. Otherwise it
sounds like the MS alloca is doing enough work to negate much of the
performance gain.
>
> > I don't see any need to change the current behavior of the
> bdb_idl functions
> > since they don't nest recursively. At most, bdb_idl_insert_key() calls
> > bdb_idl_fetch_key() once to retrieve an existing key. So there
> are at most
> > two
> > BDB_IDL_DB_SIZE arrays on the stack. With the current sizes,
> this is never a
> > problem. If the objective here is to increase the sizes, that's another
> > story.
> > Otherwise, the code in filterindex.c is the only place that presents the
> > danger of stack blowout, which is why I focused on that.

>  From our last conversion I assumed the otherwise.

??? I don't understand the reference for this sentence.
>
>  From a security and memory optimization point of view, I'm concerned
> with OpenLDAP's use of the stack for large amounts dynamic data.  I
> think ( overall ) it limits the amount of memory optimizations that can
> be done.  This fear may very well be unfounded, but given the choice eg.
> ( maybe a configure macro ), I'd choose the to use the heap.
>
> Is the use of the stack soley due to speed optimization?  If so, how
> much are we gaining by staying on the stack?

Have a look thru the archives of this mailing list. The current releases of
slapd spend over 50% of their CPU time in the malloc library. The gain from
being more careful about allocations here is tremendous. I don't understand
your fear from an optimization point of view; there are no downsides to
using
the stack. From a security standpoint, we're dealing with items of bounded
size, so there is no risk of malicious overruns tweaking return addresses.
There is
of course the risk of a deeply nested recursion running out of stack, but it
would take an extremely complex request to encounter that. Very likely the
limit on the maximum size of an input buffer would prevent such a request
from being accepted in the first place.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support