RE: ITS#2183 IDL stack slab

[Jonghyuk Choi <jongchoi@us.ibm.com>]

If most of the searches are covered by the small setting slab cache,
more dynamic approach (such as realloc) seems fine with the big slab cache.
How about having both the per-thread small slab cache and the global big
slab cache pool ?
It seems desirable to have a memory size limit imposed to the big slab
cache,
because of the security concerns we may have otherwise.
Given the size limit, it looks better to manage the big slabs in the global
slab pool
having the maximum memory size than to manage one big slab per thread.
- Jong

------------------------
Jong Hyuk Choi
IBM Thomas J. Watson Research Center - Enterprise Linux Group
P. O. Box 218, Yorktown Heights, NY 10598
email: jongchoi@us.ibm.com
(phone) 914-945-3979    (fax) 914-945-4425   TL: 862-3979


"Howard Chu" <hyc@highlandsun.com> on 12/10/2002 04:10:31 PM

To:    Jonghyuk Choi/Watson/IBM@IBMUS, <openldap-devel@openldap.org>
cc:
Subject:    RE: ITS#2183 IDL stack slab



The code now in CVS makes the size configurable. Having a second setting
for
a big slab might be good. I'm not sure I'd worry about expiring a big slab;
if it's ever going to be used again you might as well keep it allocated.
Assuming you tune the "small" setting to cover 90-some percent of your
queries, you may get by with a single big slab that is grown/realloc'd as
needed.

The default size of 16 is probably overkill, someone whose system is tight
on
VM will need to set it lower...

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support

> -----Original Message-----
> From: Jonghyuk Choi [mailto:jongchoi@us.ibm.com]
>
> Thanks for letting the IDL stack slab cache design in.
>
> One isssue is on the right sizing of the IDL stack.
> If it is set to too small, searches having moderately complex
> search filters may not benefit from the slab cache.
> On the other hand, the memory overhead for large slabs may be
> quite big.
> (With SRCH_STACK_SIZE of 16, each thread would consume 8MB
> for the slab.)
>
> One approach I have been thinking of is to have two slab sizes :
> one small and the other relatively big.
> By having a TTL value to the big one and frees them after the TTL,
> the amount of memory consumption can be maintained within a bound
> while still providing preallocated memory to common searches
> having simple
> search filters.
> The size values and TTL values should also be configure by
> the cofiguration
> file.
>
> - Jong
>
> ------------------------
> Jong Hyuk Choi
> IBM Thomas J. Watson Research Center - Enterprise Linux Group
> P. O. Box 218, Yorktown Heights, NY 10598
> email: jongchoi@us.ibm.com
> (phone) 914-945-3979    (fax) 914-945-4425   TL: 862-3979
>
> ----- Original Message -----
> From: <hyc@highlandsun.com>
> To: <openldap-its@OpenLDAP.org>
> Sent: Tuesday, December 10, 2002 11:11 AM
> Subject: ITS#2183 IDL stack slab
>
>
> > Thinking about this some more; it can be greatly
> simplified. Instead of
> > having a sequence of buckets of increasing length, just use a single
> > fixed-size buffer per thread. This would also avoid the need for any
> locking
> > on the structures. (See the use of
> ldap_pvt_thread_pool_getkey/setkey in
> > cache.c.) I can code this up pretty quickly for a test.
> >
> >   -- Howard Chu
> >   Chief Architect, Symas Corp.       Director, Highland Sun
> >   http://www.symas.com              ; http://highlandsun.com/hyc
> >   Symas: Premier OpenSource Development and Support
> >
> >
> >
> >
>