Re: 2.3.19 and memory usage

[Howard Chu <hyc@symas.com>]

David Hawes wrote:
> On Wednesday 08 February 2006 18:45, Quanah Gibson-Mount wrote:
>   
> > On Wednesday 08 February 2006 13:41, David Hawes wrote:
> >     
> > > I have been testing OpenLDAP 2.3.19 to finally move from 2.2.26 on my
> > > production environment and have run into some issues with memory use.
> > > Basically, the settings I use for 2.2.26 do not translate to 2.3.19 (all
> > > tests were done on the same machine).  On my test, where I dump
> > > approximately 500k entries by uid, 2.3.19 uses all available system
> > > memory and then segfaults.  2.2.26 returns all entries without incident. 
> > > Tweaking the BDB cachesize and the entry cachesize appear to have some
> > > effect, but the end results are similar.
> > >       
> > I've never found BDB to work under 32 bit linux with a 2GB cachesize.  Set
> > it to 1.9GB.
> >     
>
> I should have been more specific when I wrote 2GB (I rounded up), as I was 
> using 1.9GB all along.  That said, 1.9GB substituted for 2GB in my first 
> email still yields the same results.
>   

I've reproduced this behavior, it's due to the same reason as ITS#4385. 
Now fixed in HEAD back-bdb/cache.c. A potential workaround is to issue a 
few no-op requests to the slapd server while the large search operation 
is in progress. (Basically the task that purges excess entries from the 
entry cache isn't getting started right away; it won't start until the 
current search finishes or any new request is received.) You can use 
this patch:
Index: cache.c
===================================================================
RCS file: /repo/OpenLDAP/pkg/ldap/servers/slapd/back-bdb/cache.c,v
retrieving revision 1.116
retrieving revision 1.117
diff -u -r1.116 -r1.117
--- cache.c     7 Jan 2006 10:06:32 -0000       1.116
+++ cache.c     9 Feb 2006 05:53:49 -0000       1.117
@@ -659,6 +659,7 @@
                                       bdb->bi_dbenv_home );
                       }
                       ldap_pvt_thread_mutex_unlock( &slapd_rq.rq_mutex );
+                       slap_wake_listener();
               }
       }
}

--
 -- Howard Chu
 Chief Architect, Symas Corp.  http://www.symas.com
 Director, Highland Sun        http://highlandsun.com/hyc
 OpenLDAP Core Team            http://www.openldap.org/project/