[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: slapd hang once a day (fwd)



Hi,

We're using ldap for resolving email address through sendmail (8.10.2).  The 
traffic volume per day is around 2000+ mail.  We've another replicated ldap 
server with the same configuration (OpenLDAP 1.2.9 + Solaris 2.6) used for 
updated and user query, which doesn't have the problem.

I wonder if sendmail be another "brain-damaged" client.  

Would anyone please help?

Thanks a lot!
Regards,
ST Wong

> I wonder if it is coincidence that both of you are reporting a similar
> problem (auth services cease after a while) and that both of you are
> using ldap to authenticate squid.  Maybe I've way off base tying the
> two together.  It is quite likely that one of your clients (perhaps
> a client the two of you have in common) is "brain-damaged", as Kurt
> suggested, and is failing to close its connection once the authentication
> is done.  The display below should be re-run every few hours of operation
> to see what grows.  Another possibility is to take applications out
> of service one at a time for an hour or so, while carefully monitor the
> data below to check for file descriptor leaks.  When the leak stops, you
> know you have the culprit.
> 
> For my own experience with OpenLDAP, I can assure you that there are no
> such leaks in OpenLDAP 1.2.x.
> 
> Randy
> 
> 
> > Quoting Randy Kunkee <kunkee@neosoft.com>:
> > 
> > Thanks, we've done as adviced and following is the result. How should we read 
> > the information below? Most of them are sockets rather than files. Your help 
> > in analysing the result is greatly appreciated.
> > 
> > [Fri Jul 14 00:52:45]root@aquarius:/proc# ls -l 20665/fd | less
> > lrwx------    1 root     root           64 Jul 14 00:52 0 -> /dev/null
> > l-wx------    1 root     root           64 Jul 14 00:52 1 -> 
> > pipe:[7406361]
> > lrwx------    1 root     root           64 Jul 14 00:52 10 -> 
> > /etc/openldap/user
> > db/dn2id.dbb
> > lrwx------    1 root     root           64 Jul 14 00:52 100 -> 
> > socket:[7413858]
> > lrwx------    1 root     root           64 Jul 14 00:52 101 -> 
> > socket:[7415379]
> > lrwx------    1 root     root           64 Jul 14 00:52 102 -> 
> > socket:[7416352]
> > lrwx------    1 root     root           64 Jul 14 00:52 103 -> 
> > socket:[7417422]
> > lrwx------    1 root     root           64 Jul 14 00:52 104 -> 
> > socket:[7419554]
> > lrwx------    1 root     root           64 Jul 14 00:52 105 -> 
> > socket:[7421151]
> > lrwx------    1 root     root           64 Jul 14 00:52 106 -> 
> > socket:[7420639]
> > lrwx------    1 root     root           64 Jul 14 00:52 107 -> 
> > socket:[7417866]
> > lrwx------    1 root     root           64 Jul 14 00:52 108 -> 
> > socket:[7421024]
> > lrwx------    1 root     root           64 Jul 14 00:52 109 -> 
> > socket:[7420488]
> > lrwx------    1 root     root           64 Jul 14 00:52 11 -> 
> > /etc/openldap/user
> > db/id2entry.dbb
> > lrwx------    1 root     root           64 Jul 14 00:52 110 -> 
> > socket:[7419017]
> > lrwx------    1 root     root           64 Jul 14 00:52 111 -> 
> > socket:[7421187]
> > lrwx------    1 root     root           64 Jul 14 00:52 112 -> 
> > socket:[7420445]
> > lrwx------    1 root     root           64 Jul 14 00:52 113 -> 
> > socket:[7417884]
> > lrwx------    1 root     root           64 Jul 14 00:52 114 -> 
> > socket:[7418489]
> > lrwx------    1 root     root           64 Jul 14 00:52 115 -> 
> > socket:[7420490]
> > 
> > 
> > > You may have a new bug.
> > >
> > > Most linux systems have a proc filesystem that you can go to as root
> > > to examine the open files a process has.  For example, if the pid of
> > > slapd is 100, you should be able to "ls -l /proc/100/fd", which will
> > > give you a listing of devices and inode numbers that are open by the
> > > process.
> > >
> > > You can then do a "find / -inum #" where # is the inode number of
> > > the file.  In the case of slapd, the number of places the file could
> > > be open is rather small, so you can drasticly reduce the find runs
> > > by narrowing down the directories, for example:
> > >
> > > find /etc /dev /(path to ldbm files) /(path to log files) -inum #
> > >
> > > Once you know what file is not getting closed, you'll know where to go
> > > to fix the problem.  Just because /etc/hosts.allow is the one you get
> > > the error on does not necessarily mean that it's the one not getting
> > > closed.
> > >
> > > > Unable to apply the patch as the link given in ITS is no longer valid.
> > > Can
> > > > someone pls advise if the patch is needed for OpenLDAP-1.2.9-5 and
> > > OpenLDAP
> > > > 1.2.11? If needed, where else can I find the patch or someone can email
> > > me
> > > > pls.
> > > >
> > > > Below is a detailed configuration of our server setup and hope it'll help
> > > to
> > > > diagnose the problem better.
> > > >
> > > > 1. Version of OpenLDAP
> > > > OpenLDAP-1.2.9-5
> > > >
> > > > 2. Operating System
> > > > RedHat Linux 6.2 with kernel 2.2.15
> > > >
> > > > 3. Hardware specifications
> > > > Intel Pentium III 550 MHz 512MB RAM
> > > >
> > > > 4. User population
> > > > Approximately 4500
> > > >
> > > > 5. Applications/services running on server
> > > > LDAP, BIND8, SSH, APACHE, SSL, PHP, SENDMAIL, IMAP, POP,
> > > > SQUID, MySQL
> > > >
> > > > 6. Applications used by users to access above mentioned services
> > > > MS Outlook/Outlook Express, pine, Netscape mail/navigator, IE
> > > >
> > > > 7. Error found in /var/log/messages as:
> > > >
> > > > Jul 12 12:40:05 server1 slapd[32521]: warning: cannot open
> > > > /etc/hosts.allow: Too many open files
> > > > Jul 12 12:40:05 server1 slapd[32521]: warning: cannot open
> > > /etc/hosts.deny:
> > > > Too many open files
> > > >
> > > > Quoting "S.T. Wong" <st@hp735c.csc.cuhk.edu.hk>:
> > > >
> > > > > Hello,
> > > > >
> > > > > We're running OpenLDAP 1.2.11 on Solaris 2.6, and have the same
> > > problem.
> > > > > Did the patches mentioned in the ITS cases mentioned incorporated in
> > > 1.2.11
> > > > > ?
> > > > >
> > > > > Thanks a lot.
> > > > > Regards,
> > > > > ST Wong
> > > > >
> > > > > > You likely are using brain damaged clients.
> > > > > >
> > > > > > See ITS#237, ITS#169, ITS#142, and ITS#30.
> > > > > >
> > > > > > Kurt
> > > > > >
> > > > > > At 05:09 PM 7/11/00 +0000, antidotez@catcha.com wrote:
> > > > > > >recently, ldap (slapd) has been giving problem. it will
> > > > > > >hang at least once every day and needs manual killing
> > > > > > >and restaring. below is an entry from log file:
> > > > > > >"Jul 10 09:21:55 testserver slapd[19503]: warning:
> > > > > > >cannot open /etc/hosts.allow: Toomany open files"
> > > > > > >
> > > > > > >would appreciate if someone could advise on the
> > > > > > >cause for "cannot open /etc/hosts.allow" and "too many
> > > > > > >open files" and the solution to the hanging problem.
> > > > > > >
> > > > > > >is there any tool/prog that can be used to monitor slapd
> > > > > > >and email/page when the process hang?
> > > > > > >
> > > > > > >your help is greatly appreciated.
> > > > > > >
> > > > > > >thank you very much
> > > > > > >
> > > > > > >regards
> > > > > > >
> > > > > > >
> > > > > > >-----------------------------------------------------
> > > > > > >Get your FREE email from Catcha! http://mail.catcha.com
> > > > >
> > > >
> > > >
> > > > -----------------------------------------------------
> > > > Get your FREE email from Catcha! http://mail.catcha.com
> > > >
> > >
> > > 
> > 
> > 
> > -----------------------------------------------------
> > Get your FREE email from Catcha! http://mail.catcha.com