RE: [ldap] Implementation Suggestions

["Spicer, Kevin" <KevinS@bmrb.co.uk>]

From: Daniel Henninger [mailto:daniel@ncsu.edu] 
>> It may be worth adding the noatime flag to the filesystems that hold
>> data and (bdb) logs.  You are indexing a lot of attributes, do you  
>> need

>No atime?  That's interesting.  I've never heard of that, what does  
>that do?
>From mount_ufs(1M)
         noatime
                      By default, the file system is mounted with
                      normal  access  time  (atime) recording. If
                      noatime is specified, the file system  will
                      ignore access time updates on files, except
                      when they  coincide  with  updates  to  the
                      ctime  or  mtime.  See stat(2). This option
                      reduces disk activity on file systems where
                      access  times are unimportant (for example,
                      a Usenet news spool).

                      noatime turns  off  access  time  recording
                      regardless of dfratime or nodfratime.

                       The POSIX standard  requires  that  access
                      times  be marked on files. -noatime ignores
                      them unless the file is also modified.  
I think this was introduced in Sol7 or 8, 


>As for the BDB logs, is it a reasonable thing to think  
>"this is totally read only and if i lose the data, I regenerate  
>it . . . that means I don't really need BDB's transaction logs"?  I  
>didn't see how to turn them off exactly, but I don't know that I need  
>them given that I'd sooner regen everything than I would attempt to  
>get BDB to roll back.

I'm not 100% sure but I think you do need at least the most recent
logfile (depending on logfile size and checkpoint interval) - otherwise
slapd won't be able to recover from an unclean shutdown. 


>There are not based off actual indexes, they are based more off of  
>everything folk have suggested they would search on.  I noticed that  
>if I don't have an index at all, that searches appear to flat out not  
>work.  So I seem to have to have -something- if I want folk to be  
>able to search on that field.  Is this not true?  Was something else  
>causing that hooha?  My primary goal here was to provide "as  
>searchable as possible" of a database, and try to govern as little as  
>possible how people want to use it.


Definately worth auditing actual index usage then, theres little point
indexing something that isn't being searched against if its hitting
write performance.  ldapstats.pl from
http://www.daemons.net/~matty/index.html may help (note you need
loglevel 256 for this purpose).

>> You have enough memory for 1.25 Gb of bdb cache?  You don't post your

>I should hope so, they have 2.5 Gb of ram each.  

Good good, but keep an eye out for excessive swapping, just in case.

> The entire pool is load balanced via  
>Ldirector (linux-ha.org) and, for example, if one croaks and I needed  
>to regenerate the database, I would just take it out of the pool and  
>repopulate it instead of trying to repair the tattered remains of  
>BDB.  

If thats your attitude and you just want raw performance you might
consider running your slave dbs in a ramdisk - so long as you appreciate
the risks in doing so!

>> machine spec so its difficult to say if that is sane, but make sure  
>> its
>> not using lots of swap.  I note that your bdb logs are on the same  
>> disk
>> as your swap (and your syslogs presumably), so if you are swapping
(or
>> anything is logging heavily) that will have an impact on bdb writes.

> Given I only currently have 2 disks, is this an important "you need  
>to add another disk" or ... ?  I read some articles about how I  
>needed to have the data separate in the past but that the logs would  
>be ok.  Also note again that I'm still not sure I need those logs.

Well, given the costs of Sun disks.... you really want to quantify where
the bottleneck is before spending money on that!  I suspect though that
your biggest issue is the number of indexes, I added a few extra indexes
to mine the other day, slapindex took a while (see the note at the end
of the man page!), but I had to rebuild with a slapadd last night and it
took ages longer than last time I did it.  Quanah's post also gives some
really good information (much better than mine!) on tuning a Solais box.

=================================================================

BMRB wins two BMRA awards - http://www.bmrb.co.uk
_________________________________________________________________
This message (and any attachment) is intended only for the 
recipient and may contain confidential and/or privileged 
material.  If you have received this in error, please contact the 
sender and delete this message immediately.  Disclosure, copying 
or other action taken in respect of this email or in 
reliance on it is prohibited.  BMRB Limited accepts no liability 
in relation to any personal emails, or content of any email which 
does not directly relate to our business.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++