We used to just feed ldapmodify the differences. But we have some
problem with it.
All the source official data are provided in plain text files on a
daily basis. The way we find out changes is to compare two subsequent
datafeed files separately. And only those entries that have some info
changed in the datafeeds will be updated in the ldap directory.
However,this approach does not guarantee that the ldap update script
would not miss processing changes in the case of any unexpected
conditions (e.g. server error, ldif syntax error, etc.). In another
word, if the ldap update script misses processing the changes in one day
when the changes arise, these changes will not be changes any more and
will not be reflected in the directory. i.e.
day1's datafeed file day2's datafeed file
foo new-foo new-foo new-foo
That's why I'm thinking about blindly overwriting all the entries no
matter change or not. Our directory is used by the sendmail to forward
mails. So it needs to be up all the time.
If compareing each record in the datafeeds with ldap directly to
indentify changes, it might be involved a lot 'search' and 'compare'
operations. How are 50,000 'search' and 'compare' actions compared
with 50,000 'modify' (replace, add or delete) actions in terms of memory
consumption and server performance? (The number is not important here.
I just used 50,000 to represent a large amount).
Bing
Bing Du <bing-du@tamu.edu, 979-845-9577>
Texas A&M University, CIS, Operating Systems, Unix
>>> Allan E Johannesen <aej@WPI.EDU> 03/21/01 02:21PM >>>
>>>>> "Bing-Du" == Bing Du <Bing-Du@cis-gw.tamu.edu> writes:
Bing-Du> We use a home-grown script to build an LDIF file that contains
all
Bing-Du> modifications for all the entries in the directory first.
Then the
Bing-Du> script calls ldapmodify to perform the real change with the
directory
Bing-Du> server up.
Mine is a c program which compares the new entry to the existing one
and only
feeds ldapmodify the differences. The current record is kept in a
database.
Having thought about it, I've realized I should be comparing the new
record to
the current record from ldap itself, which would avoid skew between my
local
database and ldap itself.
I'll make that change when I alter my stuff to use openldap 2 vs my
current
openldap 1 setup.
Bing-Du> Your approach sounds like a brand new idea to me. How many
entries
Bing-Du> are their in your directory? How long does it take starting
from
Bing-Du> stopping the daemon till bring it back up after update? I
guess
Bing-Du> during this period, the directory is not available for being
looked
Bing-Du> up, right?
For some years, I used to build from scratch daily, too, but switched
to the
modification route in order to keep ldap up. Previously, I'd only used
ldap
for whitepages references, and didn't think that it being down for some
minutes
a day was a particular problem, but I'm planning to use it for
campus-wide
authentication, maybe for mail routing, and maybe more, so shutting it
down is
not an option.