Re: refresh the entire directory

[Jens Vagelpohl <jens@digicool.com>]

On 3/21/01 15:10, "Bing Du" <Bing-Du@cis-gw.tamu.edu> wrote:
the directory itself doesn't have that many entries, it's a bit over 1000
right now.

since the refresh tool is actually keeping timestamped logs i can give you
some data:

* retrieving all records from postgres and creating the LDIF: 10 seconds
* building the ldap database using slapadd: 15 seconds

slapd is brought down for the second step when the databases are built. and
yes, it's obviously unavailable for lookup access during that time.

you could decrease the slapd downtime by a lot if you create the ldap
database somewhere on the side with slapadd and then simply stop slapd for
the instant it takes to move the already created database in place of the
old one.

jens


> We use a home-grown script to build an LDIF file that contains all
> modifications for all the entries in the directory first.  Then the
> script calls ldapmodify to perform the real change with the directory
> server up.
> 
> Your approach sounds like a brand new idea to me.  How many entries are
> their in your directory?  How long does it take starting from stopping
> the daemon till bring it back up after update?  I guess during this
> period, the directory is not available for being looked up, right?
> 
> Thanks,
> 
> Bing 
> 
> Bing Du <bing-du@tamu.edu, 979-845-9577>
> Texas A&M University, CIS, Operating Systems, Unix
> 
>>>> Jens Vagelpohl <jens@digicool.com> 03/21/01 01:14PM >>>
> i am doing something similar in my setup.
> 
> every night we are pulling data out of a postgres database which we use
> as
> the "source of record" for all address and contact data collected in
> the
> company. Our mechanism builds an LDIF file that contains the complete
> LDAP
> database structure and all entries.
> 
> Our way of replacing the LDAP backend database is simply a script that
> does
> the following:
> 
> * stop the slapd daemon
> * rm -rf the contents of our backend database directory
> * use slapadd to build a brand new ldap database from the LDIF file
> * restart slapd
> 
> this way there is no real impact on the server because we don't go
> through
> the server to rewrite the backend database, we just give it a brand new
> one.
> the only really ugly piece is the code that builds the LDIF file... :)
> 
> jens
> 
> 
> On 3/21/01 13:03, "Bing Du" <Bing-Du@cis-gw.tamu.edu> wrote:
> 
>> I'm just asking for opinions on refreshing the entire directory
> every
>> day.
>> 
>> We receive the official datafeeds in the separate files every day.
>> These datafeeds are used to update the directory.  They may or may
> not
>> include personal information changes.  In order to keep updating the
>> directory simple, we currently take the approach of overwriting
> every
>> ldap entry using the information provided in the datafeeds regardless
> if
>> the record is changed or not in the datafeeds.
>> 
>> Just want to know if this type of frequent bulk modifications would
>> have any impact on the directory server?   Currently we have about
>> 80,000 entries in the directory.
>> 
>> Any thoughts are appreciated.
>> 
>> Bing 
>> 
>> Bing Du <bing-du@tamu.edu, 979-845-9577>
>> Texas A&M University, CIS, Operating Systems, Unix
>> 
> 
>