Re: indexing & hardware questions -- again

[Ron Chmara <ron@Opus1.COM>]

On Monday, August 6, 2001, at 07:54  AM, oberwetter, josh wrote:
> All, thanks for responding. Here are some follow-up
> clarifications/remarks:
> > > Finally, is slurpd actually more efficient at handling modify
> > > operations? While a directory is handling a modify operation from a
> > > client, its performance is pretty bad. So, by "efficient" I mean, do
> > > the slaves take significantly LESS of a performance hit if the slurpd
> > > process sends modify operations than if a slave handles client
> > > requests directly?
> > slurpd doesn't perform modify operations.  It mirrors such
> > operations on a
> > master slapd to the slave slapd.  At least as I understand
> > it.  slapd still is
> > the one performing the modify op.
> My specific question is this:
> I am concerned about the costs and benefits of processing modify
> operations via slurpd, with respect to how modify operations hurt the
> speed of searches.

Well, This is one of the reasons to not write applications that 
modify the directory an awful lot... :-) I generally think of 
LDAP as a read-only database, with write functions only for 
general maintenance... I frequently have clients who, once they 
"discover" LDAP, want to put real-time data changes into LDAP, 
which is generally difficult to do. It's not really optimized 
for that.... if you are expecting many changes simply because of 
your entry count, you might want to split out your servers into 
a tree, so each server is only handling a subsection's worth of 
changes, or build a "change loading only" server, or use a 
combination of row level (or even row-column) locking RDBM's and 
LDAP. (MySQL suffers similar problems under load, because of the 
locking mechanism)

> ldapadd is much more faster at importing a lot of data than some client
> that I write would be. Is slurp also more efficient?

slurpd is simply a method of "replaying" an add/modify 
operation. In relation to a "client that I write", I suppose 
you're wanting to know if slupd is more efficient at processing 
300 changes than, say, a java/C/Perl application looping through 
and acting as a slurpd, replaying changes on 3 different ldap 
servers. The answer is a qualified "maybe". Slurpd is no slouch, 
and was written to have fairly low overhead.... but you may be a 
coding fiend who could write faster C code that did the same 
thing, or was optimized for the way your data was changing (say, 
batching up an 24 hours of changes), or used a different 
methodology, for example, playing thousands of changes into a 
"loading only" ldap database on the slaves, and then swapping 
out the live database with the "load only" db, or immediately 
moved all changes to all servers (if you needed such a thing).

There's lots of ways of doing speed tuning, so I repeat my usual mantra:
"Test with a pilot, tune as needed, try a few different things 
and choose the best one."

-Ronabop

--2D426F70|759328624|00101101010000100110111101110000
ron@opus1.com, 520-326-6109, http://www.opus1.com/ron/
The opinions expressed in this email are not necessarily those 
of myself,
my employers, or any of the other little voices in my head.