Re: Number of replicated systems?

[Andrew Findlay <andrew.findlay@skills-1st.co.uk>]

On Mon, Sep 23, 2002 at 11:23:31AM -0400, Dan Lowe wrote:
> 
> In any case, I never found out if they were right since I've never gone 
> over 6 slave hosts.  Is there some data out there on how many 
> replicated hosts one can have off a single master?  I'd like to 
> rearchitect our LDAP service entirely and end up with something like a 
> dozen hosts replicated off a single master.  Is this feasible?

There does not appear to be any hard limit on the number of replicas
you can configure. The practical limit will depend on how much
modification traffic you have. Every change will result in N records
being written to replog where N is the number of slaves. slurpd then
has to pick those up and pass them on.

slurpd uses threads these days. The original version forked one copy
per replica which would have led to bottlenecks while accessing the
replication logfile. I don't know when the change occurred (i.e.
whether the iPlanet slurpd might still be the old version).

> Unfortunately I don't have enough non-production hardware to do the 
> testing myself and find out the right way.

I suspect you would be able to support 12 slaves if the update load is
not too high, but some tests are certainly indicated.... If problems
are going to show up they will affect the master rather than the
slaves, so why not try running all the slaves on one box (use
different port numbers). Ideally you want most of the slaves to accept
mods very rapidly without stressing your test box, so don't enable any
indexes, and put the database on very fast filestore (tmpfs or other
volatile memory-based store is ideal). You might even be able to use a
null backend on the database.

Tests I would try:

1)	Add 100+ entries to the master with ldapmodify and check that
	they propogate in an acceptable time.

2)	Stop one or more slaves and repeat (1) a few times.

3)	Restart slaves and check that slurpd catches up.

4)	Create a config error in a slave such that updates get
	rejected. Repeat (1) to see what happens.

5)	Try killing and restarting slurpd a few times while it is
	busy.

Watch CPU and I/O load on the master machine throughout the tests.
Try the whole lot again using strong authentication between master and
slaves - check that the added CPU load is acceptable.

Andrew
-- 
-----------------------------------------------------------------------
|                 From Andrew Findlay, Skills 1st Ltd                 |
| Consultant in large-scale systems, networks, and directory services |
|     http://www.skills-1st.co.uk/                +44 1628 782565     |
-----------------------------------------------------------------------