sql-back+replication

[Leen Besselink <leen@wirehub.nl>]

Hi every1,

Disclaimer: I'm just a webbuilder and fairly new to openldap and ldap, so
I probably won't be able to implement some of the ideas below.

Some time ago, my college said with a smile: "let's use openldap with
postgresql as backend, that would make things easier for us, also it would
be incredible cool.". So I told him: "but it would be a lot slower then
the a dbm backend, but maybe it's possible to use replication to
replicate it to openldap-slave-servers with berkeleydb for example and use
those as readonly-slaves."

It made me think about that yesterday. Now I don't know about the current
state of replication if your using back-sql, but I wouldn't be surprised
if it would 'just work', on the other hand I would be surprised if it
didn't work at all. Doesn't matter if it does, because obviously that
means you can't update the database with SQL. All updates would need to go
through slapd.

Which isn't pretty, because you'd have a full SQL-engine, but you can't
use it to it's fullest potential for making changes. :(

So I came up with a list of possible ways to do replication with
sql-backends:

1. slapd keeps a replication log... and there are triggers/procedural
language in the database that add any changes to that log..., but locking
issues make things harder also triggers need to filter out the changes
done through ldap. -- so this is a really crappy way of doing it.

2. slapd doesn't keep a replicationlog-file, but triggers in the database
write to a file in the format that slurpd understands. Triggers that write
to a file doesn't really sound like a great idea, also maybe you get
locking issues again, because several triggers at the same time... might
wanna write to the file ? well, this could work, but I'm afraid not all,
if not most databases don't supporting writing things to a file and using
procedural language to create ldif-like entries doesn't really appeal to
me. Maybe using something like a message-queue could help for this.

3. simplest way of doing replication is obviously letting the sql-backend
do replication, between sql-backends, but it wouldn't be as fast, but
you'd have loadbalancing atleast (also replication between different
database-vendors possible might not work).

but the best option is probably:

4. the database (using triggers/procedural language) keeps a tabel with
replication information, which is then read by a modified slurpd.

This seems a fairly simple solution, slurpd could even still use the same
working methods, copy-data to file and start working, check input-source
for news, sleep, etc.

Everything that is custom is near the data in the database, where
decisions are made, which makes it flexibel. Also the code for OBDC
connectivity, etc. which is in slapd van be used in an 'sql-slurpd'. 

Only thing I was kinda worried about was that if your database-server has
a crash or whatever your replication log is gone (or down) too. But if you
happen to lose data that way, any data just written to the database would
also be lost, so that could solve that problem (you would have no changes
to replicate). ;)

Well, it's just ideas, but I'd hope some1 would implement it and a very
big maybe even I might (help).

Hey I might start an interresting discussion, I like discussions,
especially if they start progression to something usefull. :)

Anyway I hope this was helpfull,
	Lennie.
_____________________________________
New things are always on the horizon.