Re: Multimaster further work

[Daniel Tiefnig <openldap@qmail.infonova.at>]

Derek Simkowiak wrote:
>> Well, it's "just a master", so scalability doesn't matter that much.
> 
> Ahem... my customers would disagree.  And so would I.  The first
> time your boss' laptop gets stolen and you can't revoke his stolen
> client-side certificate from the LDAP database, you'll change your
> mind :)

How does that relate to scalability? What you are addressing is 
availability. And that's solved by HA clusters, usually. (If the 
software itself doesn't provide some means of HA, like a multimaster 
setup would.)
What i meant to say is that write and replication load on your master 
should never get higher than read and write load on your slaves. If it 
does, you may don't want to use LDAP but maybe mysql or oracle.

>> No. Absolute minimum would be twice MAX_LAG_TIME. (round-trip)
> 
> I should clarify what I meant by MAX_LAG_TIME.
> 
> It is NOT the amount of time it takes for information to travel
> across some network (although, that is one component).  It is the
> maximum amount of time it would take for us to receive an update from
> the furthest away master.

Well, I see the point, but I think that doesn't change anything. You 
still have to wait at least twice that time.

I'll try to illustrate that with server A and B:
(Switch to fixed width font, if that doesn't make any sense at all. If 
it still doesn't make any sense, drop me a note. :o) )

-----------------o------o-o------o-o---------> time (in msec :o)
                 ^      ^ ^      ^ ^
                 |      | |      | |
update on A -----'      | |      | |
update on B ------------' |      | |
B gets update from A -----|      | |
that's also MAX_LAG_TIME -'      | |
A gets update from B ------------' |
that's twice MAX_LAG_TIME ---------'

So, if you just wait for MAX_LAG_TIME, server B will see the conflict, 
server A not.

> When you write out (and replicate out) a change, we flag that as
> time_init for that DN.
> 
> Now, when we get a conflicting write from one of the other
> masters, we should wait not just MAX_LAG_TIME, but MAX_LAG_TIME plus
> the maximum amount of time it could take for our last write to get to
> the other masters -- call it "time_difference".  This way we can be
> sure that all the masters can our conflicting write candidate for
> consideration.

Hmm, i disagree with that somehow. Looking at the timeline above, it 
makes sense to me to fixate the following:

1) When one of the Nodes gets a genuine update, it waits at least twice 
MAX_LAG_TIME for conflicts.
2) When one of the Nodes gets a replicated update, it waits at least 
twice MAX_LAG_TIME for conflicts _and_ before replicating out a new 
genuine update, that arrived after the replicated one.

> The var "time_period_required_to_ensure_our_write_was_received"
> above will be different for each master, thus "sync'ing" up the
> servers in the global time frame.

Hmm, I'll have to look into that, but at the first glance, I think that 
results in the same timeframe as (approx., when _not_ using an 386sx) 
twice MAX_LAG_TIME. (init + timediff + (MLT - timediff) + MLT == init + 
2MLT) At least for server B in the above example. And as i said, server 
A wouldn't know about the conflict at all.

> ...I think.

:o)

>  I'd like to know if that makes sense to other people.

Well, kind of.

>> I think we may avoid this, by delaying genuine updates, until we have
>> committed replicated (and conflicting) ones, but I'm not quite sure
>> about that.
> 
> I think that waiting MAX_LAG_TIME before commiting and replicating
> the first write is pointless;

Ah, no. That's not the point. We're delaying genuine updates only, if 
there allready is a conflict that has to be solved. (I don't want to 
insert a third (or fourth, or..) update, when there are allready two 
updates known as conflicting.)

> One DN write might update, say, a uid, and nothing else.  But a
> conflicting write may update that DN's uid, AND the homeDirectory. 
> What if the uid update was actually to the same value?  Is that really
> a conflict?  It seems more like an addendum to me.

Good point. Applying all updates in series will enhance results in some 
cases. And, as a side effect, will somehow rebuild single master 
behaviour.

>> Generally I think this topic is quite interesting and bears some fun,
>> so please keep me/us uptodate on its progression.
> 
> I think this list makes a great forum.  And I think this topic is
> quite interesting also.

Well, of course it does, I just thought it may make sense to change to a 
forum where people are actually interested to develop OpenLDAP.

> Yesterday I noticed
> 
> http://www.ietf.org/html.charters/ldup-charter.html

Don't wake sleeping dogs.. ;o)


lg,
daniel
-- 
.. an initial underscore already conveys strong feelings of magicalness
to a C programmer.
          -- Larry Wall