RE: LDUP Working Group Agenda - Summary of objections to requirements draft

["Steven Legg" <steven.legg@adacel.com.au>]

Albert,

> -----Original Message-----
> From: Albert Langer [mailto:Albert.Langer@Directory-Designs.org]
> Sent: Thursday, 27 July 2000 16:16
> To: 'Christopher Apple'; ietf-ldup@imc.org; ietf-ldapext@netscape.com
> Cc: agenda@ietf.org; johns@cisco.com
> Subject: RE: LDUP Working Group Agenda - Summary of objections to
> requirements draft

[snip]

> III. Discussion of LDUP Requirements Document
> 
> [Albert]
> I am also sending this to LDAPEXT as the direction taken by 
> LDUP could have
> major consequences for LDAPEXT (eg the current proposals for 
> ACL simply will
> not replicate correctly

Servers that don't store access controls as values of the ldapACI
attribute will have to give the appearance that they do for the
purposes of LDUP. Having done that, the access controls will replicate
as any other attribute values would.

> and future support for transactions will be
> difficult if not impossible).

I've outlined an extension to LDUP for providing strong transactional
consistency with a configurable degree of availability. Though you
might not agree with the philosophy, no one has yet pointed out any
fatal flaw in the procedure.

[snip]

> SUMMARY OF OBJECTIONS TO REQUIREMENTS DRAFT
> 
> The three key points are:
> 
> 1) There is no requirement for convergence or "eventual consistency".
> 
> This looks like just poor expression, but in fact the LDUP 
> architecture and
> Update Reconciliation Procedures do specify proposed standards that
> guarantee long term divergence by relying on timestamps

The timestamp in the CSN is just a version number that happens
to increment without visible update activity. A version number
scheme that leaves gaps in the runs of version numbers isn't
broken as long as the version numbers from a server are
monotonically increasing. The LDUP CSN is monotonically increasing too.
The only difference with the LDUP CSN is that the gaps just happen
to have a correlation to elapsed time.

> and 
> allowing DSAs to
> transmit changes out of order and drop changes when clocks 
> are out of sync.
> This is easily fixed, once a requirement to fix it is agreed on.
> 
> Details on how to fix it based on the Coda replication protocols also
> adopted by Active Directory, and a semi-formal proof that the 
> fix would be
> robust in the face of DSAs crashing and being restored from 
> backups, network
> partitioning etc etc, is included in my draft below.

Your proof neglects the effects of the purging mechanism. Restoring
a crashed DSA from a backup works if no change information is ever
removed. However if a backup restores a DSA to a state prior to the
purge point of any of the other replicas there exists the possibility
that the other DSAs have forgotten changes that the restored DSA
needs to bring it up to date and consistent with the others.

I have a procedure for solving the replica consistency problems of
restored replicas and rejoined partitions but it is written in terms
of a log-based implementation using a different purging mechanism.
I'm still in the process of recasting it in state-based terms with
an update vector.

[snip]
 
> 2) There is no requirement for atomic operations.
> 
> Again this is obscured by poor expression and nonsensical 
> definitions, but
> in fact the architecture and URP drafts merge changes to individual
> attribute values made concurrently at different replicas. The 
> fact that this
> obviously breaks the ACL standards being developed in LDUP,

URP doesn't merge changes to individual values. The current ldapACI
attribute type definition equates two values if they are semantically
the same, so URP will only have the effect of changing the meaning of
a collection of ACIs because of an explicit user change request to
remove ACIs. On the other hand, MDCR will arbitrarily throw away
previously accepted ACIs because of a potential, but probably
non-existent, semantic conflict between the original change requests.
MDCR assumes that changes to the same entry at different replicas
are automatically in conflict and one of them has to lose.
 
I contend that URP does less damage and therefore has a better chance
of achieving a favourable final AC state than MDCR.

> despite an
> overlap in authorship between the two documents, strongly 
> confirms that the
> consequences for existing applications are simply not 
> understand and should
> be studied through a requirements analysis by actively explaining the
> implications and soliciting input from other areas 
> (operations etc) that may
> be affected.
> 
> Fixing this would require substantial changes to the current 
> architecture
> and URP. I have sketched one possible way to do so in the draft below.
> 
> 3) There is no requirement to support mandatory operational 
> attributes of
> LDAP.
> 
> The operational attribute "modifiersName" cannot be supported 
> meaningfully
> as nobody in particular can be said to be responsible for a 
> change that has
> in fact been merged from two or more concurrent changes made 
> independently
> and without knowledge of each other.

Even though we talk about changes being "merged" the reality is
that one of them will take precedence over the others. The
operational attribute updates from that one change also take
precedence, so the value of modifiersName corresponds to the
latest apparent change to the entry, which is exactly what
you'd expect from the single master case.

> This severely complicates system
> administration as the first thing anybody would want to know 
> after receiving
> a problem report is "who changed what".

You've over estimated the utility of the modifiersName attribute.
It only tells you who last changed something, not what they changed.

[snip]

> In my view, both an explicit requirement for atomic operations and a
> requirement that the results be 1) predictable and 2) make some kind of
> sense to users, should be in the final requirements draft.

I don't think obliterating all trace of a user's previously
accepted change to an entry because someone else changed some unrelated
information in the same entry qualifies as either predictable or
sensible to users.
 
[snip]

Regards,
Steven