Re: syncrepl multicast MMR

[Andrew Findlay <andrew.findlay@skills-1st.co.uk>]

On Sun, Feb 08, 2015 at 12:52:40PM +0000, Howard Chu wrote:

> Been thinking this would be worth trying for a while now. Set a
> config option for syncprov to send Persist messages to a multicast
> group instead of the original TCP session. All the consumers would
> also join the group and listen for updates. This would also exercise
> the cldap:// support in libldap.

It certainly makes sense to have the network do more of the work
where it can. This would be particularly valuable in the high fan-out
case.

Encryption and message signing needs some thought - this is usually
harder to get right in datagram protocols than in streams.

While we are talking datagrams and multicast, have you looked at Fountain Codes?
It seems to me that they would be an ideal way to initialise a large set
of replica servers. They could also be used in the persist update case,
avoiding the need for any sort of back-channel.

For those who have not met them, Fountain Codes allow you to broadcast
large datasets to an unknown number of receivers over lossy channels.
If well designed, each receiver needs to collect any randomly-chosen
subset of datagrams adding up to a few percent more bytes than the
source data. One such code is described in RFC5053, though there
appear to be patent issues to be considered.

Andrew
-- 
-----------------------------------------------------------------------
|                 From Andrew Findlay, Skills 1st Ltd                 |
| Consultant in large-scale systems, networks, and directory services |
|     http://www.skills-1st.co.uk/                +44 1628 782565     |
-----------------------------------------------------------------------