We are testing n-way master replication along with a large number of records using lmdb. Here's the config:
- 8 way replication with 8 nodes (each node having 7 other connections)
- 50k records
- Inserting the records into one cluster node to replicate to
all the rest
The problems obvserved:
- Some nodes are faster at replication than others. In general, the time to complete replication is slower than expected. In my test environment I found that 50k records can take up to 2 hours for some nodes to complete. The fastest nodes complete in 1.5 hours. Because these records are brand new insertions, delta based replication wouldn't help here.
- When the replication is completed, some of the data.mdb files
are larger than others (sometimes by an order of magnitude).
We would like to understand the reason behind these two problems
above. First, the replication system seems unusually slow. Second,
we need to understand why the data.mdb file grows sometimes far
larger on one node than the rest of the nodes. For example, in our
production environment, while most nodes were around 1GB in data
size, one node stored in excess of 40GB in data.mdb. In my testing
lab, my the 50k record insertion left most nodes with a data.mdb
size of 150MB. On one of the nodes, the data size was 262MB.
Note that I've also tried alternative replication connectivity
approaches to attempt to reduce the number of connections per
server, but that did not improve replication performance or the
varying data sizes in the end.
If updating to a newer version helps resolve the above observed
problems, please let me know.
Any tuning or debugging advice here would be appreciated.
Thanks.
|
Brian Wright |
 |