[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: multiway replication

Venkata Pingali wrote:


Thanks, Howard! I will surely check with 2.3. Meanwhile I got another configuration "sort-of" working (see diagram).

What I had to do was the following:

1. Have the sites write into a "local" (ou=local,ou=xbone) subtree
2. The local subtree mods are propogated to a "proxy" slapd listening on port 10000 at theHQ machine. This "proxy" runs a slapd-meta configuration to rewrite "ou=local,ou=xbone" to "ou=global,ou=xbone" and forward the updates to the main slapd server (port 636) with the rewritten DNs.
3. The sites then synrepl the global shared tree (ou=global,ou=xbone)

In effect there is no loop. However, I had two problems:

1. meta backend at the proxy was filtering timestamp attributes when forwarding mods to the main slapd
2. the main slapd's contextCSN was not getting updated.

For (1) I had temporarily comment the following lines in back-meta/add.c:

          if ( a->a_desc->ad_type->sat_no_user_mod  ) {
                       continue;
               }


In back-ldap this is handled with:

   isupdate = be_shadow_update( op );
...
       if ( !isupdate && a->a_desc->ad_type->sat_no_user_mod  ) {
           continue;
       }

Probably the same change needs to be present in back-meta.

Also, you should probably be using back-ldap anyway, you're not doing anything special here that requires the features of back-meta. And eventually back-meta is going to be phased out.


I am not sure about the appropriateness of the action and I was going to ask you all.

For (2) I had a background job that just fiddled (add and delete an entree) with the main server to force an update. Hopefully I wont have to do this with 2.3. 

Right, no need for that in 2.3.


Do you have an idea about (1) above?

thanks!
-Venkata

Howard Chu wrote:

Sounds like an interesting solution. It should work using the 2.3 syncprov overlay, as the method for recording the contextCSN has been changed (among other things).

Venkata Pingali wrote:

What:

A tree that is (1) shared at multiple sites (arbitrary number) (2) each
site responsible for its part of the subtree. However when a site makes
changes the changes have to propagate to all site so that everyone sees
the same tree. We dont know ahead of time how many sites there will be.
The data volume is expected to be low i.e., tree is quite small.

One solution:

I have looked at the multimaster solution in which each site updates
N other sites through N replica commands.

The problem is that as new sites get added, the configuration at every
site will have to changed.

Alternate solution:

An alternative design is shown in the attached text file. Basically there is a
central repository Each site pushes changes into the tree at the central
repository using slurpd and sucks in any updates using the syncrepl
functionality. (All slapd/slurpd compiled with multimaster flag enabled)
A nice property of this setup is that the central server need not
track how many sites exist and any site that is added get
reflected automatically in the tree.

Problem:

Each site is able to push the changes to the central slapd server but not
pull changes using syncrepl. If the changes are done at the central site
using ldapadd the sites are able to pull the updates.

After some digging I found that the ldapsync object's contextCSN
(a time stamp) attribute at the central tree was not getting updated
after site slurpd modifies the tree at the central server. Therefore
when the site slapd server tries to look for updates using syncrepl,
it doesnt find any even when the tree has been updated
(since the timestamp - contextCSN - has not been updated).

Is there a compile time/runtime option to force the contextCSN
updates? Alternatively is it possible to update using some offline
mechanism that I can put into cron?

More fundamentally, does this sound like a reasonable approach
for N-way replication? Are you aware of other solutions?

thanks!
Venkata Pingali
USC/ISI

------------------------------------------------------------------------ 



Shows slapd at HQ and slapd/slurpd combination at one site.
                     HQ             +--------------------+
         +->|        slapd       |
        /   +--------------------+
       /     |                 /      |
+-------------------+      +----
|     /       V     |      |
|   +-+      +-+    |      |
|   | |<-----| |    | .... |
|   +-+      +-+    |      |
|  slurpd   slapd   |      |
+-------------------+      +-----

SITE 1 SITE 2 ...





------------------------------------------------------------------------



HQ +-----+ +-----+
+->|meta |---->|slapd| | +-----+ +-----+ | |
| +--------+
+-------------+-----+ +----
| | V | |
| +-+ +-+ | |
| | |<-----| | | .... |
| +-+ +-+ | |
| slurpd slapd | |
+-------------------+ +-----

       SITE 1                SITE 2 ...



--
 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support