[Date Prev][Date Next] [Chronological] [Thread] [Top]

Re: slapd-meta as a proxy for a monolithic namespace

Hummel, Wolfgang wrote:

Hello OpenLDAP community,

we are currently planning for a largescale directory with

1 Bio. entries in a single namespace.


The idea is to divide the DB in 10 equal sized DBs and let
them access by slapd-meta proxies.

Example:
dn: ou=rsp1,c=de,o=mno
entries with uid=79101234567890, 79101234567891 etc.
791 is always fix

In our scenario each server would have 100 Mio. entries
using the last digit of uid as a naturally even balanced distribution mechanism.

Here are the questions:

- is slapd-meta a feasible approach for this scenario ?
- how could the slapd.conf for the proxy look like ?

Here is a (non working) example with 2 backend servers.
What is wrong about it ?
Try using the correct DN to authenticate with. Your config uses "cn=admin" and your ldapsearch used "uid=admin" so naturally the request failed.
Other folks have been looking to do sharding this way as well. I think back-meta is a good starting point but it isn't the most convenient for it in its current form. 


...
moduleload      back_meta
moduleload      back_ldap
...
#######################################################################
# Meta database
#######################################################################
database meta
suffix " ou=rsp1,c=de,o=mno"
dncache-ttl forever
lastmod off
rootdn "cn=admin,ou=rsp1,c=de,o=mno"
rootpw secret
network-timeout 1
uri    "ldap://10.11.12.170/ ou=rsp1,c=de,o=mno"
rewriteEngine on
#rewriteContext searchFilterAttrDN
rewriteContext searchFilter
rewriteRule '^uid=[0-9]{11}1,.*' 'ldap://10.11.12.170/%0' ':@'
uri    "ldap://10.11.12.180/ ou=rsp1,c=de,o=mno"
rewriteEngine on
#rewriteContext searchFilterAttrDN
rewriteContext searchFilter
rewriteRule '^uid=[0-9]{11}2,.*' 'ldap://10.11.12.180/%0' ':@'
...

logfile snippet for
# ldapsearch -LLL -xD uid=admin,ou=rsp1,c=de,o=mno -w secret -b ou=rsp1,c=de,o=mno uid=791720001981
ldap_bind: Invalid credentials (49)

Apr 23 08:44:13  slapd[26200]: >>> dnPrettyNormal: <uid=admin,ou=rsp1,c=de,o=mno>
Apr 23 08:44:13  slapd[26200]: <<< dnPrettyNormal: <uid=admin,ou=rsp1,c=de,o=mno>, <uid=admin,ou=rsp1,c=de,o=mno>
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0 BIND dn="uid=admin,ou=rsp1,c=de,o=mno" method=128
Apr 23 08:44:13  slapd[26200]: do_bind: version=3 dn="uid=admin,ou=rsp1,c=de,o=mno" method=128
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0 meta_back_bind: dn="uid=admin,ou=rsp1,c=de,o=mno".
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0: meta_back_getconn[0]
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0: meta_back_getconn[1]
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0 meta_back_getconn: candidates=2 conn=ANON fetched
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0 >>> meta_back_search_start[0]
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0 >>> meta_search_dobind_init[0]
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0 <<< meta_search_dobind_init[0]=1
Apr 23 08:44:13  slapd[26200]: ==> rewrite_context_apply [depth=1] string='uid=admin,ou=rsp1,c=de,o=mno'
Apr 23 08:44:13  slapd[26200]: ==> rewrite_context_apply [depth=1] res={0,'NULL'}
Apr 23 08:44:13  slapd[26200]: [rw] searchBase: "uid=admin,ou=rsp1,c=de,o=mno" -> "uid=admin,ou=rsp1,c=de,o=mno"
Apr 23 08:44:13  slapd[26200]: ==> rewrite_context_apply [depth=1] string='(objectClass=*)'
Apr 23 08:44:13  slapd[26200]: ==> rewrite_rule_apply rule=''^uid=[0-9]{11}1,.*'' string='(objectClass=*)' [1 pass(es)]
Apr 23 08:44:13  slapd[26200]: ==> rewrite_context_apply [depth=1] res={0,'(objectClass=*)'}
Apr 23 08:44:13  slapd[26200]: [rw] searchFilter: "(objectClass=*)" -> "(objectClass=*)"
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0 <<< meta_back_search_start[0]=1
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0 >>> meta_back_search_start[1]
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0 >>> meta_search_dobind_init[1]
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0 <<< meta_search_dobind_init[1]=1
Apr 23 08:44:13  slapd[26200]: ==> rewrite_context_apply [depth=1] string='uid=admin,ou=rsp1,c=de,o=mno'
Apr 23 08:44:13  slapd[26200]: ==> rewrite_context_apply [depth=1] res={0,'NULL'}
Apr 23 08:44:13  slapd[26200]: [rw] searchBase: "uid=admin,ou=rsp1,c=de,o=mno" -> "uid=admin,ou=rsp1,c=de,o=mno"
Apr 23 08:44:13  slapd[26200]: ==> rewrite_context_apply [depth=1] string='(objectClass=*)'
Apr 23 08:44:13  slapd[26200]: ==> rewrite_rule_apply rule=''^uid=[0-9]{11}2,.*'' string='(objectClass=*)' [1 pass(es)]
Apr 23 08:44:13  slapd[26200]: ==> rewrite_context_apply [depth=1] res={0,'(objectClass=*)'}
Apr 23 08:44:13  slapd[26200]: [rw] searchFilter: "(objectClass=*)" -> "(objectClass=*)"
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0 <<< meta_back_search_start[1]=1
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0 meta_back_search: ncandidates=2 cnd="**"
Apr 23 08:44:13  slapd[26200]: daemon: activity on 1 descriptor
Apr 23 08:44:13  slapd[26200]: daemon: activity on:
Apr 23 08:44:13  slapd[26200]:
Apr 23 08:44:13  slapd[26200]: daemon: epoll: listen=8 active_threads=0 tvp=zero
Apr 23 08:44:13  slapd[26200]: daemon: epoll: listen=9 active_threads=0 tvp=zero
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0 meta_back_search[0] match="" err=32 (No such object).
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0 meta_back_search[1] match="" err=32 (No such object).
Apr 23 08:44:13  slapd[26200]: send_ldap_result: conn=1015 op=0 p=3
Apr 23 08:44:13  slapd[26200]: send_ldap_result: err=32 matched="ou=rsp1,c=de,o=mno" text=""
Apr 23 08:44:13  slapd[26200]: conn=1015 op=0 meta_back_bind: no target for dn "uid=admin,ou=rsp1,c=de,o=mno" (32).






--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/