OpenLDAP's backend for performance and high reliability

[Tommy Pham <tommyhp2@yahoo.com>]

Hi everyone,

I'm trying to convert from MS based platform to *nix/Linux, specifically FreeBSD.  I have a few questions and concerns about OpenLDAP's backend for performance and high reliability.

I have no problems setting OpenLDAP 2.3.38 to run with BDB 4.4.20.4 inside FreeBSD's (6.2 RELEASE) jail, using core, cosine, and inetorgperson schemas.  Using the Quick-Start guide at openldap.org's website, I manage to create the layout of OUs, CNs, etc. to my needs using phpLDAPadmin after adding the base ldif file via command line:

dn: dc=example,dc=com
objectclass: dcObject
objectclass: organization
o: Example Company
dc: example

After thinking about the robustness of OpenLDAP due to it's BDB backend, I tried to convert over to back-sql and use MySQL 5.0.45 for it's backend.  The SQL account is granted with full permission, including reference, to the specified database.  The
 tables are populated using sample sql files
  - testdb_create.sql
  - backsql_create.sql
  - testdb_metadata.sql

Then I tried to add the same base ldif file via command line and get this error:

ldapadd: Server is unwilling to perform (53)
        additional info: operation not permitted within namingContext

and all of my problems begin even though the unixODBC connection is working properly, despite it's not logging to a file via syslog (not a big concern ATM).  After a few days of frustration, research on documentation, and log analyzing, am I wrong to conclude after reading the contents of the sql files and this snip from log:
...
slapd startup: initiated.
backend_startup_one: starting "cn=config"
config_back_db_open
config_build_entry: "cn=config"
config_build_entry: "cn=include{0}"
config_build_entry: "cn=include{1}"
config_build_entry:
 "cn=include{2}"
config_build_entry: "cn=module{0}"
config_build_entry: "cn=schema"
config_build_entry: "cn={0}core"
config_build_entry: "cn={1}cosine"
config_build_entry: "cn={2}inetorgperson"
config_build_entry: "olcDatabase={-1}frontend"
config_build_entry: "olcDatabase={0}config"
WARNING: No dynamic config support for database sql.
config_build_entry: "olcDatabase={1}sql"
backend_startup_one: starting "dc=sointe,dc=net"
==>backsql_db_open(): testing RDBMS connection
backsql_db_open(): concat func not specified (use "concat_pattern" directive in slapd.conf)
backsql_db_open(): subtree search SQL condition not specified (use "subtree_cond" directive in slapd.conf)
backsql_db_open(): setting "ldap_entries.dn LIKE CONCAT('%',?)" as default
backsql_db_open(): setting "ldap_entries.dn=?" as default
backsql_db_open(): objectclass mapping SQL statement not specified (use "oc_query" directive in slapd.conf)
backsql_db_open():
 setting "SELECT id,name,keytbl,keycol,create_proc,delete_proc,expect_return FROM ldap_oc_mappings" by default
...
to conclude for me to use any RDBMS for OpenLDAP's back-sql, I have to setup all the tables, functions and/or stored procedures, and insert all of those appropriately as data into the 4 tables:
  - ldap_attr_mappings
  - ldap_entries
  - ldap_entry_objclasses
  - ldap_oc_mappings
before I can add the base ldif file via command line and use phpLDAPadmin to build & maintain it?  What happens when I need to change directory layout/structure because my needs change?  Is it feasible?

Here are a few case studies scenario where I see issues:

A) Small company
This can be accomplished with OpenLDAP and the database servers on the same box.  If the need requires more, OpenLDAP and the database server on separate boxes.  (This scenario can also be accomplished using BDB for
 backend.)

A - 1) Small company grows (still 1 site)
OpenLDAP becomes it's own box if it's not already and act as master.  Add more OpenLDAP box(es) for proxy requests while the master handles updates/additions.  The database server is then reconfigured to be clustered.  All OpenLDAP servers connect to the database cluster. (Alternatives?  Still possible with OpenLDAP+back-bdb in master/slave replication?  What about performance and high reliability?)

A - 2) Company grows and expand to multi-site
HQ (is the above scenario A - 1) and each site will have it's own OpenLDAP and database (as in scenario A) depending on requirements of each site and data connection to HQ.  (Alternatives?  Still possible with OpenLDAP+back-bdb in master/slave replication?  What about performance and high reliability?)

B) Enterprise ( or company in scenario A - 2 grows even more)
HQ will be setup as in scenario A - 1 but with
 more servers, both OpenLDAP and databases.  Each site will be setup as in scenario A or A - 1 depending upon the requirement and function of each site.

C) What happens (to performance and reliability) when the total entries in OpenLDAP reaches 250,000? 1 million? or 1 billion (most likely to happen in scenario B)?  Will it "shrugs it's shoulders as if nothing happens" and still function?

Based on the above the scenarios, should I invest the time to create all the tables, functions/stored procedures, and insert them as data into the 4 core tables as mentioned above?  I guess there could be work around for all the scenarios except scenario C.  Or should I go elsewhere for my directory service/server like OpenDS or Sun's Directory Server?

Thanks,
Tommy