Re: OpenLDAP 2.4

["Gavin Henry" <ghenry@suretecsystems.com>]

<quote who="Howard Chu">
> The Admin Guide still has not been updated with all of the relevant
> changes,
> so here are some notes on new features in the 2.4 release... I believe all
> of
> the manpages are up to date, so you can get specifics from them.
>
> More complete cn=config functionality:
>     There is a new slapd-config(5) manpage for the cn=config backend.
>
>     the original design called for auto-renaming of config entries when
> you
> insert or delete entries with ordered names, but that was not implemented
> in
> 2.3. It is now in 2.4. This means, e.g., if you have
> 	olcDatabase={1}bdb,cn=config
> 	olcSuffix: dc=example,dc=com
>
> and you want to add a new subordinate, now you can:
> ldapadd olcDatabase={1}bdb,cn=config
> 	olcSuffix: dc=foo,dc=example,dc=com
>
> this will insert a new BDB database in slot 1 and bump all following
> databases down one, so the original BDB database will now be named
> 	olcDatabase={2}bdb,cn=config
> 	olcSuffix: dc=example,dc=com
>
>     In 2.3 you were only able to add new schema elements, not delete or
> modify existing elements. In 2.4 you can modify schema at will. (Except
> for
> the hardcoded system schema, of course.)
>
>
> More sophisticated syncrepl configurations:
>     the original implementation of syncrepl in OpenLDAP 2.2 was intended
> to
> support multiple consumers within the same database, but that feature
> never
> worked and was removed from OpenLDAP 2.3. I.e., you could only configure a
> single consumer in any database.
>
>     In 2.4 you can configure multiple consumers in a single database. The
> configuration possibilities here are quite complex and numerous. You can
> configure consumers over arbitrary subtrees of a database (disjoint or
> overlapping). Any portion of the database may in turn be provided to other
> consumers using the syncprov overlay. The syncprov overlay works with any
> number of consumers over a single database or over arbitrarily many glued
> databases.
>
>     As a consequence of the work to support multiple consumer contexts,
> the
> syncrepl system now supports full N-way multimaster replication with
> entry-level conflict resolution. There are some important constraints, of
> course: In order to maintain consistent results across all servers, you
> must
> maintain tightly synchronized clocks across all participating servers
> (e.g.,
> you must use NTP on all servers). The entryCSNs used for replication now
> record timestamps with microsecond resolution, instead of just seconds.
> The
> delta-syncrepl code has not been updated to support multimaster usage yet,
> that will come later in the 2.4 cycle.
>
>     On a related note, syncrepl was explicitly disabled on cn=config in
> 2.3.
> It is now fully supported in 2.4; you can use syncrepl to replicate an
> entire
> server configuration from one server to arbitrarily many other servers.
> It's
> possible to clone an entire running slapd using just a small (less than 10
> lines) seed configuration, or you can just replicate the schema subtrees,
> etc. Tests 049 and 050 in the test suite provide working examples of these
> capabilities.
>
>     In 2.3 you could configure syncrepl as a full push-mode replicator by
> using it in conjunction with a back-ldap pointed at the target server. But
> because the back-ldap database needs to have a suffix corresponding to the
> target's suffix, you could only configure one instance per slapd.
>
>     In 2.4 you can define a database to be "hidden" which means that its
> suffix is ignored when checking for name collisions, and the database will
> never be used to answer requests received by the frontend. Using this
> hidden
> database feature allows you to configure multiple databases with the same
> suffix, allowing you to set up multiple back-ldap instances for pushing
> replication of a single database to multiple targets. There may be other
> uses
> for hidden databases as well (e.g., using a syncrepl consumer to maintain
> a
> *local* mirror of a database on a separate filesystem).
>
>
> More extensive TLS configuration control:
>     In 2.3, the TLS configuration in slapd was only used by the slapd
> listeners. For outbound connections used by e.g. back-ldap or syncrepl
> their
> TLS parameters came from the system's ldap.conf file.
>     In 2.4 all of these sessions inherit their settings from the main
> slapd
> configuration but settings can be individually overridden on a
> per-config-item basis. This is particularly helpful if you use
> certificate-based authentication and need to use a different client
> certificate for different destinations.
>
>
> Various performance enhancements:
>     Too many to list. Some notable changes - ldapadd used to be a couple
> of
> orders of magnitude slower than "slapadd -q". It's now at worst only about
> half the speed of slapadd -q. A few weeks ago I did some comparisons of
> all
> the 2.x OpenLDAP releases; the results are in the slides from my SCALE
> presentation and you can find a copy here:
> 	http://www.highlandsun.com/hyc/scale2007.pdf
>
>     That compared 2.0.27, 2.1.30, 2.2.30, 2.3.33, and HEAD (as of a couple
> weeks ago). Toward the latter end of the "Cached Search Performance" chart
> it
> gets hard to see the difference because the runtimes are so small, but the
> new code is about 25% faster than 2.3, which was about 20% faster than
> 2.2,
> which was about 100% faster than 2.1, which was about 100% faster than
> 2.0,
> in that particular search scenario. That test basically searched a 1.3GB
> DB
> of 380836 entries (all in the slapd entry cache) in under 1 second. i.e.,
> on
> a 2.4GHz CPU with DDR400 ECC/Registered RAM we can search over 500
> thousand
> entries per second. The search was on an unindexed attribute using a
> filter
> that would not match any entry, forcing slapd to examine every entry in
> the
> DB, testing the filter for a match.
>     Essentially the slapd entry cache in back-bdb/back-hdb is so efficient
> the search processing time is almost invisible; the runtime is limited
> only
> by the memory bandwidth of the machine. (The search data rate corresponds
> to
> about 3.5GB/sec; the memory bandwidth on the machine is only about 4GB/sec
> due to ECC and register latency.)
>
> I think it goes without saying that no other Directory Server in the world
> is
> this fast or this efficient. Couple that with the scalability,
> manageability,
> flexibility, and just the sheer know-how behind this software, and nothing
> else is even remotely comparable.
> --
>    -- Howard Chu
>    Chief Architect, Symas Corp.  http://www.symas.com
>    Director, Highland Sun        http://highlandsun.com/hyc
>    Chief Architect, OpenLDAP     http://www.openldap.org/project/
>

One e-mail to be framed! ;-)

You mentioned benchmarks etc. with bdb 4.6.2, being the fastest read/write
ever. Any news on 4.6.x going stable?

Gavin.

-- 
Kind Regards,

Gavin Henry.
Managing Director.

T +44 (0) 1224 279484
M +44 (0) 7930 323266
F +44 (0) 1224 824887
E ghenry@suretecsystems.com

Open Source. Open Solutions(tm).

http://www.suretecsystems.com/