random topics

[Howard Chu <hyc@symas.com>]

Some of these should go onto the roadmap or TODO list. Just scribbling them 
down here before I forget.

Had a chat with tridge and abartlet about the LDAP TXN support they're looking 
for with Samba4. One of their requirements was being able to do reads inside 
the TXN, and that is explicitly not supported by the LDAP TXN draft. But after 
talking a bit more about how they do things in LDB, we decided to take the 
same approach here - LDB only allows one writer when a TXN is active. We can 
write a (global) overlay that gives this same behavior, and then not have to 
worry about any of the other TXN details.

While thinking about back-mdb, I realized that since we're mapping storage to 
memory we don't really need a traditional disk-DB structure. Instead we can 
just manage the space as its own heap and use in-memory structures like AVL 
trees and such. I've been thinking about extending our current AVL library to 
a T-tree implementation. Further reading along those lines led me to 
Cache-Sensitive T-trees (CST-Trees) which also have the important property of 
being friendly to CPU cacheline sizes, and so behaving a lot better in 
multi-processor systems. This is a pretty generic project - if anyone's been 
looking for a relatively self-contained bit of work they could contribute, 
this would be ideal. You can get the paper here:

http://ids.snu.ac.kr/wiki/CST-Trees:_Cache_Sensitive_T-Trees

(Note that these things are only good because they can fit all the keys into a 
single node - i.e., they only have their most desirable properties when keys 
are small. We could certainly use them for the id2entry index, and maybe a few 
others, but in the general case we need to be able to use pointers to 
arbitrary data as keys.)

ITS#6301 also fed into this search, but I think we just want to use threaded 
AVL here instead of the Red-Black tree code. That would shrink the size of the 
patch down considerably, which would also be a good thing. Again, anyone 
looking for a relatively small project, this one is pretty straightforward.

Also a completely fringe topic, but still interesting - we've got OpenLDAP 
running well on Android now; on a G1 phone. But it's still just a set of 
command-line tools. UnboundID showed off their Java LDAP SDK in a simple app 
also running on Android. A combination of these two would make an extremely 
powerful package:

Develop a canned slapd config that uses syncrepl to sync with a remote LDAP 
address book. Use proxied multimaster, to allow local changes to be propagated 
back to the remote directory. Write a java app that uses LDAP for the G1's 
contact book backend instead of the SQLite ****stuff that it's currently 
using. Accomplishing this will necessitate writing a few GUI menus for 
configuring the handful of variables needed for the slapd config.

This also brings me to another topic - adopting features from OpenDS... They 
expose a cn=Tasks tree which can be used for submitting tasks via LDAP. 
Currently we expose our runqueue under cn=Monitor but that's only read-only. 
It would be nice to be able to submit/schedule/trigger tasks on the fly... In 
particular, it would be nice to have a defined task for triggering a syncrepl 
refresh.

The Android slapd config would be a refreshOnly syncrepl consumer. It would 
also have a hidden back-ldap refreshOnly consumer replicating from its local 
database and writing to the remote master. This way both push and pull would 
be under control of the Android device. (You have no idea how much of a bill 
you can rack up with fully automatic background synching. It's much better to 
have this under complete user control.) The GUI would just submit a request to 
cn=Tasks to trigger a synch.

OpenDS also has matching rules defined for comparing timestamp attributes to 
"current server time". This is extremely handy for a lot of things. Again, 
this is a small, self-contained project that should be simple for someone to 
jump in on.

More ideas later...
--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/