Re: pcache rfe

[Howard Chu <hyc@symas.com>]

Remco Post wrote:

> Hi all,
>
> I've been playing with the pcache (HEAD code) and it does seem to be 
> very usefull as it is but:
>
> I'd like to be able to cache only a subset of the data. We have fields 
> like userPassword and loginShell that should preferably allways be 
> retrieved from the slq backend because these values can change quite 
> rappidly (if a user has used all his budget, his shell changes, when 
> more budget is added his shell changes back, comparable things happen 
> to userPassword) so these should not be cached, or cached only for a 
> short time, while other attributes are more static and could be cached 
> for ages (one hour or even more).
>
> Even worse, one client does not even list the requested attributes in 
> it's ldap-queries (I've notified the vendor that I think it should), 
> so non of that clients queries are cachable.
>
> as I see it, currently the pcache does not allow for only part of a 
> query to be retrieved from cache and the rest of the attributes to be 
> retrieved from the backend (in my case sql). It would be nice (I 
> think) if this were possible in the future, I guess this might be 
> quite intrusive in the way the cache works, but it would make the 
> cache more versatile and in my case it makes a huge impact on the 
> usefullness of the cache.

I believe what you suggest would actually completely negate the 
usefulness of the cache. If any part of the entry is not cached, then 
all requests for the entry will have to be passed through to the 
underlying database. Of course, this depends on the actual queries; if 
there are no queries that request both cached and uncached attributes at 
the same time then I guess it might be practical.

We (Symas) have already developed merging code to coalesce attributes 
from entries in two separate databases into a single entry. This will be 
appearing in HEAD soon as the "translucent" overlay. This overlay was 
briefly mentioned at the OpenLDAP Developer Day conference in San Diego 
last month. The idea is that you have a remote database that carries 
authoritative, generic information for the DIT and a local database that 
contains partial entries carrying attributes to augment the remote 
entries. This is particularly useful for application-specific local 
directories that have no write-access to the schema of the 
remote/authoritative directory, but need to associate 
application-specific information with the existing entries.

E.g., the master database might have
  dn: cn=Joe User,o=example.org
  objectclass: person
  cn: Joe
  sn: User
  userPassword: mySecret

and the local database might have
  dn: cn=Joe User,o=example.org
  objectclass: CustomAppObject
  customAttr: a bunch of info

Searching on the local database for "Joe User" would return a single 
merged entry:
  dn: cn=Joe User,o=example.org
  objectclass: person
  objectclass: CustomAppObject
  cn: Joe
  sn: User
  userPassword: mySecret
  customAttr: a bunch of info

This feature can probably be incorporated into the proxycache code to 
achieve what you want. (This statement does not imply that anybody I 
know of is planning to do so, just that a sufficiently motivated person 
could do it.)

--
 -- Howard Chu
 Chief Architect, Symas Corp.       Director, Highland Sun
 http://www.symas.com               http://highlandsun.com/hyc
 Symas: Premier OpenSource Development and Support