[Date Prev][Date Next] [Chronological] [Thread] [Top]

RE: Proxycache Documentation - again , . . . .

Howard,

You are right in your doubt if it is worthwhile to maintain a more
sophisticated cache, or precaching. All these things come obviously at a
certain cost. I think that the only person who can answer this question is
the administrator of the cache. 

Understand where I am trying to go ? 

Every directory installation is different, and therefore requirements are
also. It may happen that a cache is absolut useless, inasmuch the majority
of queries are so different from each other that you may have the same query
once a week . . . . . On the other hand it may also happen that you have the
same query or a contained query several times a minute.

What if the admin of the ldap proxy could configure the server in a way to
fit best the actual needs ?

For example if he could choose between
A) query containtment as implemented now, or 
B) query containment matching also for subsets of the previous cached
results. 

In such a way the administrator has to understand if the overhead caused by
a more sophisticated query containment manager costs to much or if it pays
off the benefit he hopes to have. Remain to understand however if there's a
chance that someone opts for the second possibility, I admit, that's
unclear.

Cheers
Reinhard




-----Original Message-----
From: Howard Chu [mailto:hyc@highlandsun.com] 
Sent: martedì 6 aprile 2004 12.15
To: ando@sys-net.it; kapurva@in.ibm.com
Cc: Voglmaier, Reinhard Erich; openldap-devel@OpenLDAP.org
Subject: RE: Proxycache Documentation - again , . . . .


> -----Original Message-----
> From: owner-openldap-devel@OpenLDAP.org 
> [mailto:owner-openldap-devel@OpenLDAP.org]On Behalf Of Pierangelo 
> Masarati

> > An important design consideration was to reduce the cost of query 
> > containment. Proxy cache is designed to improve performance
> for a subset
> > of queries which belong to specified templates. Thus containment is 
> > checked between queries of the same template. Apart from
> reducing the
> > number of filter and attribute set comparisons, it greatly
> simplifies
> > filter containment (at least for conjunctive queries). This
> does result
> > in slightly reduced cache answerability.
>
> Apurva,
>
> I clearly see your point, but Reinhard's consideration
> is really intuitive.  I'm working on query templating
> for a totally unrelated issue, but I'll try to get inspiration from 
> your code.  I think the possibiity to recognize sub-contained queries 
> should be investigated.
>
> I'll have a look at it.

There are definite tradeoffs to consider here. The actual usefulness of such
a change is limited, given the constraints on the cacheability
determination. If you keep the current cacheability configuration, then only
a small percentage of your queries will keep the cache up to date. The
information will age out despite the other queries that are answerable from
it.

If you go to a more free format cacheability decision, (i.e., cache any
queries that match the filter template and any subset of these attributes)
then determining answerability becomes much more expensive.

Consider - if you use an attrset with 20 attributes, then somebody must make
a query requesting all 20 of those attributes before any query data enters
the cache, and somebody must repeat this query on a regular basis to keep
the cache fresh (since we don't use syncrepl yet to refresh it). All the
other queries that only want one or two attributes can only be satisfied
after the 20-attr query has been done.

If you change the nature of the cache such that querying for any subset of
the 20 attributes will update the cache, then you have the complex job of
tracking which individual cached queries can answer a given request. You can
reduce the number of comparisons by using sorted attr sets and other such
tricks, but you have to inspect every cached query in the database. (You
could probably index it if you limit attrsets to 32 or less, and use a
bitmap to represent the attrs present in each query.)

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support