Re: overlay order important and not documented?

[Francis Swasey <Frank.Swasey@uvm.edu>]

On 3/16/07 2:36 PM, Howard Chu wrote:
> Pierangelo Masarati wrote:
> > Francis Swasey wrote:
> > > I have found using OpenLDAP 2.3.34 on a RHEL4 system (a locally built 
> > > RPM) that the order of the overlays can lead to a problem (slapd 
> > > locks up and doesn't do anything).
>
> It's already documented that overlays are executed in a specific order. 
> Obviously if the order didn't matter we wouldn't worry about it.

Yes, and it is documented that the overlays receive control in the 
reverse order that they are configured in.  I therefore, thought that 
since accesslog was configured in slapd.conf before refint that refint 
would receive control first and so refint's multiple operations would 
all then get logged by accesslog.

> > > I have found that using syncprov, accesslog, auditlog, unique will 
> > > work fine.  But, adding refint to the end of that chain and then do a 
> > > modrdn that triggers refint to want to change an attribute in another 
> > > entry and your slapd is locked up.  There is now one thread that will 
> > > not end and you will only be able to shutdown slapd with a "kill -9".
> > >
> > > However, if I change the order so that the refint is prior to the 
> > > accesslog overlay, then it works.
> > >
> > > Is this interaction between the two overlays and the importance of 
> > > their relative order a known (or at least expected) -- or should I 
> > > open an ITS?
> >
> > AFAIK, it is not known, although some strange and adverse interaction 
> > is possible (and feared).  I suggest you file an ITS.
>
> Not documented... The accesslog overlay serializes all write operations 
> by taking a lock. This means only one write operation is allowed to 
> progress at any time.

That fact is not obvious from reading the source code, is not mentioned 
in slapo-accesslog, and is not in any of the brief statements in the 
admin guide about overlays.

So, I do not believe that Howard's paragraph is clear to anyone using 
accesslog (other than people that are deeply involved in the code and 
where the variable name li has some special significance).

> The refint overlay creates multiple write operations from a single write 
> operation. If the accesslog overlay has already locked the current 
> operation, then yes, the refint overlay will deadlock at that point 
> because its write operations still go through the entire overlay stack. 
> I think the fix for this will be to change the refint overlay to bypass 
> any overlays above it when performing its own writes.

But if the refint overlay is specified in slapd.conf before the 
accesslog overlay it works and if they are configured with accesslog 
first and refint second it doesn't work.  Which says (to me) that the 
slapd.conf manual page's declaration

Overlays are pushed onto a stack over the database, and so they will 
execute in the reverse of  the  order in  which  they  were  configured 
 and  the database itself will receive control last of all.

is wrong.

Frank

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature