RE: out-of-order replog (ITS#2623) ?

[hyc@highlandsun.com]

-----Original Message-----
>From: Jonghyuk Choi [mailto:jongchoi@us.ibm.com]

>Re: ITS#2623 and Howard's previous note to congestion control.

>Is the out-of-order replog problem related to the Berkeley DB rollback ?
>By looking at the code, all the replog()s called outside the transaction.
>(In fact, all but two of them are called from the frontend).

>Do we need to move slap_get_time() to somewhere after
>ldap_pvt_thread_mutex_lock(&replog_mutex), inside replog() ?
>- Jong

I think this would only mask the problem... The timestamps would always be in
order, but there's still no guarantee that two operations that completed
around the same time will appear in the replog in the same order in which
they executed. If they are serially dependent, e.g., two successful modrdn's
on the same entry that occur back-to-back, this is still a problem. It seems
that having slurpd sort its queue would work, on the assumption that the
result from slap_get_time() really reflects the time that an operation
completed. But since the granularity of slap_get_time() is only 1 second, it
still doesn't solve the problem when many operations complete around the same
time.

Maybe now that we have a fine-grained CSN to work with, slurpd should sort
its queue based on that instead of the 1-second timestamp. Of course the CSN
is generated based on the time the operation began executing, as opposed to
when it completed. Not sure if that's better or worse.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support