RE: IDL management in idl.c

["Howard Chu" <hyc@highlandsun.com>]

> -----Original Message-----
> From: Jonghyuk Choi [mailto:jongchoi@us.ibm.com]

> >alloca() is nice in general, but as I recall it's only reliable
> in GCC. If
> >we can rely on its behavior, then it would be good to use in the
> filterindex
> >patch.
>
> Yes, we should check the availability/reliability of alloca() upon
> configure.
>
> >If the objective here is to increase the sizes, that's another story.
> >Otherwise, the code in filterindex.c is the only place that presents the
> >danger of stack blowout, which is why I focused on that.
>
> Yes, there's not many recursions, but the current BDB_IDL_DB_SIZE seems to
> be small.
> I started looking at this, because of the txn_prepare/txn_commit failure
> reported by Pierangelo early this month.

I've been meaning to check into that, it should be possible to uncover the
problem with a smaller dataset just by setting the BDB_IDL_DB/UM_SIZE much
smaller.
>
> How about the dynamic management of multiple
> small-to-moderate-sized IDLs ?
> If we delare IDL as 'ID** ids;' and allocate ids[k] by BDB_IDL_DB_SIZE
> chunk upon an overflow,
> then the size of IDLs can be very large upon demand.

That starts to sound like the IDL management used in back-ldbm. I didn't
create the scheme used in back-bdb but at a guess, the current design is
aimed at doing something simpler than back-ldbm's multiple IDLs. I've also
thought about whether it would be worthwhile to make this kind of change,
but so far I'm not convinced.
>
> >On a sidenote, I see that in bdb_idl_delete_key we always assume that
> >deleting from a range is a no-op. Perhaps we should shrink the range if
> the
> >deleted item is either the lo or hi bound of the range? Hm, I guess
> there's
> >a bug here as well in the BDB_IDL_MULTI case, since it always uses a
> cursor
> >to delete the given item. If you delete the lo or hi bound of a
> range, the
> >range item will be "broken". Amazing the things you see after being away
> >from the code for a while.

I've rewritten the bdb_idl_delete_key code to handle ranges properly. I
also did a parallel rewrite of bdb_idl_insert_key so it no longer relies
on bdb_idl_fetch_key. This eliminates 256K of stack requirement in the
bdb_idl_insert_key case.

  -- Howard Chu
  Chief Architect, Symas Corp.       Director, Highland Sun
  http://www.symas.com               http://highlandsun.com/hyc
  Symas: Premier OpenSource Development and Support