Re: (ITS#3851) Berkeley DB Scalability Patch

[quanah@symas.com]

--On Thursday, September 01, 2005 10:03 AM -0400 Jong-Hyuk 
<jongchoi@watson.ibm.com> wrote:

>
>>
>> Final load time with the BDB scalability pach:
>>
>> cgi3-linux:/tmp/ldif# time slapadd -q -l 2M-peons.ldif
>> [########################################] (100%) 02w05h  2000002 obj
>> \HH
>> 6485.120u 5834.390s 341:15:40.01 1.0%   0+0k 0+0io 981489pf+0w
>>
>>
>> So more than twice as long the 152:22:40.95 time it took to load the
>> database without the patch.
>
>
> I recommend you to confirm your system setup first. Assuming that you are
> experimenting on a reasonably up-to-date platform, more than 6 days for
> adding 2 million entries seems too much even without the BDB scalability
> patch. When I compare your numbers with my results, your configuration is
> 6.6x slower than mine originally (before applying BDB scalability patch).
> Fair comparison could follow after first bring the baseline performance
> in sync.

After moving to a new server, and running the 13 indices case, I do see 
some improvement in time to add via your patch.

With the patch:

/tmp/ldif# time slapadd -q -l 2M-peons.ldif
[########################################] (100%) 03d04h  2000002 obj
5123.990u 2264.730s 76:14:54.16 2.6%    0+0k 0+0io 423456pf+0w


Without the patch:

/tmp/ldif# time slapadd -q -l 2M-peons.ldif
[########################################] (100%) 03d15h  2000002 obj
205538.150u 1335.460s 87:48:16.93 65.4% 0+0k 0+0io 438073pf+0w


So a total savings of approximately 11 hours 30 minutes.

I will go redo the 21 indices case.

--Quanah

--
Quanah Gibson-Mount
Product Engineer
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>