Re: High load times with mdb

[Bill MacAllister <whm@stanford.edu>]

--On June 27, 2013 1:40:16 AM -0700 Howard Chu <hyc@symas.com> wrote:

>> I tried a load on an ext4 system with options 'rw,noatime, user_xattr,
>> barrier=1, data=writeback' and got a load time of 01h40m06s.  This is
>> the best time I have gotten so far loading on ext4.
> 
> Did you try commit=60 barrier=0 ?

Here are the details of a test using commit=60 barrier=0.

    mkfs -t ext4 \
        -O "^flex_bg ^huge_file ^uninit_bg ^dir_nlink ^extra_isize ^extent"
    mount -t ext4 -o rw,noatime,barrier=0,commit=60,data=writeback
    Filesystem features: has_journal ext_attr resize_inode dir_index
                         filetype needs_recovery sparse_super
                         large_file
    mount optiions: rw, noatime, user_xattr, commit=60, barrier=0,
                    data=writeback
    elapsed 01h40m55s spd 211.0 k/s

>> I ended up writing a script that creates an ext2 file systems, loads
>> the backend, umounts the partition, converts it to ext4 journaling,
>> and then mounts the partition again.  This will allow me to continue
>> with the server rebuilds, but it is a pretty ugly hack.

I am now getting close to ext2 performance using ext3, but ext4 is
consistently too slow in all of my tests.  Here are the results of the
fastest ext3 and fastest ext4 tests.

  * mkfs -t ext3 -O has_journal
    mount -t ext3 -o rw,noatime,data=writeback
    Filesystem features: has_journal ext_attr resize_inode dir_index
                         filetype needs_recovery sparse_super
                         large_file
    mount options: rw, noatime, errors=continue, user_xattr, acl,
                   barrier=1, data=writeback
    elapsed 22m03s spd 965.6 k/s

  * mkfs -t ext4 -O "^flex_bg ^huge_file ^uninit_bg ^dir_nlink ^extra_isize"
    mount -t ext4 -o rw,noatime,data=writeback
    Filesystem features: has_journal ext_attr resize_inode dir_index
                         filetype needs_recovery extent sparse_super
                         large_file
    mount options: rw, noatime, user_xattr, barrier=1, data=writeback
    elapsed 01h32m19s spd 230.6 k/s

During a load the status display would stall periodically.  The worse the
load time the more frequently the display stalled and the longer it stalled
for.  I guessing that this is flushing data to the disk.  I am also
guessing that since mdb is using memory mapped files that some tuning of
memory management might help improve performance.  I am not familiar with
the tuning knobs there so any pointers would be appreciated.

Bill

-- 

Bill MacAllister
Infrastructure Delivery Group, Stanford University