hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sujee Maniyam <su...@sujee.net>
Subject using sysbench to evaluate disk performance for hdfs
Date Fri, 09 Apr 2010 03:22:31 GMT
Wondering if I can use any tool like sysbench to get a _approximate
idea_ of performance of various disk setups (RAID-0, RAID-1,  ext4,
xfs ...etc) that would be used by HDFS.
(I do understand the the real performance of HDFS/HBase depends on the
final overall system and workload.  )

for example take sysbench:
$ sysbench --num-threads=16 --test=fileio --file-total-size=3G
--file-test-mode=rndrw prepare
$ sysbench --num-threads=16 --test=fileio --file-total-size=3G
--file-test-mode=rndrw run
$ sysbench --num-threads=16 --test=fileio --file-total-size=3G
--file-test-mode=rndrw cleanup

here is what the above means:
---
Number of threads: 16
Extra file open flags: 0
128 files, 24Mb each
3Gb total file size
Block size 16Kb
Number of random requests for random IO: 10000
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
--

One thing jumps out at me is the block_size, it needs to be upped to
64M.  Any other params I should tweak?

Would like to hear from the community what they have used to bench disk i/o.

thanks
Sujee
http://sujee.net

Mime
View raw message