hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ferdy Galema <ferdy.gal...@kalooga.com>
Subject Re: our experiences with various filesystems and tuning options
Date Fri, 06 May 2011 07:44:19 GMT
No unfortunately not, we couldn't because of our kernel versions.

On 05/06/2011 04:00 AM, ShengChang Gu wrote:
> Many thanks.
>
> We use xfs all the time.Have you try the ext4 filesystem?
>
> 2011/5/6 Ferdy Galema <ferdy.galema@kalooga.com
> <mailto:ferdy.galema@kalooga.com>>
>
>     Hi,
>
>     We've performed tests for ext3 and xfs filesystems using different
>     settings. The results might be useful for anyone else.
>
>     The datanode cluster consists of 15 slave nodes, each equipped
>     with 1Gbit ethernet, X3220@2.40GHz quadcores and 4x1TB disks. The
>     disk read speeds vary from about 90 to 130MB/s. (Tested using
>     hdparm -t).
>
>     Hadoop: Cloudera CDH3u0 (4 concurrent mappers / node)
>     OS: Linux version 2.6.18-238.5.1.el5
>     (mockbuild@builder10.centos.org
>     <mailto:mockbuild@builder10.centos.org>) (gcc version 4.1.2
>     20080704 (Red Hat 4.1.2-50))
>
>     #our command
>     for i in `seq 1 10`; do ./hadoop jar
>     ../hadoop-examples-0.20.2-cdh3u0.jar randomwriter
>     -Ddfs.replication=1 /rand$i && ./hadoop fs -rmr /rand$i/_logs
>     /rand$i/_SUCCESS && ./hadoop distcp -Ddfs.replication=1 /rand$i
>     /rand-copy$i; done
>
>     Our benchmark consists of a standard random-writer job followed by
>     a distcp of the same data, both using a replication of 1. This is
>     to make sure only the disks get hit. Each benchmark is ran several
>     times for every configuration. Because of the occasional hickup, I
>     will list both the average and the fastest times for each
>     configuration. I read the execution times off the jobtracker.
>
>     The configurations (with exection times in seconds of Avg-writer /
>     Min-writer / Avg-distcp / Min-distcp)
>     ext3-default 158 / 136 / 411 / 343
>     ext3-tuned 159 / 132 / 330 / 297
>     ra1024 ext3-tuned 159 / 132 / 292 / 264
>     ra1024 xfs-tuned 128 / 122 / 220 / 202
>
>     To explain, ext3-tuned is with tuned mount options
>     [noatime,nodiratime,data=writeback,rw] and ra1024 means a
>     read-ahead buffer of 1024 blocks. The xfs disks are created using
>     mkfs options [size=128m,lazy-count=1] and mount options
>     [noatime,nodiratime,logbufs=8].
>
>     In conclusion it seems that using tuned xfs filesystems combined
>     with increased read-ahead buffers increased our basic hdfs
>     performance with about 10% (random-writer) to 40% (distcp).
>
>     Hopefully this is useful to anyone. Although I won't be performing
>     more tests soon I'd be happy to provide more details.
>     Ferdy.
>
>
>
>
> -- 
> 阿昌

Mime
View raw message