hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stephen mulcahy <stephen.mulc...@deri.org>
Subject Re: Hadoop performance - xfs and ext4
Date Fri, 23 Apr 2010 13:12:12 GMT
Andrew Klochkov wrote:
> Hi,
> 
> Just curious - did you try ext3? Can it be faster then ext4? Hadoop wiki
> suggests ext3 as it's used mostly for hadoop clusters:
> 
> http://wiki.apache.org/hadoop/DiskSetup

For completeness, I rebuilt one more time with ext3

mkfs.ext3 -T largefile4 DEV
(mounted with noatime)
gives me a cluster which runs TeraSort in about 22.5 minutes

So ext4 looks like the winner, from a performance perspective, at least 
for running the TeraSort on my cluster with it's specific configuration.

-stephen

-- 
Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
http://di2.deri.ie    http://webstar.deri.ie    http://sindice.com

Mime
View raw message