hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stephen mulcahy <stephen.mulc...@deri.org>
Subject Re: Hadoop performance - xfs and ext4
Date Fri, 23 Apr 2010 13:12:12 GMT
Andrew Klochkov wrote:
> Hi,
> Just curious - did you try ext3? Can it be faster then ext4? Hadoop wiki
> suggests ext3 as it's used mostly for hadoop clusters:
> http://wiki.apache.org/hadoop/DiskSetup

For completeness, I rebuilt one more time with ext3

mkfs.ext3 -T largefile4 DEV
(mounted with noatime)
gives me a cluster which runs TeraSort in about 22.5 minutes

So ext4 looks like the winner, from a performance perspective, at least 
for running the TeraSort on my cluster with it's specific configuration.


Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
http://di2.deri.ie    http://webstar.deri.ie    http://sindice.com

View raw message