hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Duxbury <br...@rapleaf.com>
Subject Re: Optimal Filesystem (and Settings) for HDFS
Date Tue, 19 May 2009 21:50:57 GMT
We use XFS for our data drives, and we've had somewhat mixed results.  
One of the biggest pros is that XFS has more free space than ext3,  
even with the reserved space settings turned all the way to 0.  
Another is that you can format a 1TB drive as XFS in about 0 seconds,  
versus minutes for ext3. This makes it really fast to kickstart our  
worker nodes.

We have seen some weird stuff happen though when machines run out of  
memory, apparently because the XFS driver does something odd with  
kernel memory. When this happens, we end up having to do some fscking  
before we can get that node back online.

As far as outright performance, I actually *did* do some tests of xfs  
vs ext3 performance on our cluster. If you just look at a single  
machine's local disk speed, you can write and read noticeably faster  
when using XFS instead of ext3. However, the reality is that this  
extra disk performance won't have much of an effect on your overall  
job completion performance, since you will find yourself network  
bottlenecked well in advance of even ext3's performance.

The long and short of it is that we use XFS to speed up our new  
machine deployment, and that's it.


On May 18, 2009, at 10:31 AM, Alex Loddengaard wrote:

> I believe Yahoo! uses ext3, though I know other people have said  
> that XFS
> has performed better in various benchmarks.  We use ext3, though we  
> haven't
> done any benchmarks to prove its worth.
> This question has come up a lot, so I think it'd be worth doing a  
> benchmark
> and writing up the results.  I haven't been able to find a detailed  
> analysis
> / benchmark writeup comparing various filesystems, unfortunately.
> Hope this helps,
> Alex
> On Mon, May 18, 2009 at 8:54 AM, Bob Schulze  
> <b.schulze@ecircle.com> wrote:
>> We are currently rebuilding our cluster - has anybody  
>> recommendations on
>> the underlaying file system? Just standard Ext3?
>> I could imagine that the block size could be larger than its  
>> default...
>> Thx for any tips,
>>        Bob

View raw message