hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcos Ortiz <mlor...@uci.cu>
Subject Re: our experiences with various filesystems and tuning options
Date Tue, 10 May 2011 13:14:55 GMT
On 05/10/2011 06:56 AM, Jonathan Disher wrote:
> In a previous life, I've had extreme problems with XFS, including 
> kernel panics and data loss under high load.
>
> Those were database servers, not Hadoop nodes, and it was a few years 
> ago.  But, ext3/ext4 seems to be stable enough, and it's more widely 
> supported, so it's my preference.
>
> -j
>
> On May 10, 2011, at 3:59 AM, Rita wrote:
>
>> I keep asking because I wasn't able to use a XFS filesystem larger 
>> than 3-4TB. If the XFS file system is larger than 4TB hdfs won't 
>> recognize the space. I am on a 64bit RHEL 5.3 host.
>>
>>
>> On Tue, May 10, 2011 at 6:30 AM, Will Maier <wcmaier@hep.wisc.edu 
>> <mailto:wcmaier@hep.wisc.edu>> wrote:
>>
>>     On Tue, May 10, 2011 at 12:03:09AM -0400, Rita wrote:
>>     > what filesystem are they using and what is the size of each
>>     filesystem?
>>
>>     It sounds nuts, but each disk has its own ext3 filesystem. Beyond
>>     switching to
>>     the deadline IO scheduler, we haven't done much tuning/tweaking.
>>     A script runs
>>     every ten minutes to test all of the data mounts and reconfigure
>>     hdfs-site.xml
>>     and restart the datanode if necessary. So far, this approach has
>>     allowed us to
>>     avoid loss of space to RAID without correlating the risk of disk
>>     failure by
>>     building larger RAID0s.
>>
>>     In the future, we expect to deprecate the script and rely on the
>>     datanode process
>>     itself to handle missing/failing disks.
>>
>>     --
>>
>>     Will Maier - UW High Energy Physics
>>     cel: 608.438.6162 <tel:608.438.6162>
>>     tel: 608.263.9692 <tel:608.263.9692>
>>     web: http://www.hep.wisc.edu/~wcmaier/
>>     <http://www.hep.wisc.edu/%7Ewcmaier/>
>>
>>
>>
>>
>> -- 
>> --- Get your facts first, then you can distort them as you please.--
>
Jonathan, I had the same issues on my PostgreSQL servers, and the main 
issues was given by the kernel version that I was using.
I upgrade the kernel to the last version supported by Red Hat, and 
everything worked OK.

My prefered filesystem is ZFS, It's a shame that Linux support is very 
inmature yet. For that reason, I changed my PostgreSQL hosts to 
FreeBSD-8.0 to use
ZFS like filesystem and it's really rocks.

Had anyone tested a Hadoop cluster with this filesystem?
On Solaris or FreeBSD?

Regards

-- 
Marcos Luís Ortíz Valmaseda
  Software Engineer (Large-Scaled Distributed Systems)
  University of Information Sciences,
  La Habana, Cuba
  Linux User # 418229
  http://about.me/marcosortiz


Mime
View raw message