hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcos Ortiz <mlor...@uci.cu>
Subject Re: our experiences with various filesystems and tuning options
Date Tue, 10 May 2011 13:06:10 GMT
On 05/10/2011 06:29 AM, Rita wrote:
> I keep asking because I wasn't able to use a XFS filesystem larger 
> than 3-4TB. If the XFS file system is larger than 4TB hdfs won't 
> recognize the space. I am on a 64bit RHEL 5.3 host.
> On Tue, May 10, 2011 at 6:30 AM, Will Maier <wcmaier@hep.wisc.edu 
> <mailto:wcmaier@hep.wisc.edu>> wrote:
>     On Tue, May 10, 2011 at 12:03:09AM -0400, Rita wrote:
>     > what filesystem are they using and what is the size of each
>     filesystem?
>     It sounds nuts, but each disk has its own ext3 filesystem. Beyond
>     switching to
>     the deadline IO scheduler, we haven't done much tuning/tweaking. A
>     script runs
>     every ten minutes to test all of the data mounts and reconfigure
>     hdfs-site.xml
>     and restart the datanode if necessary. So far, this approach has
>     allowed us to
>     avoid loss of space to RAID without correlating the risk of disk
>     failure by
>     building larger RAID0s.
>     In the future, we expect to deprecate the script and rely on the
>     datanode process
>     itself to handle missing/failing disks.
>     --
>     Will Maier - UW High Energy Physics
>     cel: 608.438.6162 <tel:608.438.6162>
>     tel: 608.263.9692 <tel:608.263.9692>
>     web: http://www.hep.wisc.edu/~wcmaier/
>     <http://www.hep.wisc.edu/%7Ewcmaier/>
> -- 
> --- Get your facts first, then you can distort them as you please.--
I saw this problem before with 64 bits version of Red Hat EL 5.3.
Which is the kernel version that you are using?

Can you upgrade the system to 5.5 or to 6.0? There are a lot of bugs 
corrections and performance gaining with these releases.
Another issue is that since the 5.4 vesion, Red Hat added preliminary 
XFS support specifically to address the need for filesystem more
large, and their RHEL 6 release treats it as a fully supported 
filesystem on par with ext3 and ext4.

One last issue: XFS can handle files greather than 16 TB. The primary 
problem is the tools to read and write those files. (ext4 virtually too can
handle this huge files, but the problems is on the mkfs utility that is 
not optimized for this)


Marcos Luís Ortíz Valmaseda
  Software Engineer (Large-Scaled Distributed Systems)
  University of Information Sciences,
  La Habana, Cuba
  Linux User # 418229

View raw message