hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Parks" <davidpark...@yahoo.com>
Subject What's the best disk configuration for hadoop? SSD's Raid levels, etc?
Date Sat, 11 May 2013 06:30:11 GMT
We've got a cluster of 10x 8core/24gb nodes, currently with 1 4TB disk (3
disk slots max), they chug away ok currently, only slightly IO bound on
average.

 

I'm going to upgrade the disk configuration at some point (we do need more
space on HDFS) and I'm thinking about what's best hardware-wise:

 

.         Would it be wise to use one of the three disk slots for a 1TB SSD?
I wouldn't use it for HDFS, but for map-output and sorting it might make a
big difference no?

.         If I put in either 1 or 2 more 4TB disks for HDFS, should I RAID-0
them for speed, or will HDFS balance well across multiple partitions on its
own?

.         Would anyone suggest 3 4TB disks and a RAID-5 configuration to
guard against disk replacements over the above options?

 

Dave

 


Mime
View raw message