hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luca Pireddu <pire...@crs4.it>
Subject Re: HDFS using SAN
Date Thu, 18 Oct 2012 12:32:57 GMT
On 10/18/2012 02:21 AM, Pamecha, Abhishek wrote:
> Tom
> Do you mean you are using GPFS instead of HDFS? Also, if you can share,
> are you deploying it as DAS set up or a SAN?
> Thanks,
> Abhishek

Though I don't think I'd buy a SAN for a new Hadoop cluster, we have a 
SAN and are using it *instead of HDFS* with a small/medium Hadoop 
MapReduce cluster (up to 100 nodes or so, depending on our need).  We 
still use the local node disks for intermediate data (mapred local 
storage).  Although this set-up does limit our possibility to scale to a 
large number of nodes, that's not a concern for us.  On the plus, we 
gain the flexibility to be able to share our cluster with non-Hadoop 
users at our centre.

Luca Pireddu
CRS4 - Distributed Computing Group
Loc. Pixina Manna Edificio 1
09010 Pula (CA), Italy
Tel: +39 0709250452

View raw message