hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Shvachko <...@yahoo-inc.com>
Subject Re: HDFS Question re adding additional storage
Date Fri, 03 Aug 2007 19:38:34 GMT
There is no down time to add new nodes. You don't even need to stop the 
cluster.
Just start datanodes on the new nodes and they will join the cluster.
Hadoop does not have automatic rebalancing as of today though (coming 
soon), meaning the new nodes will
fill upon new data is added but their utilization will stay lower than 
the utilization of the old nodes for some time.
I'd say buy later if you don't need them now.

Regards,
Konstantin

C G wrote:

>Is it possible to additional space to HDFS (in the form of new datanodes) with minimal/no
fuss?  In other words, if I have 8T across 16 machines, and I want to go to 16T across 32
machines, can I roll in new machines easily, or do I need to plan considerable downtime to
rebuild things and move data around?
>   
>  There are obvious implications here for how big an initial system to build, and the
costs associated with buying now and buying later.
>   
>  Thanks,
>  C G
>   
>
>       
>---------------------------------
>Got a little couch potato? 
>Check out fun summer activities for kids.
>       
>---------------------------------
>Got a little couch potato? 
>Check out fun summer activities for kids.
>  
>


Mime
View raw message