hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vikas Srivastava <vikas_srivast...@apple.com>
Subject Re: How quickly can I increase the number of replicas?
Date Mon, 30 Jun 2014 06:29:17 GMT
Hey,

You can set the below properties to speed up replication

<property>
  <!-- 100Mbit/s -->
  <name>dfs.balance.bandwidthPerSec</name>
  <value>131072000</value>
</property>
<property>
  <name>dfs.max-repl-streams</name>
  <value>50</value>
</property>

Thanks
Vikas

On Jun 29, 2014, at 10:50 PM, 조주일 <tjstory@kgrid.co.kr> wrote:

>  
> My cluser version hdfs 2.2 stable ( 2 ha namenodes, 10 datanodes)
> 
>  
> I was command bin/hdfs dfs -setrep -R 2 / ( replication 1 to 2 )
> 
>  
> I found that HDFS is actually replicating the under replicated blocks but it works very
slowly. 
> 
> HDFS performs the replication about 1 block per second.
> 
> I have about 400000 under replicated blocks so it will take about 4 more days.
> 
> Is there any way to speed it up?
> 


Mime
View raw message