hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nan Zhu <zhunans...@gmail.com>
Subject Re: what will happen when HDFS restarts but with some dead nodes
Date Wed, 30 Jan 2013 16:45:18 GMT
I think Chen is asking replication lost,  

so, according to Harsh's reply, in safe mode, NN will know all blocks which has less replications
than 3(by default setup) but no less than 1, and after getting out from safe mode, it will
instruct the real replicating works? Hope I understand it correctly

Best,  

-- 
Nan Zhu
School of Computer Science,
McGill University



On Wednesday, 30 January, 2013 at 11:39 AM, Harsh J wrote:

> Yes, if there are missing blocks (i.e. all replicas lost), and the
> block availability threshold is set to its default of 0.999f (99.9%
> availability required), then NN will not come out of safemode
> automatically. You can control this behavior by configuring
> dfs.namenode.safemode.threshold.
> 
> On Wed, Jan 30, 2013 at 10:06 PM, Chen He <airbots@gmail.com (mailto:airbots@gmail.com)>
wrote:
> > Hi Harsh
> > 
> > I have a question. How namenode gets out of safemode in condition of data
> > blocks lost, only administrator? Accordin to my experiences, the NN (0.21)
> > stayed in safemode about several days before I manually turn safemode off.
> > There were 2 blocks lost.
> > 
> > Chen
> > 
> > 
> > On Wed, Jan 30, 2013 at 10:27 AM, Harsh J <harsh@cloudera.com (mailto:harsh@cloudera.com)>
wrote:
> > > 
> > > NN does recalculate new replication work to do due to unavailable
> > > replicas ("under-replication") when it starts and receives all block
> > > reports, but executes this only after out of safemode. When in
> > > safemode, across the HDFS services, no mutations are allowed.
> > > 
> > > On Wed, Jan 30, 2013 at 8:34 AM, Nan Zhu <zhunansjtu@gmail.com (mailto:zhunansjtu@gmail.com)>
wrote:
> > > > Hi, all
> > > > 
> > > > I'm wondering if HDFS is stopped, and some of the machines of the
> > > > cluster
> > > > are moved, some of the block replication are definitely lost for moving
> > > > machines
> > > > 
> > > > when I restart the system, will the namenode recalculate the data
> > > > distribution?
> > > > 
> > > > Best,
> > > > 
> > > > --
> > > > Nan Zhu
> > > > School of Computer Science,
> > > > McGill University
> > > > 
> > > 
> > > 
> > > 
> > > 
> > > --
> > > Harsh J
> > > 
> > 
> > 
> 
> 
> 
> 
> -- 
> Harsh J
> 
> 



Mime
View raw message