hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bertrand Dechoux <decho...@gmail.com>
Subject Re: what will happen when HDFS restarts but with some dead nodes
Date Wed, 30 Jan 2013 17:10:32 GMT
Well, the documentation is more explicite.

Specifies the percentage of blocks that should satisfy the minimal
replication requirement defined by* dfs.namenode.replication.min*.

Which happens to be 1 by default but doesn't need to stay that way.

Regards

Bertrand

On Wed, Jan 30, 2013 at 5:45 PM, Nan Zhu <zhunansjtu@gmail.com> wrote:

>  I think Chen is asking replication lost,
>
> so, according to Harsh's reply, in safe mode, NN will know all blocks
> which has less replications than 3(by default setup) but no less than 1,
> and after getting out from safe mode, it will instruct the real replicating
> works? Hope I understand it correctly
>
> Best,
>
> --
> Nan Zhu
> School of Computer Science,
> McGill University
>
>
> On Wednesday, 30 January, 2013 at 11:39 AM, Harsh J wrote:
>
> Yes, if there are missing blocks (i.e. all replicas lost), and the
> block availability threshold is set to its default of 0.999f (99.9%
> availability required), then NN will not come out of safemode
> automatically. You can control this behavior by configuring
> dfs.namenode.safemode.threshold.
>
> On Wed, Jan 30, 2013 at 10:06 PM, Chen He <airbots@gmail.com> wrote:
>
> Hi Harsh
>
> I have a question. How namenode gets out of safemode in condition of data
> blocks lost, only administrator? Accordin to my experiences, the NN (0.21)
> stayed in safemode about several days before I manually turn safemode off.
> There were 2 blocks lost.
>
> Chen
>
>
> On Wed, Jan 30, 2013 at 10:27 AM, Harsh J <harsh@cloudera.com> wrote:
>
>
> NN does recalculate new replication work to do due to unavailable
> replicas ("under-replication") when it starts and receives all block
> reports, but executes this only after out of safemode. When in
> safemode, across the HDFS services, no mutations are allowed.
>
> On Wed, Jan 30, 2013 at 8:34 AM, Nan Zhu <zhunansjtu@gmail.com> wrote:
>
> Hi, all
>
> I'm wondering if HDFS is stopped, and some of the machines of the
> cluster
> are moved, some of the block replication are definitely lost for moving
> machines
>
> when I restart the system, will the namenode recalculate the data
> distribution?
>
> Best,
>
> --
> Nan Zhu
> School of Computer Science,
> McGill University
>
>
>
>
> --
> Harsh J
>
>
>
>
> --
> Harsh J
>
>
>

Mime
View raw message