hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11090) Leave safemode immediately if all blocks have reported in
Date Tue, 15 Nov 2016 23:06:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668671#comment-15668671

Andrew Wang commented on HDFS-11090:

Hi Konst, thanks for the insight,

bq. Andrew, "empty cluster" as a special case was always there. It was somewhat tempered when
DataNodes where added to SafeMode conditions. I guess even if there are no blocks in the cluster
you would at least want to wait for DN registrations to assure you have anything to write

You've exactly identified what I'd like to do with this JIRA. We have an empty cluster with
the min datanodes threshold set to a non-zero value. HDFS still goes into safemode extension

> Leave safemode immediately if all blocks have reported in
> ---------------------------------------------------------
>                 Key: HDFS-11090
>                 URL: https://issues.apache.org/jira/browse/HDFS-11090
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 2.7.3
>            Reporter: Andrew Wang
>            Assignee: Yiqun Lin
>         Attachments: HDFS-11090.001.patch
> Startup safemode is triggered by two thresholds: % blocks reported in, and min # datanodes.
It's extended by an interval (default 30s) until these two thresholds are met.
> Safemode extension is helpful when the cluster has data, and the default % blocks threshold
(0.99) is used. It gives DNs a little extra time to report in and thus avoid unnecessary replication
> However, we can leave startup safemode early if 100% of blocks have reported in.
> Note that operators sometimes change the % blocks threshold to > 1 to never automatically
leave safemode. We should maintain this behavior.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message