Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 12 May 2016 22:22:13 +0000 (UTC)
From: "Sangjin Lee (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12644597.1366895637000.181238.1463091733441@Atlassian.JIRA>
In-Reply-To: <JIRA.12644597.1366895637000@Atlassian.JIRA>
References: <JIRA.12644597.1366895637000@Atlassian.JIRA> <JIRA.12644597.1366895637329@arcas>
Subject: [jira] [Commented] (HDFS-4754) Add an API in the namenode to mark a
 datanode as stale
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 12 May 2016 22:22:15 -0000


    [ https://issues.apache.org/jira/browse/HDFS-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15282157#comment-15282157 ] 

Sangjin Lee commented on HDFS-4754:
-----------------------------------

There has been no movement on this for a while. We'll need to move it out of scope for 2.8.0 soon. Let me know if you disagree.

> Add an API in the namenode to mark a datanode as stale
> ------------------------------------------------------
>
>                 Key: HDFS-4754
>                 URL: https://issues.apache.org/jira/browse/HDFS-4754
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client, namenode
>            Reporter: Nicolas Liochon
>            Assignee: Nicolas Liochon
>            Priority: Critical
>              Labels: BB2015-05-TBR
>         Attachments: 4754.v1.patch, 4754.v2.patch, 4754.v4.patch, 4754.v4.patch
>
>
> There is a detection of the stale datanodes in HDFS since HDFS-3703, with a timeout, defaulted to 30s.
> There are two reasons to add an API to mark a node as stale even if the timeout is not yet reached:
>  1) ZooKeeper can detect that a client is dead at any moment. So, for HBase, we sometimes start the recovery before a node is marked staled. (even with reasonable settings as: stale: 20s; HBase ZK timeout: 30s
>  2) Some third parties could detect that a node is dead before the timeout, hence saving us the cost of retrying. An example or such hw is Arista, presented here by [~tsuna] http://tsunanet.net/~tsuna/fsf-hbase-meetup-april13.pdf, and confirmed in HBASE-6290.
> As usual, even if the node is dead it can comeback before the 10 minutes limit. So I would propose to set a timebound. The API would be
> namenode.markStale(String ipAddress, int port, long durationInMs);
> After durationInMs, the namenode would again rely only on its heartbeat to decide.
> Thoughts?
> If there is no objections, and if nobody in the hdfs dev team has the time to spend some time on it, I will give it a try for branch 2 & 3.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org