Return-Path: 
 <hdfs-issues-return-144597-apmail-hadoop-hdfs-issues-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 678291872D
	for <apmail-hadoop-hdfs-issues-archive@minotaur.apache.org>;
 Mon,  9 Nov 2015 23:38:11 +0000 (UTC)
Received: (qmail 32322 invoked by uid 500); 9 Nov 2015 23:38:11 -0000
Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org
Received: (qmail 32271 invoked by uid 500); 9 Nov 2015 23:38:11 -0000
Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:hdfs-issues-help@hadoop.apache.org>
List-Unsubscribe: <mailto:hdfs-issues-unsubscribe@hadoop.apache.org>
List-Post: <mailto:hdfs-issues@hadoop.apache.org>
List-Id: <hdfs-issues.hadoop.apache.org>
Reply-To: hdfs-issues@hadoop.apache.org
Delivered-To: mailing list hdfs-issues@hadoop.apache.org
Received: (qmail 32250 invoked by uid 99); 9 Nov 2015 23:38:11 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 09 Nov 2015 23:38:11 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id 0036D2C1F57
	for <hdfs-issues@hadoop.apache.org>; Mon,  9 Nov 2015 23:38:11 +0000 (UTC)
Date: Mon, 9 Nov 2015 23:38:10 +0000 (UTC)
From: "Haohui Mai (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12887662.1442582963000.22170.1447112290998@Atlassian.JIRA>
In-Reply-To: <JIRA.12887662.1442582963000@Atlassian.JIRA>
References: <JIRA.12887662.1442582963000@Atlassian.JIRA>
 <JIRA.12887662.1442582963749@arcas>
Subject: [jira] [Commented] (HDFS-9103) Retry reads on DN failure
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


    [ https://issues.apache.org/jira/browse/HDFS-9103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997662#comment-14997662 ] 

Haohui Mai commented on HDFS-9103:
----------------------------------

Just take a quick skim.

{code}
   template <class MutableBufferSequence, class Handler>
   void AsyncPreadSome(size_t offset, const MutableBufferSequence &buffers,
-                      const std::set<std::string> &excluded_datanodes,
-                      const Handler &handler);
+                      const Handler &handler,
+                      std::shared_ptr<NodeExclusionRule> optional_rule_override = nullptr);

151	  std::shared_ptr<NodeExclusionRule> rule = optional_exclude_rule != nullptr ?
152	                                            optional_exclude_rule : bad_node_tracker_;
{code}

Let's remove the default argument and make it explicit for the caller.

{code}
+  /* prune orphaned DNs from list periodically */
+  if(remove_counter_++ % 1024 == 0 && remove_counter_ != 0) {
+    RemoveAllExpired();
+    remove_counter_ = 0;
+  }
{code}

There is no need to do this. You can sort the lists by expiration time so that the fast path will always return in the first iteration.

> Retry reads on DN failure
> -------------------------
>
>                 Key: HDFS-9103
>                 URL: https://issues.apache.org/jira/browse/HDFS-9103
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>            Reporter: Bob Hansen
>            Assignee: James Clampffer
>             Fix For: HDFS-8707
>
>         Attachments: HDFS-9103.1.patch, HDFS-9103.2.patch, HDFS-9103.HDFS-8707.006.patch, HDFS-9103.HDFS-8707.007.patch, HDFS-9103.HDFS-8707.008.patch, HDFS-9103.HDFS-8707.3.patch, HDFS-9103.HDFS-8707.4.patch, HDFS-9103.HDFS-8707.5.patch
>
>
> When AsyncPreadSome fails, add the failed DataNode to the excluded list and try again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)