hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haohui Mai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9103) Retry reads on DN failure
Date Mon, 09 Nov 2015 23:38:10 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14997662#comment-14997662
] 

Haohui Mai commented on HDFS-9103:
----------------------------------

Just take a quick skim.

{code}
   template <class MutableBufferSequence, class Handler>
   void AsyncPreadSome(size_t offset, const MutableBufferSequence &buffers,
-                      const std::set<std::string> &excluded_datanodes,
-                      const Handler &handler);
+                      const Handler &handler,
+                      std::shared_ptr<NodeExclusionRule> optional_rule_override = nullptr);

151	  std::shared_ptr<NodeExclusionRule> rule = optional_exclude_rule != nullptr ?
152	                                            optional_exclude_rule : bad_node_tracker_;
{code}

Let's remove the default argument and make it explicit for the caller.

{code}
+  /* prune orphaned DNs from list periodically */
+  if(remove_counter_++ % 1024 == 0 && remove_counter_ != 0) {
+    RemoveAllExpired();
+    remove_counter_ = 0;
+  }
{code}

There is no need to do this. You can sort the lists by expiration time so that the fast path
will always return in the first iteration.

> Retry reads on DN failure
> -------------------------
>
>                 Key: HDFS-9103
>                 URL: https://issues.apache.org/jira/browse/HDFS-9103
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>            Reporter: Bob Hansen
>            Assignee: James Clampffer
>             Fix For: HDFS-8707
>
>         Attachments: HDFS-9103.1.patch, HDFS-9103.2.patch, HDFS-9103.HDFS-8707.006.patch,
HDFS-9103.HDFS-8707.007.patch, HDFS-9103.HDFS-8707.008.patch, HDFS-9103.HDFS-8707.3.patch,
HDFS-9103.HDFS-8707.4.patch, HDFS-9103.HDFS-8707.5.patch
>
>
> When AsyncPreadSome fails, add the failed DataNode to the excluded list and try again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message