hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8008) Support client-side back off when the datanodes are congested
Date Wed, 01 Apr 2015 18:16:54 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14391137#comment-14391137

Chris Nauroth commented on HDFS-8008:

This patch looks good, Haohui.  I have just a few small comments.

If {{backOffIfNecessary}} throws {{InterruptedException}}, then this will be passed along
to {{setLastException}} and eventually propagate to the caller.  This is inconsistent with
existing interruption handling logic in this loop, which catches {{InterruptedException}}
and then allows execution to proceed without propagating the exception.  (See lines 383 and
441.)  Shall we do the same here?

On a side note, there is a lot of swallowing of {{InterruptedException}} in this code.  It
probably ought to do {{Thread.currentThread().interrupt()}}, but that's not related to your
current patch.

Minor nitpick: the hyperlink got truncated in this comment.

   * This function sleeps for a certain amount of time when the writing
   * pipeline is congested. The function calculates the time based on a
   * decorrelated filter which is available at {@link http://www
   * .com/2015/03/backoff.html}.


> Support client-side back off when the datanodes are congested
> -------------------------------------------------------------
>                 Key: HDFS-8008
>                 URL: https://issues.apache.org/jira/browse/HDFS-8008
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>         Attachments: HDFS-8008.000.patch, HDFS-8008.001.patch
> HDFS-7270 introduces the mechanism for DataNode to signal congestions. DFSClient should
be able to recognize the signals and back off.

This message was sent by Atlassian JIRA

View raw message