hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7168) Use excludedNodes consistently in DFSOutputStream
Date Fri, 06 Jan 2017 21:52:59 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Junping Du updated HDFS-7168:
    Target Version/s:   (was: 2.8.0)

> Use excludedNodes consistently in DFSOutputStream
> -------------------------------------------------
>                 Key: HDFS-7168
>                 URL: https://issues.apache.org/jira/browse/HDFS-7168
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.7.0
>            Reporter: Colin P. McCabe
>            Assignee: Colin P. McCabe
>         Attachments: HDFS-7168.001.patch
> We currently have two separate collections of excluded nodes in the {{DFSOutputStream#DataStreamer}}.
 One is {{DFSOutputStream#failed}}; another is {{DFSOutputStream#excludedNodes}}.  Both of
these collections just deal with blacklisting nodes that we have found to be bad.  We should
just use excludedNodes for both.
> We also should make this a per-DFSOutputStream variable, rather than being per-DataStreamer.
 We don't need to forget all this information whenever a DataStreamer is torn down.  Since
{{DFSOutputStream#excludedNodes}} is a Guava cache, nodes will expire out of it once enough
time elapses, so they will not be permanently blacklisted.
> We should also remove {{DFSOutputStream#setTestFilename}}, since it is no longer needed
now that we can safely rename streams that are open for write.  And {{DFSOutputStream#getBlock}}
should be synchronized.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message