hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-7168) Use excludedNodes consistently in DFSOutputStream
Date Tue, 30 Sep 2014 00:03:07 GMT
Colin Patrick McCabe created HDFS-7168:

             Summary: Use excludedNodes consistently in DFSOutputStream
                 Key: HDFS-7168
                 URL: https://issues.apache.org/jira/browse/HDFS-7168
             Project: Hadoop HDFS
          Issue Type: Bug
            Reporter: Colin Patrick McCabe
            Assignee: Colin Patrick McCabe

We currently have two separate collections of excluded nodes in the {{DFSOutputStream#DataStreamer}}.
 One is {{DFSOutputStream#failed}}; another is {{DFSOutputStream#excludedNodes}}.  Both of
these collections just deal with blacklisting nodes that we have found to be bad.  We should
just use excludedNodes for both.

We also should make this a per-DFSOutputStream variable, rather than being per-DataStreamer.
 We don't need to forget all this information whenever a DataStreamer is torn down.  Since
{{DFSOutputStream#excludedNodes}} is a Guava cache, nodes will expire out of it once enough
time elapses, so they will not be permanently blacklisted.

We should also remove {{DFSOutputStream#setTestFilename}}, since it is no longer needed now
that we can safely rename streams that are open for write.  And {{DFSOutputStream#getBlock}}
should be synchronized.

This message was sent by Atlassian JIRA

View raw message