hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Runping Qi (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-3589) DFS should detect slow links(nodes) and avoid them
Date Wed, 18 Jun 2008 01:20:45 GMT
DFS should detect slow links(nodes) and avoid them

                 Key: HADOOP-3589
                 URL: https://issues.apache.org/jira/browse/HADOOP-3589
             Project: Hadoop Core
          Issue Type: Bug
          Components: dfs
            Reporter: Runping Qi

The current DFS does not detect slow links (nodes).
Thus, when a node or its network link is slow, it may affect the overall system performance
Specifically, when a map job needs to read data from such a node, it may progress 10X slower.
And when a DFS data node pipeline consists of such a node, the write performance degrades
This may lead to some long tails for map/reduce jobs. We have experienced such behaviors quite

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message