hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3990) NN's health report has severe performance problems
Date Thu, 18 Oct 2012 18:16:06 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479207#comment-13479207
] 

Eli Collins commented on HDFS-3990:
-----------------------------------

A null peerHostname just means you don't match, since we also check "hostName" which reported
by the DataNode which the mini cluster explicitly sets we should be good, that's the current
behavior after all right? Ie the only thing we're adding here is an additional hostname field
to check, which is null and we won't check in the tests. Related, would be good to make the
minicluster match real cluster behavior here.
                
> NN's health report has severe performance problems
> --------------------------------------------------
>
>                 Key: HDFS-3990
>                 URL: https://issues.apache.org/jira/browse/HDFS-3990
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Critical
>         Attachments: HDFS-3990.branch-0.23.patch, HDFS-3990.patch, HDFS-3990.patch, HDFS-3990.patch,
HDFS-3990.patch, HDFS-3990.patch, HDFS-3990.patch, hdfs-3990.txt, hdfs-3990.txt
>
>
> The dfshealth page will place a read lock on the namespace while it does a dns lookup
for every DN.  On a multi-thousand node cluster, this often results in 10s+ load time for
the health page.  10 concurrent requests were found to cause 7m+ load times during which time
write operations blocked.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message