hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5111) RM address DNS lookup can cause unnecessary slowness on every JHS page load
Date Wed, 03 Apr 2013 00:05:15 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13620424#comment-13620424

Sandy Ryza commented on MAPREDUCE-5111:

Attached patch uses modifies YarnConfiguration#getRMWebAppHostAndPort to use NetUtils#getConnectAddress,
which avoids trying to resolve the  Verified on a pseudo-distributed cluster that
the patch fixes the page load slowness.
> RM address DNS lookup can cause unnecessary slowness on every JHS page load 
> ----------------------------------------------------------------------------
>                 Key: MAPREDUCE-5111
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5111
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobhistoryserver
>    Affects Versions: 2.0.3-alpha
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>         Attachments: MAPREDUCE-5111.patch
> When I run the job history server locally, every page load takes in the 10s of seconds.
 I profiled the process and discovered that all the extra time was spent inside YarnConfiguration#getRMWebAppURL,
trying to resolve to a hostname.  When I changed my yarn.resourcemanager.address to
localhost, the page load times decreased drastically.
> There's no that we need to perform this resolution on every page load.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message