hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5111) RM address DNS lookup can cause unnecessary slowness on every JHS page load
Date Wed, 03 Apr 2013 00:05:15 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13620424#comment-13620424
] 

Sandy Ryza commented on MAPREDUCE-5111:
---------------------------------------

Attached patch uses modifies YarnConfiguration#getRMWebAppHostAndPort to use NetUtils#getConnectAddress,
which avoids trying to resolve the 0.0.0.0.  Verified on a pseudo-distributed cluster that
the patch fixes the page load slowness.
                
> RM address DNS lookup can cause unnecessary slowness on every JHS page load 
> ----------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5111
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5111
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobhistoryserver
>    Affects Versions: 2.0.3-alpha
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>         Attachments: MAPREDUCE-5111.patch
>
>
> When I run the job history server locally, every page load takes in the 10s of seconds.
 I profiled the process and discovered that all the extra time was spent inside YarnConfiguration#getRMWebAppURL,
trying to resolve 0.0.0.0 to a hostname.  When I changed my yarn.resourcemanager.address to
localhost, the page load times decreased drastically.
> There's no that we need to perform this resolution on every page load.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message