hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
Date Thu, 28 Sep 2017 02:18:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183572#comment-16183572

Allen Wittenauer commented on YARN-7207:

Actually, let me expand on that a bit, because we're running directly into "better practices"
in a space that many may not understand the details.

A process requests a host resolution of a name/ip that is associated with the machine that
the process is running on (localhost, whatever hostname() returns, etc, etc).  That resolution
should be going through the local cache (nscd, sssd, lookupd, whatever).  That cache should
be configured such that it resolves through files (e.g., /etc/hosts) and then through DNS.
 /etc/hosts SHOULD have all known names and IPs for the local machine, eliminating the need
for any DNS lookup. 

A misconfigured machine will either by not having a cache or having the cache misconfigured
ask DNS or some other naming service first.  This will *definitely* impact system performance.
But it's also a misconfiguration; this won't just impact YARN but pretty much every single
process on the box.  Need to write to syslog?  Yup, gonna ask DNS....

> Cache the local host name when getting application list in RM
> -------------------------------------------------------------
>                 Key: YARN-7207
>                 URL: https://issues.apache.org/jira/browse/YARN-7207
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: RM
>    Affects Versions: 3.1.0
>            Reporter: Yufei Gu
>            Assignee: Yufei Gu
>         Attachments: YARN-7207.001.patch, YARN-7207.002.patch
> {{getLocalHostName()}} is invoked for generating the report for each application, which
means it is called 1000 times for each {{getApplications()}} if there are 1000 apps in RM.
Some user got a performance issue when {{getLocalHostName()}} is slow under some network envs.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message