hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4024) YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat
Date Thu, 06 Aug 2015 18:23:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-4024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660559#comment-14660559

Sunil G commented on YARN-4024:

Hi [~leftnoteasy]
Thank you for bringing this, which is a potential area to cause delay n heartbeat.

I feel we can cache IP addresses in RM side during NM registration, and can look into this
storage as needed. Also apart from switch on/off this, cud we try to resolve IP if this cache
lookup fails and can add to this list. 
So a basic lookup in cache during heartbeat, and upon failure we can resolve using existing
way. A possible distant problem is that some one uses proxy server ahead of NM and which changes
IP every time. I feel it is too negative case.

How do you feel?

> YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat
> ----------------------------------------------------------------------
>                 Key: YARN-4024
>                 URL: https://issues.apache.org/jira/browse/YARN-4024
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
> Currently, YARN RM NodesListManager will resolve IP address every time when node doing
heartbeat. When DNS server becomes slow, NM heartbeat will be blocked and cannot make progress.

This message was sent by Atlassian JIRA

View raw message