hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xianyin Xin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3630) YARN should suggest a heartbeat interval for applications
Date Thu, 28 May 2015 02:38:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14562211#comment-14562211
] 

Xianyin Xin commented on YARN-3630:
-----------------------------------

Thanks for your comments, [~vvasudev]!
{quote}
your patch doesn't check if the calculated interval is greater than the ping interval to determine
liveliness for the AM and the NM. Is that by design?
{quote}
It's true we should do that. But in this patch I haven't add the mechanism of determining
a up limit for the {{nextHeartbeatInterval}}. I think the limit should much less than the
ping interval which is 10 minutes by default. Other hand, do you think a hard configurable
limit is accepted? 
{quote}
With respect to adaptive heartbeats for the NMs - my concern is that the proposed solution
will lead to behaviour where the NMs will be told to back off - the NMs will wait for sometime
- the RM will receive a flood of NM updates - leading to the NMs being told to back off and
so on and so forth. We'll end up in a situation where the pings will become clustered around
particular time intervals, leading to container allocation and release delays. You might be
better off picking a random interval between the default interval and the calculated interval
to spread out the NM pings
{quote}
Thanks for reminding, it's a situation I didn't think much. I think your suggestion is a nice
choice. 

> YARN should suggest a heartbeat interval for applications
> ---------------------------------------------------------
>
>                 Key: YARN-3630
>                 URL: https://issues.apache.org/jira/browse/YARN-3630
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager, scheduler
>    Affects Versions: 2.7.0
>            Reporter: Zoltán Zvara
>            Assignee: Xianyin Xin
>            Priority: Minor
>         Attachments: Notes_for_adaptive_heartbeat_policy.pdf, YARN-3630.001.patch.patch,
YARN-3630.002.patch
>
>
> It seems currently applications - for example Spark - are not adaptive to RM regarding
heartbeat intervals. RM should be able to suggest a desired heartbeat interval to applications.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message