hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaodi Ke (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3923) Introduce adaptive heartbeat between RM and AM
Date Tue, 14 Jul 2015 15:50:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-3923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14626529#comment-14626529
] 

Xiaodi Ke commented on YARN-3923:
---------------------------------

We already have some ideas and are working on the prototype and design doc. We will post the
design doc and patch once it is ready. 

> Introduce adaptive heartbeat between RM and AM
> ----------------------------------------------
>
>                 Key: YARN-3923
>                 URL: https://issues.apache.org/jira/browse/YARN-3923
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: api, applications, resourcemanager
>            Reporter: Xiaodi Ke
>
> Currently, the communication between RM and AM are based on pull-based heartbeat protocol.
Along with AM heartbeat, AM sends the container requests to RM and retrieves the new allocated
containers. Also, it will update RM's view of resource requests of applications. How frequently
AM sends the heartbeat will impact the task throughput and latency of YARN scheduler.  Although
the heartbeat interval can be configured in yarn-stie.xml (mapred-site.xml for MR AppMaster),
if it is configured too short, it will increase the load of RM and bring unnecessary overhead.

> We propose the adaptive heartbeat between RM and AM to achieve a balance between updating
AM's info promptly and minimizing the overhead of extra heartbeats. With adaptive heartbeat,
AM still honors the current heartbeat interval and sends the heartbeat regularly. However,
a heartbeat is triggered as soon as a container request is made. To prevent AM sending heartbeat
too frequently, a minimum interval can be used to throttle the frequency of heartbeat.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message