hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maysam Yabandeh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1067) AMRMClient heartbeat interval should not be static
Date Sat, 07 Sep 2013 07:20:55 GMT

    [ https://issues.apache.org/jira/browse/YARN-1067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13760951#comment-13760951
] 

Maysam Yabandeh commented on YARN-1067:
---------------------------------------

Thanks for the comments [~sseth].                                             
                                                                       
| I think it'll be useful to have a minimum - i.e. wait for at least this minimum time before
sending another heartbeat, irrespective of whether a resource request is pending or not. Gives
the RM time to actually allocate some containers, for the AM to attempt an allocation of these
containers, and is possibly better for the consistency of tables maintained by the AM and
RM.                                                    

The way heartbeatIntervalMs is used in the patch, meant to serve this purpose: the heartbeat
is sent by this interval if there is no update. Otherwise, the updates are checked and sent
by the shorter interval of resourceUpdateIntervalMs.  
                                                                       
| Also the heartbeat interval could be scaled back - higher than the minimum if there's no
allocations.

The heartbeatIntervalMs in the patch represents the interval in which we should send a heartbeat
weather there is an update or not. If there is some allocation, it is sent by a shorter interval
of resourceUpdateIntervalMs. To me this is an equivalent way to achieve the toning-up-heartbeat-interval
objective of the jira when an update is waiting.          

| Could some of the sleeps be replaced with wait/notify ?              
                                                                       
Yeah, I guess it could. I found the required changes the minimum when I stick with the existing
sleep approach.


                
> AMRMClient heartbeat interval should not be static
> --------------------------------------------------
>
>                 Key: YARN-1067
>                 URL: https://issues.apache.org/jira/browse/YARN-1067
>             Project: Hadoop YARN
>          Issue Type: Improvement
>    Affects Versions: 2.0.5-alpha
>            Reporter: Siddharth Seth
>         Attachments: YARN-1067.patch
>
>
> The heartbeat interval can be modified dynamically - more often when there are pending
requests, and toned down when the heartbeat is solving no purpose other than a ping.
> There's a couple of jiras which are trying to change the scheduling loop - at which point
this becomes useful.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message