hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1010) FairScheduler: decouple container scheduling from nodemanager heartbeats
Date Thu, 01 Aug 2013 18:23:59 GMT

    [ https://issues.apache.org/jira/browse/YARN-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13726704#comment-13726704

Sandy Ryza commented on YARN-1010:

Agreed that it would be a good change to include in all schedulers.  However, I think the
current interfaces to the scheduler, which send it events on NM and AM heartbeats, are the
correct ones.  I don't think there will be much shared code that would be useful to factor
> FairScheduler: decouple container scheduling from nodemanager heartbeats
> ------------------------------------------------------------------------
>                 Key: YARN-1010
>                 URL: https://issues.apache.org/jira/browse/YARN-1010
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: scheduler
>    Affects Versions: 2.1.0-beta
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>            Priority: Critical
> Currently scheduling for a node is done when a node heartbeats.
> For large cluster where the heartbeat interval is set to several seconds this delays
scheduling of incoming allocations significantly.
> We could have a continuous loop scanning all nodes and doing scheduling. If there is
availability AMs will get the allocation in the next heartbeat after the one that placed the

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message