hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siqi Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3891) Apps are starved for resource due to default FairSharePolicy in Fair Scheduler
Date Tue, 07 Jul 2015 06:59:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14616287#comment-14616287
] 

Siqi Li commented on YARN-3891:
-------------------------------

Yes, I think this is related to  MAPREDUCE-6302. So if headroom calculation never return 0,
MR will never preempt reducers for mappers?

> Apps are starved for resource due to default FairSharePolicy in Fair Scheduler
> ------------------------------------------------------------------------------
>
>                 Key: YARN-3891
>                 URL: https://issues.apache.org/jira/browse/YARN-3891
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>            Reporter: Siqi Li
>            Priority: Critical
>
> Based on FairSharePolicy, FSLeafQueue will try to schedule an app that has the lowest
resource consumption first.
> However, we have seen some apps that have several thousand reducers stuck at waiting
for some newly scheduled mappers to launch. However, it would be very difficult for such apps
to get resources if there are many other jobs with low resource consumption and the queue
is full.
> If the queue keeps getting new applications, then the stuck reducers would never finish.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message