hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1662) Capacity Scheduler reservation issue cause Job Hang
Date Mon, 04 May 2015 09:46:06 GMT

    [ https://issues.apache.org/jira/browse/YARN-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526473#comment-14526473

Sunil G commented on YARN-1662:

Yes [~jianhe]
we can close this issue. After YARN-1769, we have a better reservation too.

I checked this and its not happening now.

> Capacity Scheduler reservation issue cause Job Hang
> ---------------------------------------------------
>                 Key: YARN-1662
>                 URL: https://issues.apache.org/jira/browse/YARN-1662
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager, scheduler
>    Affects Versions: 2.2.0
>         Environment: Suse 11 SP1 + Linux
>            Reporter: Sunil G
> There are 2 node managers in my cluster.
> NM1 with 8GB
> NM2 with 8GB
> I am submitting a Job with below details:
> AM with 2GB
> Map needs 5GB
> Reducer needs 3GB
> slowstart is enabled with 0.5
> 10maps and 50reducers are assigned.
> 5maps are completed. Now few reducers got scheduled.
> Now NM1 has 2GB AM and 3Gb Reducer_1    [Used 5GB]
> NM2 has 3Gb Reducer_2			         [Used 3GB]
> A Map has now reserved(5GB) in NM1 which has only 3Gb free.
> It hangs forever.
> Potential issue is, reservation is now blocked in NM1 for a Map which needs 5GB.
> But the Reducer_1 hangs by waiting for few map ouputs.
> Reducer side preemption also not happened as few headroom is still available.

This message was sent by Atlassian JIRA

View raw message