hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6703) Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers
Date Thu, 19 May 2016 23:22:12 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15292327#comment-15292327

Arun Suresh commented on MAPREDUCE-6703:

[~jianhe], thanks for taking a look..

So in the first version, we decided to ignore locality (and for that matter, node labels too),
since the point of YARN-2877 is to improve allocation latencies at the cost of relaxing some
scheduling constraints. The sweet-spot would be for Applications that have large number of
really short lived tasks that do not care about locality.

Reduce tasks are always guaranteed (since the cost of preempting a reduce task is quite high)
and failed and retried map tasks are also always guaranteed..

> Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers
> ----------------------------------------------------------------------
>                 Key: MAPREDUCE-6703
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6703
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>         Attachments: MAPREDUCE-6703.001.patch
> YARN-2882 and YARN-4335 introduces the concept of container ExecutionTypes and specifically
> The default ExecutionType is GUARANTEED. This JIRA proposes to allow users to provide
hints via config to the MR framework as to the number of containers it would like to schedule

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org

View raw message