hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6703) Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers
Date Thu, 19 May 2016 23:22:12 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15292327#comment-15292327
] 

Arun Suresh commented on MAPREDUCE-6703:
----------------------------------------

[~jianhe], thanks for taking a look..

So in the first version, we decided to ignore locality (and for that matter, node labels too),
since the point of YARN-2877 is to improve allocation latencies at the cost of relaxing some
scheduling constraints. The sweet-spot would be for Applications that have large number of
really short lived tasks that do not care about locality.

Reduce tasks are always guaranteed (since the cost of preempting a reduce task is quite high)
and failed and retried map tasks are also always guaranteed..

> Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers
> ----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6703
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6703
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>         Attachments: MAPREDUCE-6703.001.patch
>
>
> YARN-2882 and YARN-4335 introduces the concept of container ExecutionTypes and specifically
OPPORTUNISTIC containers.
> The default ExecutionType is GUARANTEED. This JIRA proposes to allow users to provide
hints via config to the MR framework as to the number of containers it would like to schedule
as OPPORTUNISTIC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message