hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5311) Replace SLOTS_MILLIS counters with MEM_MILLIS
Date Wed, 07 Aug 2013 00:08:49 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13731466#comment-13731466

Sandy Ryza commented on MAPREDUCE-5311:

I am missing how the change would break every single MR app - we would leave the API intact.
 However, does this sound like a reasonable compromise:  We keep SLOTS_MILLIS_MAPS, but instead
of trying to map resources to slots (which will make even less sense when we introduce other
resources), we compute it as the sum of the execution times for all map tasks.  Essentially
SLOTS_MILLIS_MAPS functions as CONTAINERS_MILLIS_MAPS would and is computed independently
from container capabilities.  This will break semantic compatibility in some cases for people
using the mapred.cluster.(map,reduce).memory.mb properties, but it will restore semantic compatibility
for people not using those properties (which I believe is a much larger number). It will also
make the counter function in a more understandable way that is more in line with its description.

> Replace SLOTS_MILLIS counters with MEM_MILLIS
> ---------------------------------------------
>                 Key: MAPREDUCE-5311
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5311
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: applicationmaster
>    Affects Versions: 2.0.4-alpha
>            Reporter: Alejandro Abdelnur
>            Assignee: Sandy Ryza
>            Priority: Blocker
>             Fix For: 2.1.0-beta
>         Attachments: MAPREDUCE-5311-1.patch, MAPREDUCE-5311.patch, MAPREDUCE-5311.patch
> Per discussion in MAPREDUCE-5310 and comments in the code we should remove all the related
logic and just leave the counter constant for backwards compatibility and deprecate the counter

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message