hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5311) Replace SLOTS_MILLIS counters with MEM_MILLIS
Date Thu, 15 Aug 2013 06:07:49 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13740733#comment-13740733

Bikas Saha commented on MAPREDUCE-5311:

Let me understand this a bit.
There is existing yarn.min config. This is used by schedulers to do their internal logic.
MR AM uses this same config. This is not optimal but has the virtue of being simple and relatively
error free since there is only 1 config to change.

If we change yarn.min to yarn.sched-name.min and introduce mr.min for MR then who will keep
both of these in sync. Also the sync value depends on the scheduler being currently used.
This is also sub optimal but is less preferable than the former because 1) new config 2) manual
synch of 2 configs based on scheduler being used. 

If YARN-1004 doesnt touch yarn.min then we are good. Currently both schedulers are using the
min values and will probably continue to use it for the foreseeable future. IMO yarn.max is
not scheduler specific. Its an admin config that must be enforced by any scheduler. Lets just
leave these configs as they are because removing them is causing more grief than gain (including
this jira). And let MR use these configs to do internal calculations. I wish we had more foresight
earlier and done the right thing for slot_millis when we wrote the original code.
> Replace SLOTS_MILLIS counters with MEM_MILLIS
> ---------------------------------------------
>                 Key: MAPREDUCE-5311
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5311
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: applicationmaster
>    Affects Versions: 2.0.4-alpha
>            Reporter: Alejandro Abdelnur
>            Assignee: Sandy Ryza
>            Priority: Blocker
>             Fix For: 2.1.0-beta
>         Attachments: MAPREDUCE-5311-1.patch, MAPREDUCE-5311.patch, MAPREDUCE-5311.patch
> Per discussion in MAPREDUCE-5310 and comments in the code we should remove all the related
logic and just leave the counter constant for backwards compatibility and deprecate the counter

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message