hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3812) Change default memory slot sizes to be 1.5GB
Date Sat, 21 Apr 2012 17:22:35 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258899#comment-13258899
] 

Harsh J commented on MAPREDUCE-3812:
------------------------------------

Didn't realize this patch changed the way allocs would get read, or that it was doing more
than just changing default AM memory requirements. Would save me a lot of work I've otherwise
been doing on MAPREDUCE-4026 and MAPREDUCE-4027.

I'll take over this one since it looks to have gone stale. Thanks for revamping the whole
default way of reading the allocs - much simpler way now. No idea why there was per-scheduler
configs before when you can't use more than one scheduler anyway.
                
> Change default memory slot sizes to be 1.5GB
> --------------------------------------------
>
>                 Key: MAPREDUCE-3812
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3812
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: mrv2, performance
>    Affects Versions: 0.23.0
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>         Attachments: MAPREDUCE-3812-20120205.txt, MAPREDUCE-3812-20120206.1.txt, MAPREDUCE-3812-20120206.txt,
MAPREDUCE-3812.patch, MAPREDUCE-3812.patch
>
>
> After a few performance improvements tracked at MAPREDUCE-3561, like MAPREDUCE-3511 and
MAPREDUCE-3567, even a 100K maps job can also run within 1GB vmem. We earlier increased AM
slot size from 1 slot to two slots to work around the issues with AM heap. Now that those
are fixed, we should go back to 1GB.
> This is just a configuration change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message