hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-2345) Optimize jobtracker's memory usage
Date Mon, 09 Mar 2015 22:49:39 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Allen Wittenauer resolved MAPREDUCE-2345.
-----------------------------------------
    Resolution: Won't Fix

Closing as won't fix

> Optimize jobtracker's  memory usage  
> -------------------------------------
>
>                 Key: MAPREDUCE-2345
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2345
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobtracker
>    Affects Versions: 0.21.0
>            Reporter: MengWang
>              Labels: hadoop
>         Attachments: jt-memory-useage.bmp
>
>
> Too many tasks will eat up a considerable amount of JobTracker's heap space. According
to our observation, 50GB heap size can support to 5,000,000 tasks, so we should optimize jobtracker's
memory usage for more jobs and tasks. Yourkit java profile show that counters, duplicate strings,
task waste too much memory. Our optimization around these three points reduced jobtracker's
memory to 1/3. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message