hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "MengWang (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-2345) Optimize jobtracker's memory usage
Date Fri, 25 Feb 2011 03:26:38 GMT
Optimize jobtracker's  memory usage  

                 Key: MAPREDUCE-2345
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2345
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: jobtracker
    Affects Versions: 0.21.0
            Reporter: MengWang
             Fix For: 0.23.0

To many tasks will eat up a considerable amount of JobTracker's heap space. According to our
observation, 50GB heap size can support to 5,000,000 tasks, so we should optimize jobtracker's
memory usage for more jobs and tasks. Yourkit java profile show that counters, duplicate strings,
Task waste too much memory. Our optimization around these three points reduced jobtracker's
memory to 1/3. 

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message