hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hong Tang (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-841) Protect Job Tracker against memory exhaustion due to very large InputSplit or JobConf objects
Date Sat, 08 Aug 2009 02:47:14 GMT
Protect Job Tracker against memory exhaustion due to very large InputSplit or JobConf objects
---------------------------------------------------------------------------------------------

                 Key: MAPREDUCE-841
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-841
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: jobtracker
    Affects Versions: 0.20.1
            Reporter: Hong Tang
             Fix For: 0.21.0


JobTracker only needs to examine a subset of information contained by InputSplit or JobConf
objects. But currently JobTracker loads the complete user-defined InputSplit and JobConf objects
in memory. This design would leave JobTracker susceptible to memory exhaustion particularly
in cases when some bugs in user code which could result in very large input splits or job
conf objects (e.g. PIG-901).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message