hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-1943) Implement limits on per-job JobConf, Counters, StatusReport, Split-Sizes
Date Thu, 11 Aug 2011 19:02:27 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tom White updated MAPREDUCE-1943:
---------------------------------

    Fix Version/s: 0.20.203.0

This was fixed in 0.20.203.0 (see Subversion Commits tab, also commit r1077730).

> Implement limits on per-job JobConf, Counters, StatusReport, Split-Sizes
> ------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1943
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1943
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Mahadev konar
>            Assignee: Mahadev konar
>             Fix For: 0.20.203.0
>
>         Attachments: MAPREDUCE-1943-0.20-yahoo.patch, MAPREDUCE-1943-0.20-yahoo.patch,
MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch,
MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch,
MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch,
MAPREDUCE-1943-yahoo-hadoop-0.20S.patch
>
>
> We have come across issues in production clusters wherein users abuse counters, statusreport
messages and split sizes. One such case was when one of the users had 100 million counters.
This leads to jobtracker going out of memory and being unresponsive. In this jira I am proposing
to put sane limits on the status report length, the number of counters and the size of block
locations returned by the input split. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message