Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Message-ID: <28578407.31441283668413571.JavaMail.jira@thor>
Date: Sun, 5 Sep 2010 02:33:33 -0400 (EDT)
From: "Mahadev konar (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Subject: [jira] Updated: (MAPREDUCE-1943) Implement limits on per-job
 JobConf, Counters, StatusReport, Split-Sizes
In-Reply-To: <3045016.383431279142036184.JavaMail.jira@thor>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/MAPREDUCE-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mahadev konar updated MAPREDUCE-1943:
-------------------------------------

    Attachment: MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch

the fix addendum with a test case. 

I will be uploading a single patch for trunk soon.


> Implement limits on per-job JobConf, Counters, StatusReport, Split-Sizes
> ------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1943
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1943
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Mahadev konar
>            Assignee: Mahadev konar
>             Fix For: 0.22.0
>
>         Attachments: MAPREDUCE-1943-0.20-yahoo.patch, MAPREDUCE-1943-0.20-yahoo.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S.patch
>
>
> We have come across issues in production clusters wherein users abuse counters, statusreport messages and split sizes. One such case was when one of the users had 100 million counters. This leads to jobtracker going out of memory and being unresponsive. In this jira I am proposing to put sane limits on the status report length, the number of counters and the size of block locations returned by the input split. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.