Return-Path: Delivered-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org Received: (qmail 77228 invoked from network); 10 Sep 2010 19:02:03 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 10 Sep 2010 19:02:03 -0000 Received: (qmail 74892 invoked by uid 500); 10 Sep 2010 19:02:03 -0000 Delivered-To: apmail-hadoop-mapreduce-issues-archive@hadoop.apache.org Received: (qmail 74863 invoked by uid 500); 10 Sep 2010 19:02:03 -0000 Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-issues@hadoop.apache.org Delivered-To: mailing list mapreduce-issues@hadoop.apache.org Received: (qmail 74855 invoked by uid 99); 10 Sep 2010 19:02:03 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Sep 2010 19:02:03 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.22] (HELO thor.apache.org) (140.211.11.22) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Sep 2010 19:02:02 +0000 Received: from thor (localhost [127.0.0.1]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o8AJ1gpP011262 for ; Fri, 10 Sep 2010 19:01:42 GMT Message-ID: <17807748.124581284145302436.JavaMail.jira@thor> Date: Fri, 10 Sep 2010 15:01:42 -0400 (EDT) From: "Mahadev konar (JIRA)" To: mapreduce-issues@hadoop.apache.org Subject: [jira] Updated: (MAPREDUCE-1943) Implement limits on per-job JobConf, Counters, StatusReport, Split-Sizes In-Reply-To: <3045016.383431279142036184.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/MAPREDUCE-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-1943: ------------------------------------- Attachment: MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch sorry attached a wrong file. > Implement limits on per-job JobConf, Counters, StatusReport, Split-Sizes > ------------------------------------------------------------------------ > > Key: MAPREDUCE-1943 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1943 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Reporter: Mahadev konar > Assignee: Mahadev konar > Fix For: 0.22.0 > > Attachments: MAPREDUCE-1943-0.20-yahoo.patch, MAPREDUCE-1943-0.20-yahoo.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S.patch > > > We have come across issues in production clusters wherein users abuse counters, statusreport messages and split sizes. One such case was when one of the users had 100 million counters. This leads to jobtracker going out of memory and being unresponsive. In this jira I am proposing to put sane limits on the status report length, the number of counters and the size of block locations returned by the input split. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.