Return-Path: X-Original-To: apmail-hadoop-mapreduce-dev-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B05777550 for ; Thu, 25 Aug 2011 20:18:56 +0000 (UTC) Received: (qmail 37679 invoked by uid 500); 25 Aug 2011 20:18:56 -0000 Delivered-To: apmail-hadoop-mapreduce-dev-archive@hadoop.apache.org Received: (qmail 37417 invoked by uid 500); 25 Aug 2011 20:18:55 -0000 Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-dev@hadoop.apache.org Delivered-To: mailing list mapreduce-dev@hadoop.apache.org Received: (qmail 37408 invoked by uid 99); 25 Aug 2011 20:18:55 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Aug 2011 20:18:55 +0000 X-ASF-Spam-Status: No, hits=-2000.5 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Aug 2011 20:18:53 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id 66B77CF880 for ; Thu, 25 Aug 2011 20:18:33 +0000 (UTC) Date: Thu, 25 Aug 2011 20:18:33 +0000 (UTC) From: "Owen O'Malley (JIRA)" To: mapreduce-dev@hadoop.apache.org Message-ID: <1062669997.14967.1314303513399.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] [Resolved] (MAPREDUCE-1943) Implement limits on per-job JobConf, Counters, StatusReport, Split-Sizes MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/MAPREDUCE-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen O'Malley resolved MAPREDUCE-1943. -------------------------------------- Resolution: Fixed > Implement limits on per-job JobConf, Counters, StatusReport, Split-Sizes > ------------------------------------------------------------------------ > > Key: MAPREDUCE-1943 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1943 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Reporter: Mahadev konar > Assignee: Mahadev konar > Fix For: 0.20.203.0 > > Attachments: MAPREDUCE-1943-0.20-yahoo.patch, MAPREDUCE-1943-0.20-yahoo.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S-fix.patch, MAPREDUCE-1943-yahoo-hadoop-0.20S.patch > > > We have come across issues in production clusters wherein users abuse counters, statusreport messages and split sizes. One such case was when one of the users had 100 million counters. This leads to jobtracker going out of memory and being unresponsive. In this jira I am proposing to put sane limits on the status report length, the number of counters and the size of block locations returned by the input split. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira