Return-Path: X-Original-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 548E910F17 for ; Fri, 7 Feb 2014 23:13:33 +0000 (UTC) Received: (qmail 41809 invoked by uid 500); 7 Feb 2014 23:13:21 -0000 Delivered-To: apmail-hadoop-mapreduce-issues-archive@hadoop.apache.org Received: (qmail 41633 invoked by uid 500); 7 Feb 2014 23:13:20 -0000 Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-issues@hadoop.apache.org Delivered-To: mailing list mapreduce-issues@hadoop.apache.org Received: (qmail 41612 invoked by uid 99); 7 Feb 2014 23:13:20 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Feb 2014 23:13:20 +0000 Date: Fri, 7 Feb 2014 23:13:20 +0000 (UTC) From: "Konstantin Shvachko (JIRA)" To: mapreduce-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Resolved] (MAPREDUCE-3469) Port to 0.22 - Implement limits on per-job JobConf, Counters, StatusReport, Split-Sizes MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/MAPREDUCE-3469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko resolved MAPREDUCE-3469. -------------------------------------------- Resolution: Duplicate > Port to 0.22 - Implement limits on per-job JobConf, Counters, StatusReport, Split-Sizes > --------------------------------------------------------------------------------------- > > Key: MAPREDUCE-3469 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3469 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Reporter: Mahadev konar > Assignee: Konstantin Shvachko > Labels: critical-0.22.0 > > We have come across issues in production clusters wherein users abuse counters, statusreport messages and split sizes. One such case was when one of the users had 100 million counters. This leads to jobtracker going out of memory and being unresponsive. In this jira I am proposing to put sane limits on the status report length, the number of counters and the size of block locations returned by the input split. -- This message was sent by Atlassian JIRA (v6.1.5#6160)