Return-Path: <mapreduce-issues-return-85275-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org>
X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io
Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io
Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183])
	by cust-asf2.ponee.io (Postfix) with ESMTP id BC6F22009A8
	for <archive-asf-public-internal@cust-asf2.ponee.io>; Tue, 17 May 2016 21:42:14 +0200 (CEST)
Received: by cust-asf.ponee.io (Postfix)
	id BAFA9160A20; Tue, 17 May 2016 19:42:14 +0000 (UTC)
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by cust-asf.ponee.io (Postfix) with SMTP id 0DBD61607A8
	for <archive-asf-public@cust-asf.ponee.io>; Tue, 17 May 2016 21:42:13 +0200 (CEST)
Received: (qmail 65104 invoked by uid 500); 17 May 2016 19:42:13 -0000
Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:mapreduce-issues-help@hadoop.apache.org>
List-Unsubscribe: <mailto:mapreduce-issues-unsubscribe@hadoop.apache.org>
List-Post: <mailto:mapreduce-issues@hadoop.apache.org>
List-Id: <mapreduce-issues.hadoop.apache.org>
Delivered-To: mailing list mapreduce-issues@hadoop.apache.org
Received: (qmail 65067 invoked by uid 99); 17 May 2016 19:42:13 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 May 2016 19:42:13 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id E1DAA2C1F62
	for <mapreduce-issues@hadoop.apache.org>; Tue, 17 May 2016 19:42:12 +0000 (UTC)
Date: Tue, 17 May 2016 19:42:12 +0000 (UTC)
From: "zhihai xu (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <JIRA.12969928.1463384839000.214626.1463514132922@Atlassian.JIRA>
In-Reply-To: <JIRA.12969928.1463384839000@Atlassian.JIRA>
References: <JIRA.12969928.1463384839000@Atlassian.JIRA> <JIRA.12969928.1463384839658@arcas>
Subject: [jira] [Commented] (MAPREDUCE-6696) Add a configuration to limit
 the number of map tasks allowed per job.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
archived-at: Tue, 17 May 2016 19:42:14 -0000


    [ https://issues.apache.org/jira/browse/MAPREDUCE-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15287387#comment-15287387 ] 

zhihai xu commented on MAPREDUCE-6696:
--------------------------------------

All these test failures are not related to my changes:

TestMRCJCFileOutputCommitter is passed in my local build also the failure for TestMRCJCFileOutputCommitter is due to test environment problem:
2016-05-17 17:29:33,792 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(60)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable.

TestMiniMRChildTask and TestMiniMRChildTask.testTaskOldEnv failure happened at launchContainer which already pass job submission phase after my code change.
2016-05-17 17:45:48,781 WARN  [ContainersLauncher #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(245)) - Exception from container-launch with container ID: container_1463507138005_0001_01_000002 and exit code: 127
ExitCodeException exitCode=127: nice: bash: No such file or directory

	at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
	at org.apache.hadoop.util.Shell.run(Shell.java:850)
	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
	at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)


> Add a configuration to limit the number of map tasks allowed per job.
> ---------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6696
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6696
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: job submission
>    Affects Versions: 2.8.0
>            Reporter: zhihai xu
>            Assignee: zhihai xu
>         Attachments: MAPREDUCE-6696.000.patch, MAPREDUCE-6696.001.patch
>
>
> Add a configuration "mapreduce.job.max.map" to limit the number of map tasks allowed per job. It will be useful for Hadoop admin to save Hadoop cluster resource by preventing users from submitting big mapreduce jobs. A mapredeuce job with too many mappers may fail with OOM after running for long time. It will be a big waste.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org