Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AE740E0AA for ; Tue, 26 Feb 2013 11:10:03 +0000 (UTC) Received: (qmail 96764 invoked by uid 500); 26 Feb 2013 11:09:58 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 96397 invoked by uid 500); 26 Feb 2013 11:09:57 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 96372 invoked by uid 99); 26 Feb 2013 11:09:57 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Feb 2013 11:09:57 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of arindamchoudhury0@gmail.com designates 74.125.82.182 as permitted sender) Received: from [74.125.82.182] (HELO mail-we0-f182.google.com) (74.125.82.182) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Feb 2013 11:09:50 +0000 Received: by mail-we0-f182.google.com with SMTP id t57so3446317wey.13 for ; Tue, 26 Feb 2013 03:09:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=F4/hnv60bzCUmAT1Y2Al+OsMaehYDHcCi8ieb9MIEFI=; b=Ze1qrDpAkQgbd3WyHrRuL7dpOwdZnYDMvVXMODPKrUQ2Y6HFPUTArPZZI0WBZltDEO m0UOsUwEpNpKKBFAusXXiqkkzsLag/AHvYIVf4dod9lfUdVNWqQxx4IVXpuL0kDTKmT/ UPd/492/0hpM2iUj/NblvSorKLGtwQVprrN9Eh2OnSpYtkxOs/w3aaZS8aH173yIGU0G d2+4E4qtW3Tmrgv2RBmIUHzBWGMsA9Ukhw6PB4LDICnMSaXqD/YVZtyg+RJXQ7IXpKvb FTM/dEhWfzkLxdF7zUYrsqf70ZFbKBK6YS+CXaHVbWW2wsuq1SDR2fggqyeDLSwlbpBQ 0+yg== MIME-Version: 1.0 X-Received: by 10.194.20.40 with SMTP id k8mr17361221wje.16.1361876969338; Tue, 26 Feb 2013 03:09:29 -0800 (PST) Received: by 10.180.97.130 with HTTP; Tue, 26 Feb 2013 03:09:29 -0800 (PST) Date: Tue, 26 Feb 2013 12:09:29 +0100 Message-ID: Subject: Running terasort with 1 map task From: Arindam Choudhury To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=047d7b5d4404fc411104d69eb0b7 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b5d4404fc411104d69eb0b7 Content-Type: text/plain; charset=ISO-8859-1 Hi all, I am trying to run terasort using one map and one reduce. so, I generated the input data using: hadoop jar hadoop-examples-1.0.4.jar teragen -Dmapred.map.tasks=1 -Dmapred.reduce.tasks=1 32000000 /user/hadoop/input32mb1map Then I launched the hadoop terasort job using: hadoop jar hadoop-examples-1.0.4.jar terasort -Dmapred.map.tasks=1 -Dmapred.reduce.tasks=1 /user/hadoop/input32mb1map /user/hadoop/output1 I thought it will run the job using 1 map and 1 reduce, but when inspect the job statistics I found: hadoop job -history /user/hadoop/output1 Task Summary ============================ Kind Total Successful Failed Killed StartTime FinishTime Setup 1 1 0 0 26-Feb-2013 10:57:47 26-Feb-2013 10:57:55 (8sec) Map 24 24 0 0 26-Feb-2013 10:57:57 26-Feb-2013 11:05:37 (7mins, 40sec) Reduce 1 1 0 0 26-Feb-2013 10:58:21 26-Feb-2013 11:08:31 (10mins, 10sec) Cleanup 1 1 0 0 26-Feb-2013 11:08:32 26-Feb-2013 11:08:36 (4sec) ============================ so, though I mentioned to launch one map tasks, there are 24 of them. How to solve this problem. How to tell hadoop to launch only one map. Thanks, --047d7b5d4404fc411104d69eb0b7 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi all,

I am trying to run terasort using one = map and one reduce. so, I generated the input data using:

hadoop jar= hadoop-examples-1.0.4.jar teragen -Dmapred.map.tasks=3D1 -Dmapred.reduce.t= asks=3D1 32000000 /user/hadoop/input32mb1map

Then I launched the hadoop terasort job using:

hadoop jar hadoop= -examples-1.0.4.jar terasort -Dmapred.map.tasks=3D1 -Dmapred.reduce.tasks= =3D1 /user/hadoop/input32mb1map /user/hadoop/output1

I thought it wi= ll run the job using 1 map and 1 reduce, but when inspect the job statistic= s I found:

hadoop job -history /user/hadoop/output1

Task Summary
=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D
Kind=A0=A0=A0 Total=A0=A0=A0 Successful=A0=A0=A0 Failed=A0=A0=A0 Kil= led=A0=A0=A0 StartTime=A0=A0=A0 FinishTime
=A0=A0=A0
Setup=A0=A0=A0 = 1=A0=A0=A0 1=A0=A0=A0 =A0=A0=A0 0=A0=A0=A0 0=A0=A0=A0 26-Feb-2013 10:57:47= =A0=A0=A0 26-Feb-2013 10:57:55 (8sec)
Map=A0=A0=A0 24=A0=A0=A0 24=A0=A0=A0 =A0=A0=A0 0=A0=A0=A0 0=A0=A0=A0 26-Feb= -2013 10:57:57=A0=A0=A0 26-Feb-2013 11:05:37 (7mins, 40sec)
Reduce=A0=A0= =A0 1=A0=A0=A0 1=A0=A0=A0 =A0=A0=A0 0=A0=A0=A0 0=A0=A0=A0 26-Feb-2013 10:58= :21=A0=A0=A0 26-Feb-2013 11:08:31 (10mins, 10sec)
Cleanup=A0=A0=A0 1=A0= =A0=A0 1=A0=A0=A0 =A0=A0=A0 0=A0=A0=A0 0=A0=A0=A0 26-Feb-2013 11:08:32=A0= =A0=A0 26-Feb-2013 11:08:36 (4sec)
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D

so, though I mentioned to launch one map tasks, there are = 24 of them.

How to solve this problem. How to tell hadoop to launch = only one map.

Thanks,
--047d7b5d4404fc411104d69eb0b7--