Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7E3D1E3BB for ; Fri, 11 Jan 2013 03:24:17 +0000 (UTC) Received: (qmail 70660 invoked by uid 500); 11 Jan 2013 03:24:08 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 70278 invoked by uid 500); 11 Jan 2013 03:24:07 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 70166 invoked by uid 99); 11 Jan 2013 03:24:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 11 Jan 2013 03:24:07 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of yaotian@gmail.com designates 209.85.219.46 as permitted sender) Received: from [209.85.219.46] (HELO mail-oa0-f46.google.com) (209.85.219.46) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 11 Jan 2013 03:24:00 +0000 Received: by mail-oa0-f46.google.com with SMTP id h16so1368857oag.19 for ; Thu, 10 Jan 2013 19:23:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=nYx8IV2eJeBZ0VZVw8dwLktFSbUekdHRAO9ZYU0fKwY=; b=YTkGYguO3YNyAHUAKwdrB4X4K5qKSfu5wMrZsVwc43jWiDrr0LAGywcV1yevktDha7 a1ZtYYRaSl0WWiQY+rtAHcao97IMFZGfjH+Fv9G1EnehgsQIoN2ch0qlcz3K+PRVXm8s MmUTU1mG3G6GysWSWdfTAQk32E4NRA1P5OPdoTjMwNsvehtogwsuB77rw8mISoUMd3yP Qc9esoiDKnpgAyR+4vj130s+JaGFjFVtyu8C/v8HP93iDxfsdUMOPJXb2GQRLuZQ/au+ EQo8TGLMq5VJbgEWQC/a1dA8RvPOOugkTh1MP9sBf1JMpIDV1XQ1S9JgmVcUZMX/ddXA hQZA== MIME-Version: 1.0 Received: by 10.60.31.131 with SMTP id a3mr42879212oei.93.1357874618999; Thu, 10 Jan 2013 19:23:38 -0800 (PST) Received: by 10.182.131.72 with HTTP; Thu, 10 Jan 2013 19:23:38 -0800 (PST) Date: Fri, 11 Jan 2013 11:23:38 +0800 Message-ID: Subject: I am running MapReduce on a 30G data on 1master/2 slave, but failed. From: yaotian To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=e89a8fb1f59450a42604d2fad254 X-Virus-Checked: Checked by ClamAV on apache.org --e89a8fb1f59450a42604d2fad254 Content-Type: text/plain; charset=ISO-8859-1 I have 1 hadoop master which name node locates and 2 slave which datanode locate. If i choose a small data like 200M, it can be done. But if i run 30G data, Map is done. But the reduce report error. Any sugggestion? This is the information. *Black-listed TaskTrackers:* 1 ------------------------------ Kind% CompleteNum TasksPendingRunningCompleteKilledFailed/Killed Task Attempts map 100.00%45000450 00 / 1 reduce 100.00%1500002 1498 12 / 3 TaskCompleteStatusStart TimeFinish TimeErrorsCounters task_201301090834_0041_r_000001 0.00% 10-Jan-2013 04:18:54 10-Jan-2013 06:46:38 (2hrs, 27mins, 44sec) Task attempt_201301090834_0041_r_000001_0 failed to report status for 600 seconds. Killing! Task attempt_201301090834_0041_r_000001_1 failed to report status for 602 seconds. Killing! Task attempt_201301090834_0041_r_000001_2 failed to report status for 602 seconds. Killing! Task attempt_201301090834_0041_r_000001_3 failed to report status for 602 seconds. Killing! 0 task_201301090834_0041_r_000002 0.00% 10-Jan-2013 04:18:54 10-Jan-2013 06:46:38 (2hrs, 27mins, 43sec) Task attempt_201301090834_0041_r_000002_0 failed to report status for 601 seconds. Killing! Task attempt_201301090834_0041_r_000002_1 failed to report status for 600 seconds. Killing! 0 task_201301090834_0041_r_000003 0.00% 10-Jan-2013 04:18:57 10-Jan-2013 06:46:38 (2hrs, 27mins, 41sec) Task attempt_201301090834_0041_r_000003_0 failed to report status for 602 seconds. Killing! Task attempt_201301090834_0041_r_000003_1 failed to report status for 602 seconds. Killing! Task attempt_201301090834_0041_r_000003_2 failed to report status for 602 seconds. Killing! 0 task_201301090834_0041_r_000005 0.00% 10-Jan-2013 06:11:07 10-Jan-2013 06:46:38 (35mins, 31sec) Task attempt_201301090834_0041_r_000005_0 failed to report status for 600 seconds. Killing! 0 --e89a8fb1f59450a42604d2fad254 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
I have 1 hadoop master which name node locates and 2 slave whi= ch datanode locate.

If= i choose a small data like 200M, it can be done.

But if i run 30G data, Map is done. But the reduce report error. Any suggge= stion?

=

This is the info= rmation.
Black-listed TaskTrackers:= =A01
=
Kind %= CompleteNum Tasks PendingRunningComplete KilledFaile= d/Killed
Task Attempts
map 100.00%=
4500 0450 00 /=A01
<= a href=3D"http://23.20.27.135:9003/jobtasks.jsp?jobid=3Djob_201301090834_00= 41&type=3Dreduce&pagenum=3D1" target=3D"_blank" style=3D"text-decor= ation:initial">reduce 100.00%=
15000 02 149812=A0/=A03


Start Time<= tr>
TaskCompleteStatusFinish TimeErrorsCounters
task_201301090834_0041_r_000001 0.00%

10-Jan-2= 013 04:18:54
10-Jan-2013 06:46:38 (2hrs, 27mins, 44sec)
 
Task attempt_201301090834_0041_r_000001=
_0 failed to report status for 600 seconds. Killing!
Task attempt_201301090834_0041_r_000001_1 failed to report status for 602 s=
econds. Killing!
Task attempt_201301090834_0041_r_000001_2 failed to report status for 602 s=
econds. Killing!
Task attempt_201301090834_0041_r_000001_3 failed to report status for 602 s=
econds. Killing!

0
task_201301090834_0041_r_000002 0.00%

10-Jan-2= 013 04:18:54
10-Jan-2013 06:46:38 (2hrs, 27mins, 43sec)
 
Task attempt_201301090834_0041_r_000002=
_0 failed to report status for 601 seconds. Killing!
Task attempt_201301090834_0041_r_000002_1 failed to report status for 600 s=
econds. Killing!

0
task_201301090834_0041_r_000003 0.00%

10-Jan-2= 013 04:18:57
10-Jan-2013 06:46:38 (2hrs, 27mins, 41sec)
 
Task attempt_201301090834_0041_r_000003=
_0 failed to report status for 602 seconds. Killing!
Task attempt_201301090834_0041_r_000003_1 failed to report status for 602 s=
econds. Killing!
Task attempt_201301090834_0041_r_000003_2 failed to report status for 602 s=
econds. Killing!

0
task_201301090834_0041_r_000005 0.00%

10-Jan-2= 013 06:11:07
10-Jan-2013 06:46:38 (35mins, 31sec)
<= pre style=3D"white-space:pre-wrap"> Task attempt_201301090834_0041_r_000005_0 failed to report status for 600 s= econds. Killing!
0
--e89a8fb1f59450a42604d2fad254--