Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D7C11DDC9 for ; Thu, 30 Aug 2012 17:24:35 +0000 (UTC) Received: (qmail 35082 invoked by uid 500); 30 Aug 2012 17:24:30 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 34984 invoked by uid 500); 30 Aug 2012 17:24:30 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 34975 invoked by uid 99); 30 Aug 2012 17:24:30 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 30 Aug 2012 17:24:30 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of stones.gao@gmail.com designates 209.85.214.176 as permitted sender) Received: from [209.85.214.176] (HELO mail-ob0-f176.google.com) (209.85.214.176) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 30 Aug 2012 17:24:24 +0000 Received: by obbtb18 with SMTP id tb18so4825038obb.35 for ; Thu, 30 Aug 2012 10:24:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=9HhP1888bejvFhB1PKACuClT+PqSvrtluGdRdtRPepE=; b=v7H1Qwxultg22QFWW+/P4o1gGpJrbpzsp+tOF1ZwA5MMIk1HX303rFf7X6AF4Rlqfz gtG7buMueRhBbUFW17dGCcBDyZxg1Wxx++JmEJvuv/BTdNVoy8OVsGdhCqvuylw43my9 GU2UWCk6o0i/JKIK+j73LxRheJFd+gMHprz1S8jtg7ETO8uQov7khl5uGxCNZ3Lm9eNm 2aolFB1tVBeBImwI5PLRuKEbDckzdEiBkwAGLWXKJejGx2HfNcTWDws5XHW1p5n4zLvY L0ze6Es9mmFgd7Y+Vk/1P06rO5TpslgslUS31bNd58GbtjdYyCDNAV4/JA6ZPZji6gGL tSgw== MIME-Version: 1.0 Received: by 10.60.31.161 with SMTP id b1mr5355423oei.76.1346347444240; Thu, 30 Aug 2012 10:24:04 -0700 (PDT) Received: by 10.76.75.202 with HTTP; Thu, 30 Aug 2012 10:24:04 -0700 (PDT) In-Reply-To: References: Date: Fri, 31 Aug 2012 01:24:04 +0800 Message-ID: Subject: Re: Hadoop in Pseudo-Distributed mode on Mac OS X 10.8 From: Stone To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=e89a8fb1ffda2880ac04c87ef178 X-Virus-Checked: Checked by ClamAV on apache.org --e89a8fb1ffda2880ac04c87ef178 Content-Type: text/plain; charset=ISO-8859-1 I got the same issue today. map tasks finished quickly but reduce is always 0%. I am also running Mac OS X 10.8. (cdh3u4) 12/08/31 01:13:03 INFO mapred.JobClient: map 0% reduce 0% 12/08/31 01:13:07 INFO mapred.JobClient: map 100% reduce 0% 12/08/31 01:23:14 INFO mapred.JobClient: Task Id : attempt_201208310112_0001_r_000000_0, Status : FAILED Task attempt_201208310112_0001_r_000000_0 failed to report status for 600 seconds. Killing! logs for the reducer : Task Logs: 'attempt_201208310112_0001_r_000000_0' stdout logs stderr logs 2012-08-31 01:13:06.316 java[46834:1203] Unable to load realm info from SCDynamicStore syslog logs 2012-08-31 01:13:06,421 INFO org.apache.hadoop.security.UserGroupInformation: JAAS Configuration already set up for Hadoop, not re-installing. 2012-08-31 01:13:06,674 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2012-08-31 01:13:06,848 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=SHUFFLE, sessionId= 2012-08-31 01:13:06,945 INFO org.apache.hadoop.mapred.Task: Using ResourceCalculatorPlugin : null 2012-08-31 01:13:06,957 INFO org.apache.hadoop.mapred.ReduceTask: ShuffleRamManager: MemoryLimit=144965632, MaxSingleShuffleLimit=36241408 2012-08-31 01:13:06,962 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Thread started: Thread for merging on-disk files 2012-08-31 01:13:06,963 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Thread waiting: Thread for merging on-disk files 2012-08-31 01:13:06,963 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Thread started: Thread for merging in memory files 2012-08-31 01:13:06,964 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Thread started: Thread for polling Map Completion Events 2012-08-31 01:13:06,964 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Need another 1 map output(s) where 0 is already in progress 2012-08-31 01:13:06,965 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Scheduled 0 outputs (0 slow hosts and0 dup hosts) 2012-08-31 01:13:11,966 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Scheduled 1 outputs (0 slow hosts and0 dup hosts) 2012-08-31 01:14:07,996 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Need another 1 map output(s) where 1 is already in progress 2012-08-31 01:14:07,996 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Scheduled 0 outputs (0 slow hosts and0 dup hosts) 2012-08-31 01:15:08,033 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Need another 1 map output(s) where 1 is already in progress 2012-08-31 01:15:08,033 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Scheduled 0 outputs (0 slow hosts and0 dup hosts) 2012-08-31 01:16:08,069 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Need another 1 map output(s) where 1 is already in progress 2012-08-31 01:16:08,070 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Scheduled 0 outputs (0 slow hosts and0 dup hosts) 2012-08-31 01:17:08,106 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Need another 1 map output(s) where 1 is already in progress 2012-08-31 01:17:08,107 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Scheduled 0 outputs (0 slow hosts and0 dup hosts) 2012-08-31 01:18:08,147 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Need another 1 map output(s) where 1 is already in progress 2012-08-31 01:18:08,147 INFO org.apache.hadoop.mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Scheduled 0 outputs (0 slow hosts and0 dup hosts) I can successfully run the wordcount example on my old macbook (os x 10.6) previously. Any suggestions ? Best Regards, Stone On Mon, Aug 13, 2012 at 12:21 PM, Subho Banerjee wrote: > Hello, > > I am running hadoop v1.0.3 in Mac OS X 10.8 with Java_1.6.0_33-b03-424 > > > When running hadoop on pseudo-distributed mode, the map seems to work, but > it cannot compute the reduce. > > 12/08/13 08:58:12 INFO mapred.JobClient: Running job: job_201208130857_0001 > 12/08/13 08:58:13 INFO mapred.JobClient: map 0% reduce 0% > 12/08/13 08:58:27 INFO mapred.JobClient: map 20% reduce 0% > 12/08/13 08:58:33 INFO mapred.JobClient: map 30% reduce 0% > 12/08/13 08:58:36 INFO mapred.JobClient: map 40% reduce 0% > 12/08/13 08:58:39 INFO mapred.JobClient: map 50% reduce 0% > 12/08/13 08:58:42 INFO mapred.JobClient: map 60% reduce 0% > 12/08/13 08:58:45 INFO mapred.JobClient: map 70% reduce 0% > 12/08/13 08:58:48 INFO mapred.JobClient: map 80% reduce 0% > 12/08/13 08:58:51 INFO mapred.JobClient: map 90% reduce 0% > 12/08/13 08:58:54 INFO mapred.JobClient: map 100% reduce 0% > 12/08/13 08:59:14 INFO mapred.JobClient: Task Id : > attempt_201208130857_0001_m_000000_0, Status : FAILED > Too many fetch-failures > 12/08/13 08:59:14 WARN mapred.JobClient: Error reading task outputServer > returned HTTP response code: 403 for URL: > http://10.1.66.17:50060/tasklog?plaintext=true&attemptid=attempt_201208130857_0001_m_000000_0&filter=stdout > 12/08/13 08:59:14 WARN mapred.JobClient: Error reading task outputServer > returned HTTP response code: 403 for URL: > http://10.1.66.17:50060/tasklog?plaintext=true&attemptid=attempt_201208130857_0001_m_000000_0&filter=stderr > 12/08/13 08:59:18 INFO mapred.JobClient: map 89% reduce 0% > 12/08/13 08:59:21 INFO mapred.JobClient: map 100% reduce 0% > 12/08/13 09:00:14 INFO mapred.JobClient: Task Id : > attempt_201208130857_0001_m_000001_0, Status : FAILED > Too many fetch-failures > > Here is what I get when I try to see the tasklog using the links given in > the output > > > http://10.1.66.17:50060/tasklog?plaintext=true&attemptid=attempt_201208130857_0001_m_000000_0&filter=stderr > ---> > 2012-08-13 08:58:39.189 java[74092:1203] Unable to load realm info from > SCDynamicStore > > > http://10.1.66.17:50060/tasklog?plaintext=true&attemptid=attempt_201208130857_0001_m_000000_0&filter=stdout > ---> > > I have changed my hadoop-env.sh acoording to Mathew Buckett in > https://issues.apache.org/jira/browse/HADOOP-7489 > > Also this error of Unable to load realm info from SCDynamicStore does not > show up when I do 'hadoop namenode -format' or 'start-all.sh' > > I am also attaching a zipped copy of my logs > > > Cheers, > > Subho. > --e89a8fb1ffda2880ac04c87ef178 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
I got the same issue today. map tasks finished quickly but reduce is a= lways 0%. I am also running Mac OS X 10.8. (cdh3u4)

12/08/31 01:13:03 INFO mapred.JobClient: =A0map 0% reduce 0%
12/08/31 01:13:07 INFO mapred.JobClient: =A0map 100% reduce 0%
12= /08/31 01:23:14 INFO mapred.JobClient: Task Id : attempt_201208310112_0001_= r_000000_0, Status : FAILED
Task attempt_201208310112_0001_r_0000= 00_0 failed to report status for 600 seconds. Killing!



logs for the reduce= r :=A0

Task Logs: 'attempt_201208310112_0= 001_r_000000_0'



stdout logs


stderr logs
2= 012-08-31 01:13:06.316 java[46834:1203] Unable to load realm info from SCDy= namicStore


syslog logs
2012-08-31 01:13:06,421 INFO org.apache.hadoop.security.UserGroupInformatio= n: JAAS Configuration already set up for Hadoop, not re-installing.
2012-08-31 01:13:06,674 WARN org.apache.hadoop.util.NativeCodeLoader: Un= able to load native-hadoop library for your platform... using builtin-java = classes where applicable
2012-08-31 01:13:06,848 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:= Initializing JVM Metrics with processName=3DSHUFFLE, sessionId=3D
2012-08-31 01:13:06,945 INFO org.apache.hadoop.mapred.Task: =A0Using Reso= urceCalculatorPlugin : null
2012-08-31 01:13:06,957 INFO org.apache.hadoop.mapred.ReduceTask: Shuf= fleRamManager: MemoryLimit=3D144965632, MaxSingleShuffleLimit=3D36241408
2012-08-31 01:13:06,962 INFO org.apache.hadoop.mapred.ReduceTask: a= ttempt_201208310112_0001_r_000000_0 Thread started: Thread for merging on-d= isk files
2012-08-31 01:13:06,963 INFO org.apache.hadoop.mapred.ReduceTask: atte= mpt_201208310112_0001_r_000000_0 Thread waiting: Thread for merging on-disk= files
2012-08-31 01:13:06,963 INFO org.apache.hadoop.mapred.Redu= ceTask: attempt_201208310112_0001_r_000000_0 Thread started: Thread for mer= ging in memory files
2012-08-31 01:13:06,964 INFO org.apache.hadoop.mapred.ReduceTask: atte= mpt_201208310112_0001_r_000000_0 Thread started: Thread for polling Map Com= pletion Events
2012-08-31 01:13:06,964 INFO org.apache.hadoop.map= red.ReduceTask: attempt_201208310112_0001_r_000000_0 Need another 1 map out= put(s) where 0 is already in progress
2012-08-31 01:13:06,965 INFO org.apache.hadoop.mapred.ReduceTask: atte= mpt_201208310112_0001_r_000000_0 Scheduled 0 outputs (0 slow hosts and0 dup= hosts)
2012-08-31 01:13:11,966 INFO org.apache.hadoop.mapred.Red= uceTask: attempt_201208310112_0001_r_000000_0 Scheduled 1 outputs (0 slow h= osts and0 dup hosts)
2012-08-31 01:14:07,996 INFO org.apache.hadoop.mapred.ReduceTask: atte= mpt_201208310112_0001_r_000000_0 Need another 1 map output(s) where 1 is al= ready in progress
2012-08-31 01:14:07,996 INFO org.apache.hadoop.= mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Scheduled 0 outputs= (0 slow hosts and0 dup hosts)
2012-08-31 01:15:08,033 INFO org.apache.hadoop.mapred.ReduceTask: atte= mpt_201208310112_0001_r_000000_0 Need another 1 map output(s) where 1 is al= ready in progress
2012-08-31 01:15:08,033 INFO org.apache.hadoop.= mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Scheduled 0 outputs= (0 slow hosts and0 dup hosts)
2012-08-31 01:16:08,069 INFO org.apache.hadoop.mapred.ReduceTask: atte= mpt_201208310112_0001_r_000000_0 Need another 1 map output(s) where 1 is al= ready in progress
2012-08-31 01:16:08,070 INFO org.apache.hadoop.= mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Scheduled 0 outputs= (0 slow hosts and0 dup hosts)
2012-08-31 01:17:08,106 INFO org.apache.hadoop.mapred.ReduceTask: atte= mpt_201208310112_0001_r_000000_0 Need another 1 map output(s) where 1 is al= ready in progress
2012-08-31 01:17:08,107 INFO org.apache.hadoop.= mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Scheduled 0 outputs= (0 slow hosts and0 dup hosts)
2012-08-31 01:18:08,147 INFO org.apache.hadoop.mapred.ReduceTask: atte= mpt_201208310112_0001_r_000000_0 Need another 1 map output(s) where 1 is al= ready in progress
2012-08-31 01:18:08,147 INFO org.apache.hadoop.= mapred.ReduceTask: attempt_201208310112_0001_r_000000_0 Scheduled 0 outputs= (0 slow hosts and0 dup hosts)

I can successfully run the wordcount example on m= y old macbook (os x 10.6) previously.=A0

Any sugge= stions ?=A0

Best Regards,
Stone



On Mon, Aug 13, 2012 at 12:21 PM, Subho = Banerjee <subs.zero@gmail.com> wrote:

Hello,

I am running hadoop v1.0.3 in Mac OS X 10.8 with Java_1.6.0_33-b03-424

<= p style=3D"line-height:17px;font-size:13px;font-family:Arial,FreeSans,Helve= tica,sans-serif;margin:0px;padding:0px">

When running hadoop on pse= udo-distributed mode, the map seems to work, but it cannot compute the redu= ce.

12/08/13 08:58:12 INFO ma= pred.JobClient: Running job: job_201208130857_0001
12/08/13 08:58:13 INFO mapred.JobClient: map 0% reduce 0%
12/08/13 08:58= :27 INFO mapred.JobClient: map 20% reduce 0%
12/08/13 08:58:33 INFO mapr= ed.JobClient: map 30% reduce 0%
12/08/13 08:58:36 INFO mapred.JobClient:= map 40% reduce 0%
12/08/13 08:58:39 INFO mapred.JobClient: map 50% reduce 0%
12/08/13 08:5= 8:42 INFO mapred.JobClient: map 60% reduce 0%
12/08/13 08:58:45 INFO map= red.JobClient: map 70% reduce 0%
12/08/13 08:58:48 INFO mapred.JobClient= : map 80% reduce 0%
12/08/13 08:58:51 INFO mapred.JobClient: map 90% reduce 0%
12/08/13 08:5= 8:54 INFO mapred.JobClient: map 100% reduce 0%
12/08/13 08:59:14 INFO ma= pred.JobClient: Task Id : attempt_201208130857_0001_m_000000_0, Status : FA= ILED
Too many fetch-failures
12/08/13 08:59:14 WARN mapred.JobClient: Error r= eading task outputServer returned HTTP response code: 403 for URL:=A0http://10.1.66.17:50060/ta= sklog?plaintext=3Dtrue&attemptid=3Dattempt_201208130857_0001_m_000000_0= &filter=3Dstdout
12/08/13 08:59:14 WARN mapred.JobClient: Error reading task outputServer re= turned HTTP response code: 403 for URL:=A0http://10.1.66.17:50060/tasklog?plaintext=3Dtrue&= attemptid=3Dattempt_201208130857_0001_m_000000_0&filter=3Dstderr 12/08/13 08:59:18 INFO mapred.JobClient: map 89% reduce 0%
12/08/13 08:5= 9:21 INFO mapred.JobClient: map 100% reduce 0%
12/08/13 09:00:14 INFO ma= pred.JobClient: Task Id : attempt_201208130857_0001_m_000001_0, Status : FA= ILED
Too many fetch-failures

http://10.1.66.17:50060/tasklog?plaintext=3Dt= rue&attemptid=3Dattempt_201208130857_0001_m_000000_0&filter=3Dstder= r=A0--->
2012-08-13 08:58:39.189 java[74092:1203]=A0Unable to load real= m info from SCDynamicStore

http://10.1.66.17:= 50060/tasklog?plaintext=3Dtrue&attemptid=3Dattempt_201208130857_0001_m_= 000000_0&filter=3Dstdout=A0--->

I have changed my hadoop-= env.sh acoording to Mathew Buckett in=A0https://issues.apache.org/jira/browse/HADOOP-7489

Also this error of Unable= to load realm info from SCDynamicStore does not show up when I do 'had= oop namenode -format' or 'start-all.sh'

I am also attaching a zip= ped copy of my logs


Cheers,

Subho.


--e89a8fb1ffda2880ac04c87ef178--