Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A02089FF9 for ; Fri, 31 Aug 2012 04:21:24 +0000 (UTC) Received: (qmail 97280 invoked by uid 500); 31 Aug 2012 04:21:19 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 97056 invoked by uid 500); 31 Aug 2012 04:21:18 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 97031 invoked by uid 99); 31 Aug 2012 04:21:18 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 31 Aug 2012 04:21:18 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=FSL_RCVD_USER,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of yhemanth@gmail.com designates 209.85.212.182 as permitted sender) Received: from [209.85.212.182] (HELO mail-wi0-f182.google.com) (209.85.212.182) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 31 Aug 2012 04:21:11 +0000 Received: by wibhq12 with SMTP id hq12so875159wib.11 for ; Thu, 30 Aug 2012 21:20:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=xMv92ofUoNbNae7EWEUxURFhckwA4AAPQcEqwZUWeRM=; b=Y0seNjk6/FVNDQJUf3RduWyv7XgEoJ8qW9IkQJF3/BpVWYxFWn+10y5Kp/M0OglRmk twijCZ6IvGzX3fkSkMdwk2FvICvn3nSpOywAKJP/BKI7LwwXbilUdLZGOrNGaRUEud1G 9fWWvZDKIQLFtppR7RryIgQ7iDPjqUFQWGMN6FXOBq5qoHk2penygK0Eo6zVs/0SdKXZ K2W+qQB6gJRIIqDt10RiYXcjG5oy5GQ01LbenMcFY3NyaxNJjV31qG4RVrLZQASasLxt b/tI5OZ2GR35yYwqfuIcxX30hqID/JRfajuXy1H1qjX91JuCUcN9hpfTDS7ldZdStxss zPCg== MIME-Version: 1.0 Received: by 10.180.82.230 with SMTP id l6mr1639290wiy.21.1346386850749; Thu, 30 Aug 2012 21:20:50 -0700 (PDT) Received: by 10.223.76.20 with HTTP; Thu, 30 Aug 2012 21:20:50 -0700 (PDT) In-Reply-To: References:

Date: Fri, 31 Aug 2012 09:50:50 +0530 Message-ID: Subject: Re: Hadoop in Pseudo-Distributed mode on Mac OS X 10.8 From: Hemanth Yamijala To: user@hadoop.apache.org Content-Type: text/plain; charset=ISO-8859-1 The mapred.local.dir is local directories on the file system of slave nodes. In pseudo distributed mode, this would be your own machine. If you've specified any configuration for it, it should be in your mapred-site.xml. If not, it defaults to value of hadoop.tmp.dir/mapred/local. The default value of hadoop.tmp.dir is /tmp/hadoop-. Thanks Hemanth On Mon, Aug 13, 2012 at 11:42 PM, Subho Banerjee wrote: > Where do I set this? > > > On Mon, Aug 13, 2012 at 7:52 PM, Mohammad Tariq wrote: >> >> Hello Subho, >> >> Please check the permission of mapred.local.dir is. This is the >> place where map outputs are store. Reduce phase is sending the read requests >> but this directory is not accessible. As a result 403 is thrown. >> >> Regards, >> Mohammad Tariq >> >> >> >> On Mon, Aug 13, 2012 at 9:51 AM, Subho Banerjee >> wrote: >>> >>> Hello, >>> >>> I am running hadoop v1.0.3 in Mac OS X 10.8 with Java_1.6.0_33-b03-424 >>> >>> >>> When running hadoop on pseudo-distributed mode, the map seems to work, >>> but it cannot compute the reduce. >>> >>> 12/08/13 08:58:12 INFO mapred.JobClient: Running job: >>> job_201208130857_0001 >>> 12/08/13 08:58:13 INFO mapred.JobClient: map 0% reduce 0% >>> 12/08/13 08:58:27 INFO mapred.JobClient: map 20% reduce 0% >>> 12/08/13 08:58:33 INFO mapred.JobClient: map 30% reduce 0% >>> 12/08/13 08:58:36 INFO mapred.JobClient: map 40% reduce 0% >>> 12/08/13 08:58:39 INFO mapred.JobClient: map 50% reduce 0% >>> 12/08/13 08:58:42 INFO mapred.JobClient: map 60% reduce 0% >>> 12/08/13 08:58:45 INFO mapred.JobClient: map 70% reduce 0% >>> 12/08/13 08:58:48 INFO mapred.JobClient: map 80% reduce 0% >>> 12/08/13 08:58:51 INFO mapred.JobClient: map 90% reduce 0% >>> 12/08/13 08:58:54 INFO mapred.JobClient: map 100% reduce 0% >>> 12/08/13 08:59:14 INFO mapred.JobClient: Task Id : >>> attempt_201208130857_0001_m_000000_0, Status : FAILED >>> Too many fetch-failures >>> 12/08/13 08:59:14 WARN mapred.JobClient: Error reading task outputServer >>> returned HTTP response code: 403 for URL: >>> http://10.1.66.17:50060/tasklog?plaintext=true&attemptid=attempt_201208130857_0001_m_000000_0&filter=stdout >>> 12/08/13 08:59:14 WARN mapred.JobClient: Error reading task outputServer >>> returned HTTP response code: 403 for URL: >>> http://10.1.66.17:50060/tasklog?plaintext=true&attemptid=attempt_201208130857_0001_m_000000_0&filter=stderr >>> 12/08/13 08:59:18 INFO mapred.JobClient: map 89% reduce 0% >>> 12/08/13 08:59:21 INFO mapred.JobClient: map 100% reduce 0% >>> 12/08/13 09:00:14 INFO mapred.JobClient: Task Id : >>> attempt_201208130857_0001_m_000001_0, Status : FAILED >>> Too many fetch-failures >>> >>> Here is what I get when I try to see the tasklog using the links given in >>> the output >>> >>> >>> http://10.1.66.17:50060/tasklog?plaintext=true&attemptid=attempt_201208130857_0001_m_000000_0&filter=stderr >>> ---> >>> 2012-08-13 08:58:39.189 java[74092:1203] Unable to load realm info from >>> SCDynamicStore >>> >>> >>> http://10.1.66.17:50060/tasklog?plaintext=true&attemptid=attempt_201208130857_0001_m_000000_0&filter=stdout >>> ---> >>> >>> I have changed my hadoop-env.sh acoording to Mathew Buckett in >>> https://issues.apache.org/jira/browse/HADOOP-7489 >>> >>> Also this error of Unable to load realm info from SCDynamicStore does not >>> show up when I do 'hadoop namenode -format' or 'start-all.sh' >>> >>> I am also attaching a zipped copy of my logs >>> >>> >>> Cheers, >>> >>> Subho. >> >> >