Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CA01617F40 for ; Thu, 21 May 2015 17:56:20 +0000 (UTC) Received: (qmail 58177 invoked by uid 500); 21 May 2015 17:56:14 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 58067 invoked by uid 500); 21 May 2015 17:56:14 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 58056 invoked by uid 99); 21 May 2015 17:56:13 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 21 May 2015 17:56:13 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 6D1EDC1385 for ; Thu, 21 May 2015 17:56:13 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.001 X-Spam-Level: X-Spam-Status: No, score=0.001 tagged_above=-999 required=6.31 tests=[URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 2AMu6jd8hRqe for ; Thu, 21 May 2015 17:56:01 +0000 (UTC) Received: from relayvx11a.securemail.intermedia.net (relayvx11a.securemail.intermedia.net [64.78.56.46]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 1487A204F2 for ; Thu, 21 May 2015 17:56:00 +0000 (UTC) Received: from securemail.intermedia.net (localhost [127.0.0.1]) by emg-ca-1-1.localdomain (Postfix) with ESMTP id 5CD0753E4C for ; Thu, 21 May 2015 10:55:54 -0700 (PDT) Subject: Re: Could not find any valid local directory for jobcache EXCEPTION MIME-Version: 1.0 x-echoworx-emg-received: Thu, 21 May 2015 10:55:54.364 -0700 x-echoworx-msg-id: af8ecc9a-40cc-426c-8443-3ba5bb859dd7 x-echoworx-action: delivered Received: from emg-ca-1-1.securemail.intermedia.net ([10.254.155.11]) by emg-ca-1-1 (JAMES SMTP Server 2.3.2) with SMTP ID 270 for ; Thu, 21 May 2015 10:55:54 -0700 (PDT) Received: from MBX080-W4-CO-1.exch080.serverpod.net (unknown [10.224.117.101]) by emg-ca-1-1.localdomain (Postfix) with ESMTP id 2E5EC53E4C for ; Thu, 21 May 2015 10:55:54 -0700 (PDT) Received: from MBX080-W4-CO-2.exch080.serverpod.net (10.224.117.102) by MBX080-W4-CO-1.exch080.serverpod.net (10.224.117.101) with Microsoft SMTP Server (TLS) id 15.0.1044.25; Thu, 21 May 2015 10:55:52 -0700 Received: from MBX080-W4-CO-2.exch080.serverpod.net ([10.224.117.102]) by mbx080-w4-co-2.exch080.serverpod.net ([10.224.117.102]) with mapi id 15.00.1044.021; Thu, 21 May 2015 10:55:52 -0700 From: Chris Nauroth To: "user@hadoop.apache.org" Thread-Topic: Could not find any valid local directory for jobcache EXCEPTION Thread-Index: AQHQk7Qzmk1WkknTe0eB79F+pX6xe52Gt02A Date: Thu, 21 May 2015 17:55:51 +0000 Message-ID: References: <555DB8A6.9040703@nissatech.com> In-Reply-To: <555DB8A6.9040703@nissatech.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [50.248.208.113] x-source-routing-agent: Processed Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable Based on this stack trace, I'm guessing that you're running a 1.x version of Hadoop. The TaskTracker uses a set of local directories on the node for storage of submitted job files during the task's execution. This is configured in mapred-site.xml in the property named mapred.job.local.dir. The DiskErrorException means that even after trying all directories configured in mapped.job.local.dir, the TaskTracker couldn't find a place to store the files. Possible root causes are misconfiguration, permissions on the local directories blocking access, disks are full, or disks have failed and gone into read-only mode. I hope this helps. --Chris Nauroth On 5/21/15, 3:51 AM, "Marko Dinic" wrote: >I'm new to Hadoop and I'm getting the following exception when I try to >run my job on Hadoop cluster: > >org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find >any valid local directory for jobcache/job_201409031055_3865/jars/job.jar > at=20 >org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathFor >Write(LocalDirAllocator.java:376) > at=20 >org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAlloca >tor.java:146) > at=20 >org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAlloca >tor.java:127) > at=20 >org.apache.hadoop.mapred.JobLocalizer.localizeJobJarFile(JobLocalizer.java >:268) > at=20 >org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:3 >80) > at=20 >org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:3 >70) > at=20 >org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskCo >ntroller.java:232) > at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1381) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java > >Can anyone please tell me what seems to be the problem? > >Best regards, >Marko