Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3E994790E for ; Wed, 21 Dec 2011 10:02:22 +0000 (UTC) Received: (qmail 26199 invoked by uid 500); 21 Dec 2011 10:02:21 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 26131 invoked by uid 500); 21 Dec 2011 10:02:21 -0000 Mailing-List: contact hdfs-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-user@hadoop.apache.org Delivered-To: mailing list hdfs-user@hadoop.apache.org Received: (qmail 26123 invoked by uid 99); 21 Dec 2011 10:02:21 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Dec 2011 10:02:21 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of jayaseelan.e@ericsson.com designates 193.180.251.61 as permitted sender) Received: from [193.180.251.61] (HELO mailgw10.se.ericsson.net) (193.180.251.61) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Dec 2011 10:02:13 +0000 X-AuditID: c1b4fb3d-b7cfeae000005b81-2d-4ef1ae8ffcb4 Received: from esessmw0197.eemea.ericsson.se (Unknown_Domain [153.88.253.125]) by mailgw10.se.ericsson.net (Symantec Mail Security) with SMTP id 78.34.23425.F8EA1FE4; Wed, 21 Dec 2011 11:01:51 +0100 (CET) Received: from ESESSCMS0358.eemea.ericsson.se ([169.254.1.3]) by esessmw0197.eemea.ericsson.se ([153.88.115.87]) with mapi; Wed, 21 Dec 2011 11:01:51 +0100 From: Jayaseelan E To: "hdfs-user@hadoop.apache.org" CC: "harsh@cloudera.com" Date: Wed, 21 Dec 2011 11:01:49 +0100 Subject: RE: REG:error details(temporary directory does not exist) Thread-Topic: REG:error details(temporary directory does not exist) Thread-Index: Acy/xstUkQBuSzszQDaFCtyEJJPHFwAAEA4A Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_A21B6EA6764C404C86A7326E6DD937420C1E4F9F69ESESSCMS0358e_" MIME-Version: 1.0 X-Brightmail-Tracker: AAAAAA== --_000_A21B6EA6764C404C86A7326E6DD937420C1E4F9F69ESESSCMS0358e_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Harsh basically i have some 40 folders each folder size is approximately 3GB .To = move tis folder from HDFS to local location.I used the distcp.. Iam not copying files, but iam copying the whole folders which contains fol= ders and files. So i cannot use the copyToLocal command. May be i will try this distcp with NFS location thanks indeed jayaseelan ________________________________ From: Harsh J [mailto:harsh@cloudera.com] Sent: Wednesday, December 21, 2011 3:26 PM To: hdfs-user@hadoop.apache.org Subject: Re: REG:error details(temporary directory does not exist) Hey Jayaseelan, You can't distcp to a file:// destination URI, unless that location is a NF= S point commonly available across all your nodes. What you probably need for 3 GB worth of data is a simple "hadoop fs -copyT= oLocal src dest" command. On 21-Dec-2011, at 12:20 PM, Jayaseelan E wrote: Hi Folks! I am have some 40 folders whose size of each folders are aproximately 3GB.I= used distcp command to take a copy from HDFS to local machine. But it fails with the following error. /opt/hadoop/bin/hadoop distcp hdfs://bl460x1890:9000/<> = file:///<> java.io.IOException: The temporary job-output directory file:/archive/_dist= cp_logs_ndyr8m/_temporarydoesn't exist! at org.apache.hadoop.mapred.FileOutputCommitter.getWorkPath(FileOut= putCommitter.java:204) at org.apache.hadoop.mapred.FileOutputFormat.getTaskOutputPath(File= OutputFormat.java:234) My hadoop version is 2.0.1 If any one can please help me resolve this Thanks indeed jayaseelan --_000_A21B6EA6764C404C86A7326E6DD937420C1E4F9F69ESESSCMS0358e_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <= BASE=20 href=3Dx-msg://44/>
Hi Harsh
basically i have some 40 folders each folder siz= e is=20 approximately 3GB .To move tis folder from HDFS to local location.I used th= e=20 distcp..
Iam not copying files, but iam copying the whole= =20 folders which contains folders and files.
So i cannot use the copyToLocal=20 command.
May be i will try this distcp with NFS=20 location
 
thanks indeed
jayaseelan

From: Harsh J [mailto:harsh@cloudera.co= m]=20
Sent: Wednesday, December 21, 2011 3:26 PM
To:=20 hdfs-user@hadoop.apache.org
Subject: Re: REG:error details(tempor= ary=20 directory does not exist)

Hey Jayaseelan,

You can't distcp to a file:// destination URI, unless that location is= a=20 NFS point commonly available across all your nodes.

What you probably need for 3 GB worth of data is a simple "hadoop fs=20 -copyToLocal src dest" command.

On 21-Dec-2011, at 12:20 PM, Jayaseelan E wrote:
 
Hi Folks!
I am have some 40 folders whose size of each folders are aproximatel= y=20 3GB.I used distcp command to take a copy from HDFS to local machine.
But it fails with the following error.
/opt/hadoop/bin/hadoop distcp   hdfs://bl460x1890:9000/<<Source= =20 location>>  >">file:///<<targetpath>>
 
java.io.IOException: The temporary job-output directory file:/archive/_distcp_logs_ndyr8m/_temporarydoesn't=20 exist!
        at=20 org.apache.hadoop.mapred.FileOutputCommitter.getWorkPath(FileOutputCommit= ter.java:204)
        at=20 org.apache.hadoop.mapred.FileOutputFormat.getTaskOutputPath(FileOutputFor= mat.java:234)
 
My hadoop version is 2.0.1
If any one can please help me resolve this
 
Thanks indeed
jayaseelan
 

--_000_A21B6EA6764C404C86A7326E6DD937420C1E4F9F69ESESSCMS0358e_--