hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jayaseelan E <jayaseela...@ericsson.com>
Subject RE: REG:error details(temporary directory does not exist)
Date Wed, 21 Dec 2011 10:01:49 GMT
Hi Harsh
basically i have some 40 folders each folder size is approximately 3GB .To move tis folder
from HDFS to local location.I used the distcp..
Iam not copying files, but iam copying the whole folders which contains folders and files.
So i cannot use the copyToLocal command.
May be i will try this distcp with NFS location

thanks indeed

From: Harsh J [mailto:harsh@cloudera.com]
Sent: Wednesday, December 21, 2011 3:26 PM
To: hdfs-user@hadoop.apache.org
Subject: Re: REG:error details(temporary directory does not exist)

Hey Jayaseelan,

You can't distcp to a file:// destination URI, unless that location is a NFS point commonly
available across all your nodes.

What you probably need for 3 GB worth of data is a simple "hadoop fs -copyToLocal src dest"

On 21-Dec-2011, at 12:20 PM, Jayaseelan E wrote:

Hi Folks!
I am have some 40 folders whose size of each folders are aproximately 3GB.I used distcp command
to take a copy from HDFS to local machine.
But it fails with the following error.
/opt/hadoop/bin/hadoop distcp   hdfs://bl460x1890:9000/<<Source location>>  file:///<<targetpath>>

java.io.IOException: The temporary job-output directory file:/archive/_distcp_logs_ndyr8m/_temporarydoesn't
        at org.apache.hadoop.mapred.FileOutputCommitter.getWorkPath(FileOutputCommitter.java:204)
        at org.apache.hadoop.mapred.FileOutputFormat.getTaskOutputPath(FileOutputFormat.java:234)

My hadoop version is 2.0.1
If any one can please help me resolve this

Thanks indeed

View raw message