hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From samir das mohapatra <samir.help...@gmail.com>
Subject copyToLocal Failed inside the cleanup(.........) of Map task
Date Mon, 03 Jun 2013 21:11:13 GMT
Dear All,

     Is there any way to copy the intermediate output file of the mapper
into     local  folder  after each map task complete.

   Right now I am using

   FileSystem.copyToLocalFile(hdfsLocation,localLocation);
 indiste the cleanup of mapper task , but it is failing .

Exception file not found.

But if I am giving same statement after the job complete in driver class
,it is working fine. that i dont want.

protected void cleanup(Context context){
   FileSystem.copyToLocalFile(hdfsLocation,localLocation);//failed
}

Note: I need to copy the inter mediate output of the mapper to local file
system just after  each map task complete. I dont want any reducer.

   If this is not the right solution then how to solve this type of
scenario.

Any help.............

regards,
samir.

Mime
View raw message