hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chase Bradford <chase.bradf...@gmail.com>
Subject Re: Getting a temporary directory in map jobs
Date Wed, 22 Dec 2010 02:50:22 GMT
If you want a tmp file on a task's local host, just use java's createTempFile from the File
class.  It creates a file in java.io.tmp, which the task runner sets up in the task's workspace
and is cleaned by the TT even if the child jvm exits badly.

Sent from phnoe

On Dec 10, 2010, at 1:19 AM, Eric <eric.xkcd@gmail.com> wrote:

> Hi there,
> 
> I have a map-reduce job that processes binary files. I'm currently using /tmp/ as a temporary
location to write data to and perform operations like decompression. If a mapper fails, the
temporary filesare left behind on the nodes.
> 
> Is there a way to get a temp location from Hadoop, that is cleaned after the Map job
finishes? I'm sure there must be a better way to do this ;-)
> 
> I'm using Hadoop version 0.20.2 (the Cloudera distribution)
> 
> Thanks in advance!
> 
> Eric

Mime
View raw message