hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ken Goodhope <kengoodh...@gmail.com>
Subject Re: Getting a temporary directory in map jobs
Date Fri, 10 Dec 2010 17:37:28 GMT
FileSystem has a method createTempFile, or something like that.  You can
pass it the option to delete the file when the jvm exits.  If the jvm exits
abnormally, you may still get some files laying around, but under normal
circumstances they will get cleaned up.

On Fri, Dec 10, 2010 at 8:46 AM, Koji Noguchi <knoguchi@yahoo-inc.com>wrote:

>  Hi Eric.
>
> Try ‘./tmp’ (or current working directory).
>
> Koji
>
>
>
> On 12/10/10 1:19 AM, "Eric" <eric.xkcd@gmail.com> wrote:
>
> Hi there,
>
> I have a map-reduce job that processes binary files. I'm currently using
> /tmp/ as a temporary location to write data to and perform operations like
> decompression. If a mapper fails, the temporary filesare left behind on the
> nodes.
>
> Is there a way to get a temp location from Hadoop, that is cleaned after
> the Map job finishes? I'm sure there must be a better way to do this ;-)
>
> I'm using Hadoop version 0.20.2 (the Cloudera distribution)
>
> Thanks in advance!
>
> Eric
>
>

Mime
View raw message