hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: temporary file locations for YARN applications
Date Sun, 20 Oct 2013 16:48:44 GMT
Every container gets its own local work directory (You can use the
relative ./) thats auto-cleaned up at the end of the container's life.
This is the best place to store the temporary files. This is not
something you need custom configuration for.

Do the files need to be on a distributed FS or a local one?

On Sun, Oct 20, 2013 at 8:54 PM, John Lilley <john.lilley@redpoint.net> wrote:
> We have a pure YARN application (no MapReduce) that has need to store a
> significant amount of temporary data.  How can we know the best location for
> these files?  How can we ensure that our YARN tasks have write access to
> these locations?  Is this something that must be configured outside of YARN?
> Thanks,
> John

Harsh J

View raw message