hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Lyubimov <dlie...@gmail.com>
Subject Task local temp folder question
Date Thu, 14 Apr 2011 19:23:33 GMT
Hi,

is there's such a concept as 'task local working directory' that the
task could use for side info?

In my problem, map tasks are using side info which are memory-mapped
structures and as such must be downloaded to a local FS in order to be
used.

I used 'java.io.tmpdir' system property before but i wonder if there's
more general mechanism managed by a task tracker that also takes care
of the temp space cleanup, etc.? E.g. DistributedCache functionality
(as well as task classpath files) do use similar capability. Is there
a way to extend that functionality to create more local files per task
needs?

Thanks.
-Dmitriy

Mime
View raw message