hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philip Zeyliger <phi...@cloudera.com>
Subject Re: distributed cache across jobs?
Date Wed, 30 Sep 2009 06:10:22 GMT
The distributed cache, does, I believe, cache files across jobs.  The
TaskTracker keeps the files around as long as it's got space for them.  It
also reference counts the files in use so they don't get deleted while a
task might still be using them.
DistributedCache.localizeCache() is where you want to look; it's a bit
hairy.

-- Philip


On Tue, Sep 29, 2009 at 10:47 PM, Zheng Shao <zshao@facebook.com> wrote:

>  Is it true that distributed cache only work for a single job?
>
> Is it possible for 2 different jobs to share the same local copy of the
> same file from distributed cache?
>
>
>
> Thanks,
>
> Zheng
>
>
>

Mime
View raw message