hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wellington Chevreuil <wellington.chevre...@gmail.com>
Subject Re: Passing a Global Variable into a Mapper
Date Thu, 15 Sep 2011 17:51:14 GMT
Hi Arko,

you can do that using the org.apache.hadoop.filecache.DistributedCache class.

You just need to put the file in the distributed cache, in your driver class:
        ...
        DistributedCache.addCacheFile(new
Path(YOUR_FILE_PATH).toUri(), job.getConfiguration());
        ...

Then get it from the Distributed Cache, on your map class:
        ...
        Path[] cacheFiles =
DistributedCache.getLocalCacheFiles(context.getConfiguration());
        ...

Hope this helps you.

Regards,
Wellington.

2011/9/15 Arko Provo Mukherjee <arkoprovomukherjee@gmail.com>:
> Hi,
>
> Is there a way to pass some data from the driver class to the Mapper
> class without going through the HDFS?
>
> Does the API provide us with some functionality to pass some variables?
>
> Thanks a lot in advance!
> Warm regards
> Arko
>

Mime
View raw message