hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <omal...@apache.org>
Subject Re: Initial Values for Mapper
Date Mon, 15 Dec 2008 07:09:56 GMT

On Dec 14, 2008, at 10:34 PM, Bahir wrote:

> My requirement is that, each mapper should access the same values.  
> Is there
> any such location where I can put these value and each mapper will  
> acscess
> it?

Depending on the size, you can put it into the JobConf, via set

jobConf.set("my.values", "1,7,9,1000")

Then the mappers can all look at the configuration for the relevant  
attribute. If it is bigger, then the distributed file cache is  
appropriate. Write the file once into hdfs and add its url to the  
distributed file cache. Each slave will download it once to the local  
file system.

-- Owen

View raw message