hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur" <tuc...@gmail.com>
Subject Re: Global information in mapreduce
Date Tue, 20 Mar 2007 05:08:01 GMT
you could write your word set to a file in DFS somewhere outside of
the input directory and read it at map init time (within the
configure() method). you could pass the path to file as a
configuration property.

HTH

Alejandro

On 3/19/07, Ilya Vishnevsky <Ilya.Vishnevsky@e-legion.com> wrote:
> Hello! My question is about mapreduce. Is it possible to pass to the map
> function some global information? For example I have a set of words and
> a large set of documents. I want the map function to get each document
> as value and emit pairs (word-frequency) for each word in the set, where
> "frequency" is frequency of this word in the document. To do this I need
> map function to have access to the set of words each time it runs. Is it
> possible to do that?
>
>

Mime
View raw message