lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Lamprecht <>
Subject Re: Best Practices for Distributing Lucene Indexing and Searching
Date Tue, 19 Jul 2005 05:09:11 GMT
See the paper at: 

"MapReduce is a programming model and an associated implementation for
processing and generating large data sets. Users specify a map
function that processes a key/value pair to generate a set of
intermediate key/value pairs, and a reduce function that merges all
intermediate values associated with the same intermediate key. Many
real world tasks are expressible in this model, as shown in the paper.

cool stuff

On 7/18/05, Peter Gelderbloem <> wrote:
> I am thinking of having a cluster of one indexer and a few searchers 1
> to n.
> The indexer will consist of a number of stages as defined in SEDA. I
> must still do this decomposition.  the resulting index will be published
> via message q to the searchers that will stop doing searches long enough
> to update the local index.
> What is the purpose behind the nutch mapreduce component?
> Would it be useful for me to look at it in order to better decompose the
> indexer component?
> Cheers,
> Peter Gelderbloem
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message