lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cheolgoo Kang <app...@gmail.com>
Subject Re: Map-Reduce
Date Thu, 04 Aug 2005 05:46:54 GMT
Yeah, it would be great if we had a Directory subclass like MapReduceDirectory.

I'm looking for the ComputeFarm that is implemented a distributed
parallel computing environment on the JINI technology.


On 8/4/05, Paul Smith <psmith@aconex.com> wrote:
> I've been reading the Nutch MapReduce stuff[1], and the original
> Google paper [2].
> 
> I know there's a mapreduce branch in the nutch project, but is there
> any plan/talk of perhaps integrating something like that directly
> into the Lucene API?  For projects that need a lower-level API like
> Lucene, rather than the crawl-like nature of Nutch, the potential to
> index lots of information in an efficient manner is very appealing
> indeed.
> 
> I'm not suggesting this is _easy_, just curious of what folks on the
> Lucene-side of things think.  Perhaps a chance to refactor out from
> nutch a shared library?
> 
> I would love to hear anyones thoughts on the matter.
> 
> cheers,
> 
> Paul Smith
> 
> [1] http://wiki.apache.org/nutch-data/attachments/Presentations/
> attachments/oscon05.pdf
> [2] http://labs.google.com/papers/mapreduce-osdi04.pdf
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
> 
> 


-- 
Regards,
Cheolgoo Kang

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message