lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Lamprecht <clampre...@gmail.com>
Subject Re: Distributed vs Merged Searching
Date Thu, 02 Feb 2006 07:13:09 GMT
One issue is that if you are splitting the index in half (for
example), getting some results from index A and some from index B,
then you need to merge the results somewhere.  But the scores coming
from the two indexes are not related at all, for example, document 100
from index A has score 0.85, document 200 from index B has score 0.90
-- this doesn't necessarily mean that document 200 should be ranked
before document 100.    This is one issue to deal with.

I think this issue has been discussed on this mailing list before. 
Has anyone else had to deal with this issue with a distributed index? 
What does Nutch do?

-chris

On 1/31/06, Chun Wei Ho <cwho.work@gmail.com> wrote:
> I am deploying a web application serving searches on a Lucene index,
> and am deciding between distributing search between several machines
> or single searching, and was hoping that someone could tell me from
> their experiences:
>
> + Is there anything particular to watch out for if using distributed
> searching instead of  searching one merged Lucene index?
>
> + What should be the size of the index that I am looking at before I
> need to (or should be) turn to distributed searching to reduce
> response/search time? I know it would depend a lot on hardware and request
> frequency but I was wondering if anyone could post their hardware
> info and index size as a reference of when/if they had to use
> distributed search due to load issues.
>
> Thanks :)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message