lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject Re: Distributed Search Components
Date Tue, 01 Jun 2010 21:30:44 GMT

: has anyone considered the ParallelMultiSearcher for distributed search?
: Perhaps it is not neccessary to implement code that takes care of the
: distributed part of a search, if we make a Solr-server thinking, that he
: searches only one index.

AS i understand it, part of the reason this type of thing wasn't used back 
when Yonik first stared writing the Distributed code for Solr, was that it 
the MultiSeracher model is inefficient in it's assumptions about what 
types of calls (on the sub-searchers) are "cheap" so the network overhead 
would be higher.  it also doesn't allow 
for parallel work of differnet types (ie: searching, and faceting, and 
highlighting) to all take advantage of the same network requests (because 
it only knows about "searching"

The Distributed Searcher model used in Solr allows for distinct unrelated 
components (ie: seraching, facets, etc...) to each deal with multi-stage 
requests (ie: stage #1, get the top facets from each shard, stage #2 get 
the counts for the aggregate top facets from each shard) in batch, so that 
a single request can be made to every shard for *all* of the stage#1 data 
needed by *all* of the individual search components.

This underlying principle certianly makes distributed searching more 
complicated to understand then the model used in several other search 
platforms out there that i've heard of -- but it also allows solr to 
support a lot more complicated distributed features then what i've seen of 
those same products.



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message