lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karthick Sankarachary <karthick.sankarach...@gmail.com>
Subject Re: Distributed Search Components
Date Tue, 01 Jun 2010 21:58:13 GMT
Hi, All,

IIUC, the core requirement here is to be able to search across multiple
(mirror) copies of a certain index. If so, you may want to consider taking a
look at the Mirroring Split
Policy<https://issues.apache.org/jira/browse/LUCENE-2434>,
which in turn is based on An Anti-Merging Multi-Directory Indexing
Framework<https://issues.apache.org/jira/browse/LUCENE-2425>.
On the plus side, it is a no-frills distributed searcher that is highly
customizable. On the flip side, it's not officially part of Lucene (at least
not yet). For details on how to use that searcher, please see the test case
that is attached therein.

I apologize in advance if it's digressive, but hope it helps.

Best Regards,
Karthick Sankarachary


On Tue, Jun 1, 2010 at 2:30 PM, Chris Hostetter <hossman_lucene@fucit.org>wrote:

>
> : has anyone considered the ParallelMultiSearcher for distributed search?
> : Perhaps it is not neccessary to implement code that takes care of the
> : distributed part of a search, if we make a Solr-server thinking, that he
> : searches only one index.
>
> AS i understand it, part of the reason this type of thing wasn't used back
> when Yonik first stared writing the Distributed code for Solr, was that it
> the MultiSeracher model is inefficient in it's assumptions about what
> types of calls (on the sub-searchers) are "cheap" so the network overhead
> would be higher.  it also doesn't allow
> for parallel work of differnet types (ie: searching, and faceting, and
> highlighting) to all take advantage of the same network requests (because
> it only knows about "searching"
>
> The Distributed Searcher model used in Solr allows for distinct unrelated
> components (ie: seraching, facets, etc...) to each deal with multi-stage
> requests (ie: stage #1, get the top facets from each shard, stage #2 get
> the counts for the aggregate top facets from each shard) in batch, so that
> a single request can be made to every shard for *all* of the stage#1 data
> needed by *all* of the individual search components.
>
> This underlying principle certianly makes distributed searching more
> complicated to understand then the model used in several other search
> platforms out there that i've heard of -- but it also allows solr to
> support a lot more complicated distributed features then what i've seen of
> those same products.
>
>
>
> -Hoss
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

Mime
View raw message