lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From MitchK <mitc...@web.de>
Subject Re: Distributed Search Components
Date Fri, 11 Jun 2010 15:51:40 GMT

Sorry for my late response.

Thank you both for your answers.

Chris, can you tell me where exactly I can find those implementations you
are talking about?
I can't find them, probably I am searching in the wrong code-files.

I would really like to compare the sourcecodes of both implementations. 

>From what *I* have understood, the Parallel Multi Searcher asks every *node*
for results and every *node* create a response to the calling-searcher.



> The Distributed Searcher model used in Solr allows for distinct unrelated
> components (ie: seraching, facets, etc...) to each deal with multi-stage
> requests (ie: stage #1, get the top facets from each shard, stage #2 get
> the counts for the aggregate top facets from each shard) in batch, so that
> a single request can be made to every shard for *all* of the stage#1 data
> needed by *all* of the individual search components.
> 
To do so, you need to ask every node for data - in other words you need to
use one searcher-instance per hardware-server to get the desired responses.
Or not?

Maybe I am wrong, but in the simpleFacets-implementation it seems like I can
get the current searcher-instance of my solr server.
If this would be a Parallel Multi one, I don't need to reimplement anything.
However, we both seem to don't know exactly how PMS and Yonik's sharded
searcher work, so I'd really like to take a look on both implementations.

On the other hand, maybe my english-skillz are reasonable for my
missunderstanding of your post.
Maybe you mean something like "first, ask each search component for the data
it needs. For example: get the top facets, get their counts, get the normal
search-results, get the stats".
And in the second step "now we know what we need, let's ask each node for
those data and aggregate it. So we can send *one* request instead of 4 or
more".
However, from looking at some components at the moment, I don't understand
how this is realized - so again, it would need a deeper understanding of the
underlying code, no? :-)

Maybe Yonik can enlight us a little bit, if he joins the discussion.



> Hi, All,
> 
> IIUC, the core requirement here is to be able to search across multiple
> (mirror) copies of a certain index. If so, you may want to consider taking
> a look at the Mirroring Split Policy, which in turn is based on An
> Anti-Merging Multi-Directory Indexing Framework. On the plus side, it is a
> no-frills distributed searcher that is highly customizable. On the flip
> side, it's not officially part of Lucene (at least not yet). For details
> on how to use that searcher, please see the test case that is attached
> therein.
>   
> I apologize in advance if it's digressive, but hope it helps.
> 
> Best Regards,
> Karthick Sankarachary
> 
Thank you, I will have a look at both tickets.

Kind regards
- Mitch
-- 
View this message in context: http://lucene.472066.n3.nabble.com/Distributed-Search-Components-tp831275p888922.html
Sent from the Solr - Dev mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message