lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yonik Seeley <yo...@lucidimagination.com>
Subject Re: large OR-boolean query
Date Wed, 23 Sep 2009 20:39:19 GMT
On Wed, Sep 23, 2009 at 4:26 PM, Luo, Jeff <jluo@cas.org> wrote:
> We are experimenting a parallel approach to issue a large OR-Boolean
> query, e.g., keywords:(1 OR 2 OR 3 OR ... OR 102400), against several
> solr shards.
>
> The way we are trying is to break the large query into smaller ones,
> e.g.,
> the example above can be broken into 10 small queries: keywords:(1 OR 2
> OR 3 OR ... OR 1024), keywords:(1025 OR 1026 OR 1027 OR ... OR 2048),
> etc
>
> Now each shard will get 10 requests and the master will merge the
> results coming back from each shard, similar to the regular distributed
> search.

You're going to end up with a lot of custom code I think.
Where's the bottleneck... searching or faceting?

If faceting is the bottleneck, making an implementation that utilized
multiple threads would be one of the best ways.
If searching, you could develop a custom query type (QParserPlugin)
that handled your type of queries and split them across multiple
threads.

-Yonik
http://www.lucidimagination.com

Mime
View raw message