lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From CameronL <cameron_le...@citysearch.com>
Subject Re: Distributed Search Strategy / Shards
Date Fri, 08 Aug 2008 00:36:36 GMT


Yonik Seeley wrote:
> 
> On Thu, Aug 7, 2008 at 7:32 PM, CameronL <cameron_leach@citysearch.com>
> wrote:
>>
>> What is the current strategy for doing distributed search for Solr?  We
>> currently have our index divided over 3 servers:
>> solr1
>> solr2
>> solr3
>>
>> In order to balance the load, our application calls each with a 'shards'
>> parameter to get the data:
>> solr1:8080/select?shards=solr1,solr2,solr3...
>> solr2:8080/select?shards=solr1,solr2,solr3...
>> solr3:8080/select?shards=solr1,solr2,solr3...
>>
>> However, with this configuration we seem to be running into problems
>> (currently running latest build of solr 1.3) with the servers locking up.
>> It
>> looks like there is some sort of race condition that is causing threads
>> to
>> lock when hitting the search pretty hard.  In this configuration, each
>> solr
>> server is responsible for receiving requests, using the shards to send
>> out
>> additional requests (while also potentially receiving shards requests
>> from
>> the other two servers), and merging the results before sending back the
>> response.
>>
>> Is this the intended design for distributed search for solr?
> 
> It is.  You should be able to send top-level requests to any shard and
> have it send sub-requests to all of the other shards.
> 
> Does this lockup only happen under load?
> 
> -Yonik
> 
> 

The lockup happens under a moderately heavy load.  We are running stress
tests to make sure our configuration will be able to handle the amount of
traffic we expect.

Our other idea was to have one dedicated server to handle the incoming
requests, distributing those to the shards, and merging the results, but I
see by this post:
http://www.nabble.com/Distributed-Search-Caching-td16851547.html#a16884618
That it doesn't seem to be the intended design.

Does this seem like an issue that might need to be escalated to jira? We
might be able to provide you some additional info relating to these threads
locking if you need it.

-- 
View this message in context: http://www.nabble.com/Distributed-Search-Strategy---Shards-tp18882112p18883004.html
Sent from the Solr - User mailing list archive at Nabble.com.


Mime
View raw message