lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Ryan <mr...@moreover.com>
Subject RE: Slow performance on distributed search
Date Tue, 26 Mar 2013 21:07:37 GMT
Depending on your use case and the particulars of your system, a previous post I made about
using a FieldCache in SolrIndexSearcher for id retrieval (see http://osdir.com/ml/solr-user.lucene.apache.org/2013-01/msg01574.html)
may help you. In your case, it might not be the merging process on the controller itself that
is the slow point, but rather the retrieval of 100,000 documents on each of your 40 shards.
It may seem that your shards are responding in < 10ms based on QTime, but the actual time
spent retrieving the docs from disk is not included in that figure.

If it really is the merging time on the controller that is the slow point, I would think that
could also be improved. Sorting 4 million values really shouldn't take that long...

-Michael

-----Original Message-----
From: qungg [mailto:qzheng1989@gmail.com] 
Sent: Tuesday, March 26, 2013 2:55 PM
To: solr-user@lucene.apache.org
Subject: RE: Slow performance on distributed search

for start=100,000&row=10. event though each individual shard take only < 10ms to query,
the merging process done by controller would take about a minutes. 

By looking at logs, each shard is giving the controller shard 100,010 rows of data, and because
there are 40 shards in total, the controller is getting
100,010*40 rows of data, therefore merging is taking a long time. 

I have not tried solr cloud, does any one know the performance of query large start row on
solr cloud?



--
View this message in context: http://lucene.472066.n3.nabble.com/Slow-performance-on-distributed-search-tp4051434p4051492.html
Sent from the Solr - User mailing list archive at Nabble.com.

Mime
View raw message