lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Ryan <mr...@moreover.com>
Subject RE: Solr 4.0 - timeAllowed in distributed search
Date Sun, 20 Jan 2013 16:49:22 GMT
(This is based on my knowledge of 3.6 - not sure if this has changed in 4.0)

You are using rows=30000, which requires retrieving 30000 documents from disk. In a non-distributed
search, the QTime will not include the time it takes to retrieve these documents, but in a
distributed search, it will. For a *:* query, the document retrieval will almost always be
the slowest part of the query. I'd suggest measuring how long it takes for the response to
be returned, or use rows=0.

The timeAllowed feature is very misleading. It only applies to a small portion of the query
(which in my experience is usually not the part of the query that is actually slow). Do not
depend on timeAllowed doing anything useful :)

-Michael

-----Original Message-----
From: Lyuba Romanchuk [mailto:lyuba.romanchuk@gmail.com] 
Sent: Sunday, January 20, 2013 6:36 AM
To: solr-user@lucene.apache.org
Subject: Solr 4.0 - timeAllowed in distributed search

Hi,

I try to use timeAllowed in query both in distributed search with one shard and directly to
the same shard.
I send the same query with timeAllowed=500 :

   - directly to the shard then QTime ~= 600 ms
   - through distributes search to the same shard QTime ~= 7 sec.

I have two questions:

   - It seems that timeAllowed parameter doesn't work for distributes
   search, does it?
   - What may be the reason that causes the query to the shard through
   distributes search takes much more time than to the shard directly (the
   same distribution remains without timeAllowed parameter in the query)?


Test results:

Ask one shard through distributed search:


http://localhost:8983/solr/shard_2013-01-07/select?q=*:*&rows=30000&shards=127.0.0.1%3A8983%2Fsolr%2Fshard_2013-01-07&timeAllowed=500&partialResults=true&shards.info=true&debugQuery=true
<response>
<lst name="responseHeader">
<bool name="partialResults">true</bool>
<int name="status">0</int>
<int name="QTime">7307</int>
<lst name="params">
<str name="q">*:*</str>
<str name="shards">127.0.0.1:8983/solr/shard_2013-01-07</str>
<str name="partialResults">true</str>
<str name="debugQuery">true</str>
<str name="shards.info">true</str>
<str name="rows">30000</str>
<str name="timeAllowed">500</str></lst></lst>
<lst name="shards.info">
<lst name="127.0.0.1:8983/solr/shard_2013-01-07">
<long name="numFound">29574223</long>
<float name="maxScore">1.0</float>
<long name="time">646</long></lst></lst>
<result name="response" numFound="29574223" start="0" maxScore="1.0"> ...
30,000 docs
...
<lst name="debug">
<str name="rawquerystring">*:*</str>
<str name="querystring">*:*</str>
<str name="parsedquery">MatchAllDocsQuery(*:*)</str>
<str name="parsedquery_toString">*:*</str>
<str name="QParser">LuceneQParser</str>
<lst name="timing"><double name="time">6141.0</double> <lst name="prepare"><double
name="time">0.0</double> <lst name="org.apache.solr.handler.component.QueryComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.FacetComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.MoreLikeThisComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.HighlightComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.StatsComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.DebugComponent"><double
name="time">0.0</double></lst></lst>
<lst name="process"><double name="time">6141.0</double> <lst name="org.apache.solr.handler.component.QueryComponent"><double
name="time">6022.0</double></lst>
<lst name="org.apache.solr.handler.component.FacetComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.MoreLikeThisComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.HighlightComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.StatsComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.DebugComponent"><double
name="time">119.0</double></lst></lst></lst><

Ask the same shard directly:

http://localhost:8983/solr/shard_2013-01-07/select?q=*:*&rows=30000&timeAllowed=500&partialResults=true&shards.info=true&debugQuery=true
<lst name="responseHeader">
<bool name="partialResults">true</bool>
<int name="status">0</int>
<int name="QTime">617</int>
<lst name="params">
<str name="q">*:*</str>
<str name="partialResults">true</str>
<str name="debugQuery">true</str>
<str name="shards.info">true</str>
<str name="rows">30000</str>
<str name="timeAllowed">500</str></lst></lst>
<result name="response" numFound="28687243" start="0"> ...
30,000 docs
<lst name="debug"><str name="rawquerystring">*:*</str><str
name="querystring">*:*</str><str
name="parsedquery">MatchAllDocsQuery(*:*)</str><str
name="parsedquery_toString">*:*</str>
<str name="QParser">LuceneQParser</str>
<lst name="timing"><double name="time">617.0</double> <lst name="prepare"><double
name="time">0.0</double> <lst name="org.apache.solr.handler.component.QueryComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.FacetComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.MoreLikeThisComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.HighlightComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.StatsComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.DebugComponent"><double
name="time">0.0</double></lst></lst>
<lst name="process"><double name="time">617.0</double> <lst name="org.apache.solr.handler.component.QueryComponent"><double
name="time">516.0</double></lst>
<lst name="org.apache.solr.handler.component.FacetComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.MoreLikeThisComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.HighlightComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.StatsComponent"><double
name="time">0.0</double></lst>
<lst name="org.apache.solr.handler.component.DebugComponent"><double
name="time">101.0</double></lst></lst></lst></lst>

Thank you.
Best regards,
Lyuba

Mime
View raw message