lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject Re: Search performance under high load
Date Wed, 06 Apr 2005 22:54:31 GMT

: Queries: The query strings are of highly differing complexity, from
: simple x:y to long queries involving conjunctions, disjunctions and
: wildecard queries.
:
: 90% of the queries run brilliantly. Problem is that 10% of the queries
: (simple or not) take a long time, on average more that 10 seconds,
: sometimes several minutes.

without knowing the nature of the queries, these numbers are not outside
the realm of possibility.  there have been examples on the list in the
last few days of how BooleanQueries constructed with deep nesting have
particularly bad performance.

I would suggest you timing logs to your Search code so that you get one
log line per search executed telling you:

1) the time of day the search was executed
2) the total time taken by the Searcher.search(Query) call
3) the Query.toString() of the search.
4) the Hits.length() of the result.
5) any tracking information to help you identify where the search came
   from (ie: canned search from a category listing page, user entered
   freeform text, your RSS feed genertor, etc...)

This will help you determine:

 a) is there a common element to the structure of queries that take more
    then a certain amount of time?
 b) are the slow queries clustured by time of day? is anything else
    happening on that box during that time?
 c) are the "slow" queries all resulting in a high number of Hits?
 d) are the slow searches all orriginating from a single source? (ie: are
    the queries needed by categlory listing pages all really slow) can
    they be re-implimented differently?
 e) is there anything else the slow queries have in common?



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message