lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From eks dev <eks...@yahoo.co.uk>
Subject Re: Stop long running queries
Date Wed, 21 Feb 2007 14:57:40 GMT
have a look at  LuceneQueryOptimizer.java in nutch


----- Original Message ----
From: Tim Johnson <timothy.w.johnson@saic.com>
To: java-user@lucene.apache.org
Sent: Wednesday, 21 February, 2007 3:34:36 PM
Subject: Stop long running queries

I'm having issues with some queries taking in excess of 500 secs to run to
completion. The system being used consists of ~100 million docs spilt up
across ~600 indexes.  The indexes are of various sizes from 15MB to 8GB and
all searches done in the system require an exact count of matching hits.
The documents are also indexed with various fields and types ranging from 15
to 200 fields depending on the data indexed.

98% of all queries run under 4 sec, but the remaining 2% run for > 15 secs
and sometimes > 500 secs.  These long running queries usually consist of
wildcard and range queries where the number of BooleanQuery statements
reaches into the 100,000 range.  Some of these queries return a relatively
small number of results (below 300).

I've recently tried to kill the search using a separate thread to wrap the
searcher.search method, which works, but only once.  The next query to come
through throws an "IOException: The handle is invalid." in the
FSInputStream.readInternal() method.

Any suggestions on how to stop these long running queries? 

Tim Johnson
SAIC
Systems Engineer
757.226.1569



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org






		
___________________________________________________________ 
What kind of emailer are you? Find out today - get a free analysis of your email personality.
Take the quiz at the Yahoo! Mail Championship. 
http://uk.rd.yahoo.com/evt=44106/*http://mail.yahoo.net/uk 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message