Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (herse.apache.org: local policy)
From: "Tim Johnson" <timothy.w.johnson@saic.com>
To: <java-user@lucene.apache.org>
Subject: Stop long running queries
Date: Wed, 21 Feb 2007 09:34:36 -0500
Message-ID: <000f01c755c5$6ac4f740$4601a8c0@TRAVELLER>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="US-ASCII"
Content-Transfer-Encoding: 7bit
Thread-Index: AcdVxWlCSer8eLHmSpmD6G4F8we1EA==

I'm having issues with some queries taking in excess of 500 secs to run to
completion. The system being used consists of ~100 million docs spilt up
across ~600 indexes.  The indexes are of various sizes from 15MB to 8GB and
all searches done in the system require an exact count of matching hits.
The documents are also indexed with various fields and types ranging from 15
to 200 fields depending on the data indexed.

98% of all queries run under 4 sec, but the remaining 2% run for > 15 secs
and sometimes > 500 secs.  These long running queries usually consist of
wildcard and range queries where the number of BooleanQuery statements
reaches into the 100,000 range.  Some of these queries return a relatively
small number of results (below 300).

I've recently tried to kill the search using a separate thread to wrap the
searcher.search method, which works, but only once.  The next query to come
through throws an "IOException: The handle is invalid." in the
FSInputStream.readInternal() method.

Any suggestions on how to stop these long running queries? 

Tim Johnson
SAIC
Systems Engineer
757.226.1569


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org