lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jayant Kumar <jayantli...@yahoo.co.in>
Subject Re: problems with lucene in multithreaded environment
Date Fri, 04 Jun 2004 05:43:44 GMT
We conducted a test on our search for 500 requests
given in 27 seconds. We noticed that in the first 5
seconds, the results were coming in 100 to 500 ms. But
as the queue size kept increasing, the response time
of the search increased drastically to approx 80-100
seconds. 

Please find enclosed jvmdump.txt which contains a dump
of our search program after about 20 seconds of
starting the program.

Also enclosed is the file queries.txt which contains
few sample search queries.

Please note that this is done on a sample of 400,000
documents (450MB) on P4 having 1GB RAM.

Kindly let us know if this helps to identify the cause
of slow response.

Jayant

 --- Doug Cutting <cutting@apache.org> wrote: > Jayant
Kumar wrote:
> > We recently tested lucene with an index size of 2
> GB
> > which has about 1,500,000 documents, each document
> > having about 25 fields. The frequency of search
> was
> > about 20 queries per second. This resulted in an
> > average response time of about 20 seconds approx
> > per search.
> 
> That sounds slow, unless your queries are very
> complex.  What are your 
> queries like?
> 
> > What we observed was that lucene queues
> > the queries and does not release them until the
> > results are found. so the queries that have come
> in
> > later take up about 500 seconds. Please let us
> know
> > whether there is a technique to optimize lucene in
> > such circumstances. 
> 
> Multiple queries executed from different threads
> using a single searcher 
> should not queue, but should run in parallel.  A
> technique to find out 
> where threads are queueing is to get a thread dump
> and see where all of 
> the threads are stuck.  In Solaris and Linux,
> sending the JVM a SIGQUIT 
> will give a thread dump.  On Windows, use
> Control-Break.
> 
> Doug
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> lucene-user-help@jakarta.apache.org
>  

________________________________________________________________________
Yahoo! India Matrimony: Find your partner online. http://yahoo.shaadi.com/india-matrimony/
Mime
View raw message