lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jamie <ja...@stimulussoft.com>
Subject OutOfMemory ParallelMultisearcher
Date Tue, 16 Mar 2010 22:00:05 GMT
Hi There

I have an index which is 36 GB large. When I perform  eight simultaneous 
searches (performed by JMeter) on the index, an OutOfMemory error 
occurs. Since I need to potentially search across multiple indexes and 
those indexes can change from one search query to the next, each user 
has their own ParallelMultiSearcher object. Before each search 
operation, I reconstruct the ParrallelMultisearcher with the appropriate 
Searchers to each of the indexes that need to be included for that 
particular search query.

The problem is that requiring each user to have their own 
ParallelMultisearcher seems to limit the number of  users that can use 
the system at the same time.

While experimenting, when I make the ParallelMultiSearcher static, the 
same object used by all users, the OutOfMemory problem goes away and I 
am able to execute 50 simultaneous searches. The problem I have is I 
cannot make ParallelMultisearcher static, since the specific indexes 
used are variable from one search query to the next. I initially thought 
one could just cache the underlying Searchers and all would be okay, but 
this does not appear to be the case.

My question: Will ParallelMultisearcher tend to consume a large amount 
of memory by itself when used on large indices? If so, do you have any 
suggestions on how I might support the above scenario (i.e. when the 
indexes used change from one query to the next)

Thanks in advance

Jamie


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message