lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Dotzour <BDotz...@widen.com>
Subject Memory usage: IndexSearcher & Sort
Date Wed, 29 Sep 2004 13:11:03 GMT
I have been investigating a serious memory problem in our web app (using
Tapestry, Hibernate, & Lucene) and have reduced it to being the way in which
we are using Lucene to search on things.  Being a webapp, we have focused on
doing our work within a user's request.  So we basically end up opening at
least one new IndexSearcher on each individual page view.  In one particular
case, we were doing this in a loop, eventually opening ~20-~40
IndexSearchers which caused our memory usage to skyrocket.  After viewing
that one page 3 or 4 times we would exhaust the server's memory allocation.
 
Most helpful in this search was the following thread from Bugzilla:
 
http://issues.apache.org/bugzilla/show_bug.cgi?id=30628
<http://issues.apache.org/bugzilla/show_bug.cgi?id=30628> 
 
>From this thread, it sounds like constantly opening and closing
IndexSearcher objects is a "BAD THING", but it is exactly what we are doing
in our app.  
There are a few things that puzzle me and I'd love it if anyone has some
input that might clear up some of these questions.
 
1.  According to the Bugzilla thread, and from my own testing, you can open
lots of IndexSearchers in a loop and do a search WITHOUT SORTING and not
have this memory problem.  Is there an issue with the Sort code?
2.  Can anyone give a brief, technical explanation as to why opening
multiple IndexSearcher objects is bad?
3.  Certainly some of you on this list are using Lucene in a web-app
environment.  Can anyone list some best practices on managing
reading/writing/searching a Lucene index in that context?
 
 
Thank you all
Bryan
-----------------------------------------------
Some extra information about my Lucene setup:
 
Lucene 1.4.1
We maintain 5 different indexes, all in RAMDirectories.  The indexes aren't
especially big (< 100,000 total objects combined).
  
 

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message