lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Lu" <chris...@gmail.com>
Subject Re: Lucene Memory Leak
Date Fri, 05 Sep 2008 16:16:32 GMT
Are you using RAMDirectory?

I am actually also dealing with a memory leak. My case is only particular to
RAMDirectory.

http://markmail.org/message/dfgcnnjglne3wynp
However, this RAMDirectory case is not as simple as setting searcher=null,
because I found some reference to RAMDirectory is held by some ThreadLocal
variables.(but I need to reuse thread also).

Here is the reference tree:
org.apache.lucene.store.RAMDirectory
  |- directory of org.apache.lucene.store.RAMFile
      |- file of org.apache.lucene.store.RAMInputStream
          |- base of org.apache.lucene.index.CompoundFileReader$CSIndexInput
              |- input of org.apache.lucene.index.SegmentTermEnum
                  |- value of java.lang.ThreadLocal$ThreadLocalMap$Entry


I am trying to track it down now. If anyone knows about it, please let me
know.

-- 
Chris Lu
-------------------------
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com
Lucene Database Search in 3 minutes:
http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
DBSight customer, a shopping comparison site, (anonymous per request) got
2.6 Million Euro funding!


On Fri, Sep 5, 2008 at 8:43 AM, Andy33 <clever_one3@yahoo.com> wrote:

>
> If I don't keep the IndexSearcher as a Singleton and instead open and close
> a
> new one each time, I have a large memory leak (probably due to the large
> queries I am doing). After watching the memory a while, I still believe I
> have a small memory leak even when the Directory, Analyzer, and
> IndexSearcher are Singletons. My free memory slowly becomes smaller after
> each query. Any ideas on what that may be due to?
>
> Here's my updated code...
>
> private synchronized Hits doQuery(String field, String queryStr, Sort
> sortOrder, String indexDirectory) throws Exception
>    {
>        Directory directory = null;
>         Query query = null;
>        QueryParser parser = null;
>
>        try
>        {
>            directory = DirectorySingleton.getInstance(indexDirectory);
>            ivIndexSearcher = (IndexSearcher)
> SearcherSingleton.getInstance(directory);
>
>            //search the index
>            parser = new QueryParser(field,
> AnalyzerSingleton.getInstance());
>            query = parser.parse(queryStr);
>
>            return ivIndexSearcher.search(query, sortOrder);
>         }
>        finally
>        {
>            if(null != directory)
>            {
>                directory.close();
>            }
>
>            directory = null;
>             parser = null;
>            query = null;
>        }
>    }
>
>
> --------------
> Example Singleton
>
> public class SearcherSingleton
> {
>    private static volatile HashMap<Directory, Searcher> cvSearches = new
> HashMap<Directory, Searcher>();
>
>
>    protected SearcherSingleton()
>    {
>
>    }
>
>
>    public static Searcher getInstance(Directory directory) throws
> IOException
>    {
>        if(!cvSearches.containsKey(directory))
>        {
>            synchronized(SearcherSingleton.class)
>            {
>                if(!cvSearches.containsKey(directory))
>                {
>                    cvSearches.put(directory, new IndexSearcher(directory));
>                }
>            }
>        }
>
>        return cvSearches.get(directory);
>     }
>
> }
>
>
>
> 장용석 wrote:
> >
> > In fact, I think that the important reasons are Directory class and
> > Analyzer
> > class.
> > If you don't want IndexSearcher class keep open for the entire life of a
> > web
> > application, you can do it.
> > I think It will not cause memory leak problem.
> > But, Directory and Analyzer classes can cause the problem if they new
> > created by method call every time. I think...
> > Only keep two classes Directory and Analyzer by singlton and do test. :)
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Lucene-Memory-Leak-tp19276999p19333985.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message