lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Otis Gospodnetic (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-842) ParallelMultiSearcher memory leak
Date Thu, 22 Mar 2007 20:46:32 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483314
] 

Otis Gospodnetic commented on LUCENE-842:
-----------------------------------------

Thomas:
Can you write a JUnit test that builds a sample index and then runs PMS over it, eventually
OOM-ing?
>From the cursory glance over your code, it looks like you are closing your searchers after
every single search.  If that is indeed what you are doing, this may be the source of the
memory leak - try reusing your searchers.
Also, why create a new (P)MS on every search?  Create it once, and reuse it.

I'll leave this issue open for a few more days, but from the above code, it just looks like
misuse of the Lucene API.


> ParallelMultiSearcher memory leak
> ---------------------------------
>
>                 Key: LUCENE-842
>                 URL: https://issues.apache.org/jira/browse/LUCENE-842
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Search
>    Affects Versions: 2.1
>         Environment: Windows XP SP2 and Red Hat EL 4
>            Reporter: Thomas Connolly
>            Priority: Critical
>         Attachments: TestParallelMultiSearcherMemLeak.java
>
>
> When using the org.apache.lucene.search.ParallelMultiSearcher to search on a single searcher
(reading a single index), continuous runs result in a memory leak. 
> Substituting the MultiSearcher does not result in a memory leak. and is the workaround
currently used.
> And example of the code used is as follows. Note the close routine was added for the
individual searchers and the MultiSearcher otherwise the was a leak in MultiSearcher.
>     private void doSearch(Search search)
>     {
>         IndexSearcher[] indexSearchers = null;
>         
>         MultiSearcher multiSearcher = null;
>         try
>         {
>             indexSearchers = getIndexSearcher();
>             
>             // aggregate the searches across multiple indexes
>             multiSearcher = new ParallelMultiSearcher(indexSearchers); // causes LEAK
BAD
>             //multiSearcher = new MultiSearcher(indexSearchers); // NO leak GOOD
>             final QueryParser parser = new QueryParser("content", new ExtendedStandardAnalyser());
>             final Query query = parser.parse(search.getQuery());
>             
>             final Hits hits = multiSearcher.search(query, getFilter(search.getFilters()),
getSort(search.getSort()));
> 			// process hits...
>         }
>         finally
>         {
>             close(indexSearchers);
>             close(multiSearcher);
>         }
>     }
>     /**
>      * Close the index searchers.
>      * 
>      * @param indexSearchers Index Searchers.
>      */
>     private static void close(IndexSearcher[] indexSearchers)
>     {
>         if (indexSearchers != null)
>         {
>             for (IndexSearcher indexSearcher : indexSearchers)
>             {
>                 try
>                 {
>                     indexSearcher.close();
>                 }
>                 catch (IOException ioex)
>                 {
>                     LOGGER.warn("Unable to close the index searcher!", ioex);
>                 }
>             }
>         }
>     }
>     
>     /**
>      * Close the multi-searcher.
>      * 
>      * @param aMultiSearcher Index Searchers.
>      */
>     private static void close(MultiSearcher aMultiSearcher)
>     {
>         try
>         {
>             aMultiSearcher.close();
>         }
>         catch (IOException ioex)
>         {
>             LOGGER.warn("Unable to close the multi searcher!", ioex);
>         }
>     }

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message