lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-2387) IndexWriter retains references to Readers used in Fields (memory leak)
Date Fri, 09 Apr 2010 10:57:51 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12855345#action_12855345
] 

Uwe Schindler commented on LUCENE-2387:
---------------------------------------

As Tokenizers are reused, the analyzer holds also a reference to the last used Reader. The
easy fix would be to unset the Reader in Tokenizer.close(). If this is the case for you, that
may be easy to do. So Tokenizer.close() looks like this:

{code}
/** By default, closes the input Reader. */
@Override
public void close() throws IOException {
    input.close();
    input = null; // <-- new!
}
{code}

> IndexWriter retains references to Readers used in Fields (memory leak)
> ----------------------------------------------------------------------
>
>                 Key: LUCENE-2387
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2387
>             Project: Lucene - Java
>          Issue Type: Bug
>    Affects Versions: 3.0.1
>            Reporter: Ruben Laguna
>            Assignee: Michael McCandless
>         Attachments: LUCENE-2387.patch
>
>
> As described in [1] IndexWriter retains references to Reader used in Fields and that
can lead to big memory leaks when using tika's ParsingReaders (as those can take 1MB per ParsingReader).

> [2] shows a screenshot of the reference chain to the Reader from the IndexWriter taken
with Eclipse MAT (Memory Analysis Tool) . The chain is the following:
> IndexWriter -> DocumentsWriter -> DocumentsWriterThreadState -> DocFieldProcessorPerThread
 -> DocFieldProcessorPerField -> Fieldable -> Field (fieldsData) 
> -------------
> [1] http://markmail.org/thread/ndmcgffg2mnwjo47
> [2] http://skitch.com/ecerulm/n7643/eclipse-memory-analyzer

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message