lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3562) Stop storing TermsEnum in CloseableThreadLocal inside Terms instance
Date Sat, 05 Nov 2011 22:18:51 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144852#comment-13144852
] 

Uwe Schindler commented on LUCENE-3562:
---------------------------------------

+1
                
> Stop storing TermsEnum in CloseableThreadLocal inside Terms instance
> --------------------------------------------------------------------
>
>                 Key: LUCENE-3562
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3562
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: 4.0
>
>         Attachments: LUCENE-3562.patch
>
>
> We have sugar methods in Terms.java (docFreq, totalTermFreq, docs,
> docsAndPositions) that use a saved thread-private TermsEnum to do the
> lookups.
> But on apps that send many threads through Lucene, and/or have many
> segments, this can add up to a lot of RAM, especially if the codecs
> impl holds onto stuff.
> Also, Terms has a close method (closes the CloseableThreadLocal) which
> must be called, but we fail to do so in some places.
> These saved enums are the cause of the recent OOME in TestNRTManager
> (TestNRTManager.testNRTManager -seed
> 2aa27e1aec20c4a2:-4a5a5ecf46837d0e:-7c4f651f1f0b75d7 -mult 3
> -nightly).
> Really sharing these enums is a holdover from before Lucene queries
> would share state (ie, save the TermState from the first pass, and use
> it later to pull enums, get docFreq, etc.).  It's not helpful anymore,
> and it can use gobbs of RAM, so I'd like to remove it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message