Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Thu, 17 Nov 2011 16:42:51 +0000 (UTC)
From: "Michael McCandless (Updated) (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: 
 <794788413.39206.1321548171840.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1097060852.3700.1320526491437.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Updated] (LUCENE-3562) Stop storing TermsEnum in
 CloseableThreadLocal inside Terms instance
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/LUCENE-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless updated LUCENE-3562:
---------------------------------------

    Attachment: LUCENE-3562.patch

New patch; also cuts over MultiPhraseQuery to save the TermStates from weight -> scorer, and optimizes BlockTree's TermsEnum to reduce cost of init + seekExact only usages.

I think it's ready!
                
> Stop storing TermsEnum in CloseableThreadLocal inside Terms instance
> --------------------------------------------------------------------
>
>                 Key: LUCENE-3562
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3562
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: 4.0
>
>         Attachments: LUCENE-3562.patch, LUCENE-3562.patch
>
>
> We have sugar methods in Terms.java (docFreq, totalTermFreq, docs,
> docsAndPositions) that use a saved thread-private TermsEnum to do the
> lookups.
> But on apps that send many threads through Lucene, and/or have many
> segments, this can add up to a lot of RAM, especially if the codecs
> impl holds onto stuff.
> Also, Terms has a close method (closes the CloseableThreadLocal) which
> must be called, but we fail to do so in some places.
> These saved enums are the cause of the recent OOME in TestNRTManager
> (TestNRTManager.testNRTManager -seed
> 2aa27e1aec20c4a2:-4a5a5ecf46837d0e:-7c4f651f1f0b75d7 -mult 3
> -nightly).
> Really sharing these enums is a holdover from before Lucene queries
> would share state (ie, save the TermState from the first pass, and use
> it later to pull enums, get docFreq, etc.).  It's not helpful anymore,
> and it can use gobbs of RAM, so I'd like to remove it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org