Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Sat, 5 Nov 2011 20:54:51 +0000 (UTC)
From: "Michael McCandless (Created) (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: 
 <1097060852.3700.1320526491437.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Created] (LUCENE-3562) Stop storing TermsEnum in
 CloseableThreadLocal inside Terms instance
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Stop storing TermsEnum in CloseableThreadLocal inside Terms instance
--------------------------------------------------------------------

                 Key: LUCENE-3562
                 URL: https://issues.apache.org/jira/browse/LUCENE-3562
             Project: Lucene - Java
          Issue Type: Improvement
            Reporter: Michael McCandless
            Assignee: Michael McCandless
             Fix For: 4.0


We have sugar methods in Terms.java (docFreq, totalTermFreq, docs,
docsAndPositions) that use a saved thread-private TermsEnum to do the
lookups.

But on apps that send many threads through Lucene, and/or have many
segments, this can add up to a lot of RAM, especially if the codecs
impl holds onto stuff.

Also, Terms has a close method (closes the CloseableThreadLocal) which
must be called, but we fail to do so in some places.

These saved enums are the cause of the recent OOME in TestNRTManager
(TestNRTManager.testNRTManager -seed
2aa27e1aec20c4a2:-4a5a5ecf46837d0e:-7c4f651f1f0b75d7 -mult 3
-nightly).

Really sharing these enums is a holdover from before Lucene queries
would share state (ie, save the TermState from the first pass, and use
it later to pull enums, get docFreq, etc.).  It's not helpful anymore,
and it can use gobbs of RAM, so I'd like to remove it.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org