Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Message-ID: <19800063.1195594483156.JavaMail.jira@brutus>
Date: Tue, 20 Nov 2007 13:34:43 -0800 (PST)
From: "Michael McCandless (JIRA)" <jira@apache.org>
To: java-dev@lucene.apache.org
Subject: [jira] Commented: (LUCENE-1052) Add an "termInfosIndexDivisor" to
 IndexReader
In-Reply-To: <15241894.1194953511410.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/LUCENE-1052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544076 ] 

Michael McCandless commented on LUCENE-1052:
--------------------------------------------


Maybe, instead, we should simply make it "easy" to subclass
TermInfosReader whenever a SegmentReader wants to instantiate it?

Ie, the formula is such an advanced use case that it seems appropriate
to subclass instead of trying to break it out into a special
interface/abstract class?

Of course, we need to know this class at SegmentReader construction
time, so I think to specify it we should in fact take Doug's suggested
approach using generic properties.

The challenge with Lucene (and Hadoop) is how can you reach deep down
into a complex IndexReader.open static method call to change various
details of the embedded *Readers while they are being constructed,
and, after they are constructed... I agree it is messy now that we
must propogate the setTermInfosIndexInterval method up the *Reader
hierarchy when not all Readers would even use a TermInfosReader.

So ... maybe we 1) implement generic Lucene properties w/ static
classes/methods to set/get these properties, then 2) remove
set/getTermInfosIndexInterval from *Reader and make a generic property
for it instead, and 3) add another property that allows you to specify
the Class (or String name) of that is your TermInfosReader subclass
(and make it non-final)?


> Add an "termInfosIndexDivisor" to IndexReader
> ---------------------------------------------
>
>                 Key: LUCENE-1052
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1052
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>    Affects Versions: 2.2
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 2.3
>
>         Attachments: LUCENE-1052.patch, termInfosConfigurer.patch
>
>
> The termIndexInterval, set during indexing time, let's you tradeoff
> how much RAM is used by a reader to load the indexed terms vs cost of
> seeking to the specific term you want to load.
> But the downside is you must set it at indexing time.
> This issue adds an indexDivisor to TermInfosReader so that on opening
> a reader you could further sub-sample the the termIndexInterval to use
> less RAM.  EG a setting of 2 means every 2 * termIndexInterval is
> loaded into RAM.
> This is particularly useful if your index has a great many terms (eg
> you accidentally indexed binary terms).
> Spinoff from this thread:
>   http://www.gossamer-threads.com/lists/lucene/java-dev/54371

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org