lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mike Klaas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-982) Create new method optimize(int maxNumSegments) in IndexWriter
Date Tue, 21 Aug 2007 21:04:30 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12521590
] 

Mike Klaas commented on LUCENE-982:
-----------------------------------

One heuristic that has been quite useful for us is to skip optimizing segments that occupy
some fixed fraction of the index.  The remainder of the segments are optimized as usual (the
heuristic can be applied recursively).  70% is a decent number.

> Create new method optimize(int maxNumSegments) in IndexWriter
> -------------------------------------------------------------
>
>                 Key: LUCENE-982
>                 URL: https://issues.apache.org/jira/browse/LUCENE-982
>             Project: Lucene - Java
>          Issue Type: Improvement
>    Affects Versions: 2.3
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 2.3
>
>
> Spinning this out from the discussion in LUCENE-847.
> I think having a way to "slightly optimize" your index would be useful
> for many applications.
> The current optimize() call is very expensive for large indices
> because it always optimizes fully down to 1 segment.  If we add a new
> method which instead is allowed to stop optimizing once it has <=
> maxNumSegments segments in the index, this would allow applications to
> eg optimize down to say <= 10 segments after doing a bunch of updates.
> This should be a nice compromise of gaining good speedups of searching
> while not spending the full (and typically very high) cost of
> optimizing down to a single segment.
> Since LUCENE-847 is now formalizing an API for decoupling merge policy
> from IndexWriter, if we want to add this new optimize method we need
> to take it into account in LUCENE-847.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message