lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-3197) Optimize runs forever if you keep deleting docs at the same time
Date Tue, 14 Jun 2011 10:33:50 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049109#comment-13049109
] 

Michael McCandless commented on LUCENE-3197:
--------------------------------------------

One simple way to fix this would be to have IW disregard the MergePolicy if ever it asks to
do a single-segment merge of a segment that had already been produced by merging for the current
optimize call.

But... I don't really like this, as it could be some unusual MergePolicy out there sometimes
wants to do such merging.

So I think a better solution, but API breaking to the MergePolicy, which is OK because it's
@experimental, is to change the segmentsToOptimize argument; currently it's just a set recording
which segments need to be optimized away.  I think we should change it to a Map<String,Boolean>,
where the Boolean indicates whether this segment had been created by a merge in the current
optimize session.  Then I'll fix our MPs to not cascade in such a case.

> Optimize runs forever if you keep deleting docs at the same time
> ----------------------------------------------------------------
>
>                 Key: LUCENE-3197
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3197
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 3.3, 4.0
>
>
> Because we "cascade" merges for an optimize... if you also delete documents while the
merges are running, then the merge policy will see the resulting single segment as still not
optimized (since it has pending deletes) and do a single-segment merge, and will repeat indefinitely
(as long as your app keeps deleting docs).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message