lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steven Parkes (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-847) Factor merge policy out of IndexWriter
Date Tue, 07 Aug 2007 20:51:59 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12518251
] 

Steven Parkes commented on LUCENE-847:
--------------------------------------

Ah. I understand better now. I have to admit, I haven't kept up to date on some of the deeper
file stuff in LUCENE-843.

It seems to me there's quite a bit of difference between segment files and doc store files.
Since doc store files can be referred to by multiple segments, they can get quite large. I
don't have any trouble imaging that a merge policy might want to CFS 10MB segments but not
10GB doc stores?

I'm thinking maybe a MergePolicy#useCompoundDocStore( SegmentInfos ) makes sense? The naive
case would still just use the static setting we have now, but we could think about a better
implementation:

- Maybe never cfs doc store files? Is that an unreasonable default? It just strikes me that
there should be far fewer of these so that we don't need to and on the other end, if they
are big, CFS'ing them is going to be expensive.
- Do something smart but easy depending on number and size

> Factor merge policy out of IndexWriter
> --------------------------------------
>
>                 Key: LUCENE-847
>                 URL: https://issues.apache.org/jira/browse/LUCENE-847
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>            Reporter: Steven Parkes
>            Assignee: Steven Parkes
>         Attachments: concurrentMerge.patch, LUCENE-847.patch.txt, LUCENE-847.txt
>
>
> If we factor the merge policy out of IndexWriter, we can make it pluggable, making it
possible for apps to choose a custom merge policy and for easier experimenting with merge
policy variants.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message