lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Willnauer (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-5038) Don't call MergePolicy / IndexWriter during DWPT Flush
Date Tue, 11 Jun 2013 12:52:20 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-5038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Simon Willnauer updated LUCENE-5038:
------------------------------------

    Attachment: LUCENE-5038.patch

here is an updated patch. I removed _setUseCompoundFile_ from the MPs now we only have the
ratio on the MP and setUseCF on IWC. Yet, given that patch I think we can safely move the
CFS related settings from LogMP and TieredMP into MergePolicy or add an intermediate class.
The are all identical and we should consolidate this. Any objections if I pull them up to
MP?
                
> Don't call MergePolicy / IndexWriter during DWPT Flush
> ------------------------------------------------------
>
>                 Key: LUCENE-5038
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5038
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: core/index
>    Affects Versions: 4.3
>            Reporter: Simon Willnauer
>            Assignee: Simon Willnauer
>             Fix For: 4.4
>
>         Attachments: LUCENE-5038.patch, LUCENE-5038.patch, LUCENE-5038.patch, LUCENE-5038.patch
>
>
> We currently consult the indexwriter -> merge policy to decide if we need to write
CFS or not which is bad in many ways.
> - we should call mergepolicy only during merges
> - we should never sync on IW during DWPT flush
> - we should be able to make the decision if we need to write CFS or not before flush,
ie. we could write parts of the flush directly to CFS or even start writing stored fields
directly.
> - in the NRT case it might make sense to write all flushes to CFS to minimize filedescriptors
independent of the index size.
> I wonder if we can use a simple boolean for this in the IWC and get away with not consulting
merge policy. This would simplify concurrency a lot here already.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message