lucene-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sim...@apache.org
Subject [lucene-solr] 01/01: LUCENE-8813: Ensure we never apply deletes from a closed DWPTDeleteQueue (#688)
Date Sat, 01 Jun 2019 06:52:32 GMT
This is an automated email from the ASF dual-hosted git repository.

simonw pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git

commit b28de243be3144312543fdd7d7bde94de1938478
Merge: 1ff7b29 d488156
Author: Simon Willnauer <simonw@apache.org>
AuthorDate: Sat Jun 1 08:52:23 2019 +0200

    LUCENE-8813: Ensure we never apply deletes from a closed DWPTDeleteQueue (#688)
    
    Today we don't have a strong protection that we add and apply deletes / updates
    on or from an already flushed delete queue. DWPTDeleteQueue instances are replaced
    once we do a full flush in order to reopen an NRT reader or commit the IndexWriter.
    
    In LUCENE-8813 we tripped an assert that used to protect us from such an situation
    but it didn't take all corner cases from concurrent flushing into account. This change
    adds a stronger protection and ensures that we neither apply a closed delete queue nor
    add any updates or deletes to it.
    
    This change also allows to speculatively freeze the global buffer that might return
    null now if the queue has already been closed. This is now possible since we ensure that
    we never see modifications to the queue after it's been closed and that happens right
after
    the last DWPT for the ongoing full flush is done flushing.

 .../org/apache/lucene/index/DocumentsWriter.java   |  28 +++---
 .../lucene/index/DocumentsWriterDeleteQueue.java   | 103 ++++++++++++++++-----
 .../lucene/index/DocumentsWriterFlushQueue.java    |  10 +-
 .../index/TestDocumentsWriterDeleteQueue.java      |  34 ++++++-
 4 files changed, 136 insertions(+), 39 deletions(-)


Mime
View raw message