lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-8275) Push up #checkPendingDeletes to Directory
Date Mon, 30 Apr 2018 10:08:00 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-8275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458441#comment-16458441
] 

ASF subversion and git services commented on LUCENE-8275:
---------------------------------------------------------

Commit 1cd6f1b6e7f3e24906f552d9460b7878aef82268 in lucene-solr's branch refs/heads/branch_7x
from [~simonw]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1cd6f1b ]

LUCENE-8275: Fix TestDirectoryTaxonomyWriter.testRecreateAndRefresh


> Push up #checkPendingDeletes to Directory
> -----------------------------------------
>
>                 Key: LUCENE-8275
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8275
>             Project: Lucene - Core
>          Issue Type: Bug
>    Affects Versions: 7.4, master (8.0)
>            Reporter: Simon Willnauer
>            Priority: Major
>             Fix For: 7.4, master (8.0)
>
>         Attachments: LUCENE-8275.patch, LUCENE-8275.patch
>
>
>  IndexWriter checks in it's ctor if the incoming directory is an
>     FSDirectory. If that is the case it ensures that the directory retries
>     deleting it's pending deletes and if there are pending deletes it will
>     fail creating the writer. Yet, this check didn't unwrap filter directories
>     or subclasses like FileSwitchDirectory such that in the case of MDW we
>     never checked for pending deletes.
>     
>     There are also two places in FSDirectory that first removed the file
>     that was supposed to be created / renamed to from the pending deletes set
>     and then tried to clean up pending deletes which excluded the file. These
>     places now remove the file from the set after the pending deletes are checked.
>  
> This caused some test failures lately unfortunately very timing dependent:
>  
> {noformat}
> FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestSearcherManager
> Error Message:
> Captured an uncaught exception in thread: Thread[id=1567, name=Thread-1363, state=RUNNABLE,
group=TGRP-TestSearcherManager]
> Stack Trace:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception
in thread: Thread[id=1567, name=Thread-1363, state=RUNNABLE, group=TGRP-TestSearcherManager]
> Caused by: java.lang.RuntimeException: java.nio.file.FileAlreadyExistsException: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/core/test/J1/temp/lucene.search.TestSearcherManager_BA998C838D219DA9-001/tempDir-001/_0.fdt
>         at __randomizedtesting.SeedInfo.seed([BA998C838D219DA9]:0)
>         at org.apache.lucene.search.TestSearcherManager$8.run(TestSearcherManager.java:590)
> Caused by: java.nio.file.FileAlreadyExistsException: /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/core/test/J1/temp/lucene.search.TestSearcherManager_BA998C838D219DA9-001/tempDir-001/_0.fdt
>         at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:94)
>         at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
>         at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
>         at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:215)
>         at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
>         at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>         at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>         at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>         at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>         at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:129)
>         at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
>         at java.base/java.nio.file.Files.newOutputStream(Files.java:218)
>         at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:413)
>         at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:409)
>         at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
>         at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:665)
>         at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
>         at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
>         at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:116)
>         at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:128)
>         at org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsWriter(Lucene50StoredFieldsFormat.java:183)
>         at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:48)
>         at org.apache.lucene.index.StoredFieldsConsumer.initStoredFieldsWriter(StoredFieldsConsumer.java:39)
>         at org.apache.lucene.index.StoredFieldsConsumer.startDocument(StoredFieldsConsumer.java:46)
>         at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:363)
>         at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:399)
>         at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:251)
>         at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:490)
>         at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1518)
>         at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1210)
>         at org.apache.lucene.search.TestSearcherManager$8.run(TestSearcherManager.java:574)
> FAILED:  org.apache.lucene.search.TestSearcherManager.testConcurrentIndexCloseSearchAndRefresh
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message