lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-7083) BKDReader exceptions at merge from innocuous? test
Date Tue, 08 Mar 2016 18:27:40 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15185430#comment-15185430
] 

Michael McCandless commented on LUCENE-7083:
--------------------------------------------

OK I found the cause here, and I have a small test that shows itl.

The bug is in the default merge logic in {{PointsWriter}} not properly detecting when a points
field was never indexed on a given segment.  I'll push the test and fix shortly ... I'll also
add a dedicated "test sparse points fields" test soon.

> BKDReader exceptions at merge from innocuous? test
> --------------------------------------------------
>
>                 Key: LUCENE-7083
>                 URL: https://issues.apache.org/jira/browse/LUCENE-7083
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
>         Attachments: broken.patch.txt
>
>
> As part of LUCENE-7075, I tried to move join/ modules numeric join support away from
deprecated encodings.
> Basically from what i see, its encoding Numeric/SortedNumeric in a byte[] as an impl
detail, it might as well use the new 8-bit encoding rather than the 7-bit one.
> There are certain bugs in what i did, but i don't like the exceptions I see:
> {noformat}
>    [junit4] Suite: org.apache.lucene.search.join.TestJoinUtil
>    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestJoinUtil -Dtests.method=testRandomOrdinalsJoin
-Dtests.seed=49BE0A860DCFFEDB -Dtests.slow=true -Dtests.locale=tr -Dtests.timezone=Asia/Urumqi
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
>    [junit4] ERROR   1.16s J1 | TestJoinUtil.testRandomOrdinalsJoin <<<
>    [junit4]    > Throwable #1: org.apache.lucene.store.AlreadyClosedException: refusing
to delete any files: this IndexWriter hit an unrecoverable exception
>    [junit4]    > 	at __randomizedtesting.SeedInfo.seed([49BE0A860DCFFEDB:F6630B1FDD33EBD7]:0)
>    [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.ensureOpen(IndexFileDeleter.java:345)
>    [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:696)
>    [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:594)
>    [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:533)
>    [junit4]    > 	at org.apache.lucene.index.IndexWriter.checkpoint(IndexWriter.java:2337)
>    [junit4]    > 	at org.apache.lucene.index.IndexWriter.publishFlushedSegment(IndexWriter.java:2395)
>    [junit4]    > 	at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.publishFlushedSegment(DocumentsWriterFlushQueue.java:198)
>    [junit4]    > 	at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.finishFlush(DocumentsWriterFlushQueue.java:213)
>    [junit4]    > 	at org.apache.lucene.index.DocumentsWriterFlushQueue$SegmentFlushTicket.publish(DocumentsWriterFlushQueue.java:249)
>    [junit4]    > 	at org.apache.lucene.index.DocumentsWriterFlushQueue.innerPurge(DocumentsWriterFlushQueue.java:116)
>    [junit4]    > 	at org.apache.lucene.index.DocumentsWriterFlushQueue.forcePurge(DocumentsWriterFlushQueue.java:138)
>    [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:624)
>    [junit4]    > 	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2815)
>    [junit4]    > 	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2989)
>    [junit4]    > 	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2956)
>    [junit4]    > 	at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:288)
>    [junit4]    > 	at org.apache.lucene.search.join.TestJoinUtil.createContext(TestJoinUtil.java:1121)
>    [junit4]    > 	at org.apache.lucene.search.join.TestJoinUtil.testRandomOrdinalsJoin(TestJoinUtil.java:403)
>    [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
>    [junit4]    > Caused by: java.lang.IllegalArgumentException: field="fromInteger"
is unrecognized
>    [junit4]    > 	at org.apache.lucene.codecs.lucene60.Lucene60PointsReader.getBKDReader(Lucene60PointsReader.java:120)
>    [junit4]    > 	at org.apache.lucene.codecs.lucene60.Lucene60PointsReader.getNumDimensions(Lucene60PointsReader.java:199)
>    [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingPointsFormat$AssertingPointsReader.intersect(AssertingPointsFormat.java:169)
>    [junit4]    > 	at org.apache.lucene.codecs.PointsWriter$1.intersect(PointsWriter.java:59)
>    [junit4]    > 	at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.writeField(Lucene60PointsWriter.java:93)
>    [junit4]    > 	at org.apache.lucene.codecs.PointsWriter.mergeOneField(PointsWriter.java:44)
>    [junit4]    > 	at org.apache.lucene.codecs.PointsWriter.merge(PointsWriter.java:143)
>    [junit4]    > 	at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:121)
>    [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingPointsFormat$AssertingPointsWriter.merge(AssertingPointsFormat.java:262)
>    [junit4]    > 	at org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:168)
>    [junit4]    > 	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:117)
>    [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4099)
>    [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3679)
>    [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
>    [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
> {noformat}
> What is happening here? Is the exception correct or are we missing a check?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message