lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Montgomerie <montgome...@shaw.ca>
Subject Re: [Fwd: Exception in MultiLevelSkipListReader$SkipBuffer.readByte]
Date Fri, 17 Aug 2007 05:24:57 GMT
I just tried it with the latest nightly build, the problem still happens.

I think it must have to do with a corrupted index somehow.   I've also
noticed, as a separate issue, that after this period of time (4-5 days),
certain documents aren't indexed correctly.  For example, I will do a query:

Query for Field 1 with value A returns a list of documents.  In these
documents is a document with Field 2 with value B.
Query for Field 2 with value B, return 0 documents.

Therefore the index on Field 2 is somehow missing certain documents.  Is
this possible?

Yonik Seeley wrote:
> I wonder if this is related to
> https://issues.apache.org/jira/browse/LUCENE-951
>
> If it's easy enough for you to reproduce, could you try the trunk
> version of Lucene and see if it's fixed?
>
> -Yonik
>
> On 8/16/07, Scott Montgomerie <montgomerie@shaw.ca> wrote:
>   
>> I'm getting an ArrayIndexOutOfBoundsException in
>> MultiLevelSkipListReader$SkipBuffer. This happens sporadically, on a
>> fairly small index (18 MB, about 30,000 documents). The index is
>> subject to a lot of adds and deletes, some of them concurrently. It
>> happens after about 4 days of heavy usage. I was able to isolate a copy
>> of the index that causes the exception, and I can reproduce the
>> exception cleanly in a Junit test.
>> I can see that readByte(), where the error is occuring, has no bounds
>> checking, therefore I assume that the data in there must be correct?
>> Hence, the index has obviously become corrupted. Further, optimizing
>> the index fixes the problem.
>>
>> The problem is reproducible in working system. As I said, around 4-5
>> days after optimization, the same error occurs sporadically.
>> Any ideas?
>>
>> Oh and this is Lucene 2.2.0, jdk 1.5.0_12.
>>
>> The code from the junit test that calls this is pretty simple:
>>
>> Query profileQuery = new TermQuery(new
>> Term(IndexFields.bookmark_profile_id, "1"));
>> Hits h = searcher.search(profileQuery, filterPrivate());
>>
>> search is a plain old IndexSearcher, and filterPrivate() returns a
>> QueryFilter based on a 2-term BooleanQuery.
>>
>>
>> Full stack trace:
>>
>> Exception in thread "MultiSearcher thread #2"
>> java.lang.ArrayIndexOutOfBoundsException: 14
>> at
>> org.apache.lucene.index.MultiLevelSkipListReader$SkipBuffer.readByte(MultiLevelSkipListReader.java:258)
>> at org.apache.lucene.store.IndexInput.readVInt(IndexInput.java:57)
>> at
>> org.apache.lucene.index.DefaultSkipListReader.readSkipData(DefaultSkipListReader.java:110)
>> at
>> org.apache.lucene.index.MultiLevelSkipListReader.loadNextSkip(MultiLevelSkipListReader.java:140)
>> at
>> org.apache.lucene.index.MultiLevelSkipListReader.skipTo(MultiLevelSkipListReader.java:110)
>> at
>> org.apache.lucene.index.SegmentTermDocs.skipTo(SegmentTermDocs.java:164)
>> at org.apache.lucene.index.MultiTermDocs.skipTo(MultiReader.java:413)
>> at org.apache.lucene.search.TermScorer.skipTo(TermScorer.java:145)
>> at
>> org.apache.lucene.util.ScorerDocQueue.topSkipToAndAdjustElsePop(ScorerDocQueue.java:120)
>> at
>> org.apache.lucene.search.DisjunctionSumScorer.skipTo(DisjunctionSumScorer.java:229)
>> at
>> org.apache.lucene.search.BooleanScorer2.skipTo(BooleanScorer2.java:381)
>> at
>> org.apache.lucene.search.ConjunctionScorer.doNext(ConjunctionScorer.java:63)
>> at
>> org.apache.lucene.search.ConjunctionScorer.next(ConjunctionScorer.java:58)
>> at
>> org.apache.lucene.search.BooleanScorer2.score(BooleanScorer2.java:327)
>> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:146)
>> at org.apache.lucene.search.Searcher.search(Searcher.java:118)
>> at org.apache.lucene.search.Searcher.search(Searcher.java:97)
>> at
>> org.apache.lucene.search.QueryWrapperFilter.bits(QueryWrapperFilter.java:50)
>> at
>> org.apache.lucene.search.CachingWrapperFilter.bits(CachingWrapperFilter.java:58)
>> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:133)
>> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:113)
>> at
>> org.apache.lucene.search.MultiSearcherThread.run(ParallelMultiSearcher.java:250)
>>
>> java.lang.NullPointerException
>> at
>> org.apache.lucene.search.MultiSearcherThread.hits(ParallelMultiSearcher.java:280)
>> at
>> org.apache.lucene.search.ParallelMultiSearcher.search(ParallelMultiSearcher.java:83)
>> at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:74)
>> at org.apache.lucene.search.Hits.<init>(Hits.java:53)
>> at org.apache.lucene.search.Searcher.search(Searcher.java:46)
>>
>> Thanks.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>     
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>   

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message