lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <>
Subject [jira] [Commented] (LUCENE-5658) IllegalArgumentException from ByteBufferIndexInput.buildSlice
Date Sat, 10 May 2014 22:13:38 GMT


Uwe Schindler commented on LUCENE-5658:

bq. I checked the stacktrace against trunk, but not all the lines align (e.g. at$1.openFullSlice(
is only in 4.8), but I haven't dug deeper yet...

We removed that on trunk, because it just calls IndexInput.clone(). We will remove the whole
Slicer in trunk (I think Robert has a patch) and just provide a method on IndexInput to clone
a a slice.

The issue here is not slicing, its happening on openFullSlice() delegating to clone(). This
method is called millions of times while searching an index for opening TermsEnums,...

> IllegalArgumentException from ByteBufferIndexInput.buildSlice
> -------------------------------------------------------------
>                 Key: LUCENE-5658
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/store
>    Affects Versions: 4.8
>            Reporter: Shai Erera
> I've received an email with the following stacktrace:
> {noformat}
> Exception in thread "Lucene Merge Thread #73" org.apache.lucene.index.MergePolicy$MergeException:
> 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(
> 	at org.apache.lucene.index.ConcurrentMergeScheduler$
> Caused by: java.lang.IllegalArgumentException
> 	at java.nio.Buffer.limit(
> 	at
> 	at
> 	at
> 	at$1.openFullSlice(
> 	at
> 	at<init>(
> 	at org.apache.lucene.index.SegmentReader.readFieldInfos(
> 	at org.apache.lucene.index.SegmentReader.<init>(
> 	at org.apache.lucene.index.ReadersAndUpdates.getReader(
> 	at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(
> 	at org.apache.lucene.index.IndexWriter.mergeMiddle(
> 	at org.apache.lucene.index.IndexWriter.merge(
> 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(
> 	at org.apache.lucene.index.ConcurrentMergeScheduler$
> {noformat}
> According to the email, it happens randomly while indexing Wikipedia, on 4.8.0. As far
as I understood, the indexer creates 4 indexes in parallel, by a total of 48 threads. Each
index is created in a separate directory, and there's no sharing of MP or MS instances between
the writers (in fact, default settings are used). This could explain the {{Lucene Merge Thread
#73}}. The indexing process ends w/ a {{forceMerge(1)}}. If that call is omitted, the exception
doesn't reproduce. Also, since it doesn't happen always, there's no simple testcase which
> I've asked the reporter to add more info to the issue, but opening the issue now so we
could check and hopefully fix before 4.8.1.
> I checked the stacktrace against trunk, but not all the lines align (e.g. {{at$1.openFullSlice(}}
is only in 4.8), but I haven't dug deeper yet...

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message