lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adrien Grand (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-8795) ArrayIndexOutOfBoundsException during System.arraycopy in BKDWriter
Date Tue, 07 May 2019 12:27:00 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-8795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834709#comment-16834709
] 

Adrien Grand commented on LUCENE-8795:
--------------------------------------

I just did a quick review, here is the code for this particular version of Lucene in case
you would like to have a look: https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.6.0/lucene/core/src/java/org/apache/lucene/util/bkd/BKDWriter.java#L1217.

{code}
// restore lastSplitValues to what caller originally passed us:
System.arraycopy(savSplitValue, 0, lastSplitValues, splitDim * bytesPerDim + prefix, suffix);
{code}

The thing that is confusing is that there is another call to arraycopy a couple lines up that
does the opposite copy (https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.6.0/lucene/core/src/java/org/apache/lucene/util/bkd/BKDWriter.java#L1182):

{code}
System.arraycopy(lastSplitValues, splitDim * bytesPerDim + prefix, savSplitValue, 0, suffix);
{code}

It doesn't make sense to me that the arraycopy on line 1217 would fail with an AIOOBE while
the one on line 1182 would not, this looks like a JVM bug to me.

> ArrayIndexOutOfBoundsException during System.arraycopy in BKDWriter
> -------------------------------------------------------------------
>
>                 Key: LUCENE-8795
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8795
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/index
>    Affects Versions: 6.6
>         Environment: h2. Operating system/Platform
> {code:java}
> IBM i - V7R3M0
> {code}
> h2. Java version
> {code:java}
> java version "1.8.0_191"
> Java(TM) SE Runtime Environment (build 8.0.5.25 - pap6480sr5fp25-20181030_01(SR5 FP25))
> IBM J9 VM (build 2.9, JRE 1.8.0 OS/400 ppc64-64-Bit Compressed References 20181029_400846
(JIT enabled, AOT enabled)
> OpenJ9   - c5c78da
> OMR      - 3d5ac33
> IBM      - 8c1bdc2)
> JCL - 20181022_01 based on Oracle jdk8u191-b26
> NOTICE: If no version information is found above, this could indicate a corrupted Java
installation!
> Java detected was: /QOpenSys/QIBM/ProdData/JavaVM/jdk80/64bit/bin/java
> -Dmultiarchive.basepath=/home/NEXTOWN/Multi-Support/Next -Xms128m -Xmx2048m
> {code}
>            Reporter: Torben Riis
>            Priority: Major
>
> Hi,
> I’m a bit stuck here and needs a clue or two in order to continue our investigations.
Hope that someone can help. :)
> Periodically, around once a month, we get the below {{ArrayIndexOutOfBoundsException}}
on our system. We use multiple indexes and the error can originate from any of them, but the
error always occurs in line 1217 in {{BKDWriter}} (during a {{System.arraycopy}}).
> We found a couple of issues on the net regarding JIT optimization problem related to
J9, but they all looks like they have been resolved and cannot be reproduced anymore. But
nevertheless, we have added the following {{-Xjit}} flags to excludes JIT optimization for
every class / inner classes in the {{bkd}} package. Moreover, we have also made a complete
copy of the whole installation in production, and added the opposite arguments (enforce JIT
optimizations for the specific classes). First we will try with {{optLevel=hot}}, but if this
doesn’t show anything we will afterwards try with {{veryHot}} and {{scorching}}. Unfortunately
we do not have the result of this yet, but I’ll of course post it when it is known.
> Unfortunately it’s not possible to run OpenJDK on the IBM i platform, so such a test
will not be possible, but it is worth mentioning that our product is a standard product, which
typically run on the Windows or Linux platform using AdoptOpenJDK. Currently we have a couple
of hundred installations running out there on these platforms, and without any problems. But
on the IBM I platform with J9 we sometimes see this exception.
> Any good ideas for further investigations? Or have seen such issue before?
> We are using Lucene 6.6.0 and runs on IBM J9 on the IBM I platform.
> h2.  Stacktrace
> {code:java}
> Exception in thread "Lucene Merge Thread #0" 2019-05-01T06:10:07.970 CEST [Lucene Merge
Thread #0] org.apache.lucene.index.MergePolicy$MergeException: java.lang.ArrayIndexOutOfBoundsException
>             at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:703)
>             at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:683)
> Caused by: 2019-05-01T06:10:07.971 CEST [Lucene Merge Thread #0] java.lang.ArrayIndexOutOfBoundsException
>             at org.apache.lucene.util.bkd.BKDWriter.recursePackIndex(BKDWriter.java:1217)
>             at org.apache.lucene.util.bkd.BKDWriter.recursePackIndex(BKDWriter.java:1197)
>             at org.apache.lucene.util.bkd.BKDWriter.packIndex(BKDWriter.java:1078)
>             at org.apache.lucene.util.bkd.BKDWriter.writeIndex(BKDWriter.java:1245)
>             at org.apache.lucene.util.bkd.BKDWriter.access$600(BKDWriter.java:82)
>             at org.apache.lucene.util.bkd.BKDWriter$OneDimensionBKDWriter.finish(BKDWriter.java:648)
>             at org.apache.lucene.util.bkd.BKDWriter.merge(BKDWriter.java:560)
>             at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:212)
>             at org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:173)
>             at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:122)
>             at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4356)
>             at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3931)
>             at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:624)
>             at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:661)
> Exception in thread "Lucene Merge Thread #0" 2019-05-01T06:10:08.075 CEST [Lucene Merge
Thread #0] org.apache.lucene.index.MergePolicy$MergeException: java.lang.ArrayIndexOutOfBoundsException
>             at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:703)
>             at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:683)
> Caused by: 2019-05-01T06:10:08.076 CEST [Lucene Merge Thread #0] java.lang.ArrayIndexOutOfBoundsException
>             at org.apache.lucene.util.bkd.BKDWriter.recursePackIndex(BKDWriter.java:1217)
>             at org.apache.lucene.util.bkd.BKDWriter.recursePackIndex(BKDWriter.java:1197)
>             at org.apache.lucene.util.bkd.BKDWriter.packIndex(BKDWriter.java:1078)
>             at org.apache.lucene.util.bkd.BKDWriter.writeIndex(BKDWriter.java:1245)
>             at org.apache.lucene.util.bkd.BKDWriter.access$600(BKDWriter.java:82)
>             at org.apache.lucene.util.bkd.BKDWriter$OneDimensionBKDWriter.finish(BKDWriter.java:648)
>             at org.apache.lucene.util.bkd.BKDWriter.merge(BKDWriter.java:560)
>             at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:212)
>             at org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:173)
>             at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:122)
>             at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4356)
>             at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3931)
>             at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:624)
>             at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler
> {code}
> h2. On production we have disabled jit optimization for the following classes
> {code:java}
> -Xjit:verbose,exclude={org/apache/lucene/util/bkd/BKDWriter.*|org/apache/lucene/util/bkd/BKDReader.*|org/apache/lucene/util/bkd/BKDReader$IndexTree.*|org/apache/lucene/util/bkd/BKDReader$IntersectState.*|org/apache/lucene/util/bkd/BKDReader$LegacyIndexTree.*|org/apache/lucene/util/bkd/BKDReader$PackedIndexTree.*|org/apache/lucene/util/bkd/BKDWriter.*|org/apache/lucene/util/bkd/BKDWriter$OneDimensionBKDWriter.*|org/apache/lucene/util/bkd/BKDWriter$MergeReader.*|org/apache/lucene/util/bkd/BKDWriter$BKDMergeQueue.*|org/apache/lucene/util/bkd/BKDWriter$PathSlice.*|org/apache/lucene/util/bkd/DocIdsWriter.*|org/apache/lucene/util/bkd/HeapPointReader.*|org/apache/lucene/util/bkd/HeapPointWriter.*|org/apache/lucene/util/bkd/MutablePointsReaderUtils.*|org/apache/lucene/util/bkd/OfflinePointReader.*|org/apache/lucene/util/bkd/OfflinePointWriter.*|org/apache/lucene/util/bkd/PointReader.*|org/apache/lucene/util/bkd/PointWriter.*},vlog=/home/nextown/multi-support/next/logs/jit
> {code}
> h2. On the copy of production we have enforced jit optimization with the following arguments
> {code:java}
> -Xjit:{org/apache/lucene/util/bkd/BKDWriter.*|org/apache/lucene/util/bkd/BKDReader.*|org/apache/lucene/util/bkd/BKDReader$IndexTree.*|org/apache/lucene/util/bkd/BKDReader$IntersectState.*|org/apache/lucene/util/bkd/BKDReader$LegacyIndexTree.*|org/apache/lucene/util/bkd/BKDReader$PackedIndexTree.*|org/apache/lucene/util/bkd/BKDWriter.*|org/apache/lucene/util/bkd/BKDWriter$OneDimensionBKDWriter.*|org/apache/lucene/util/bkd/BKDWriter$MergeReader.*|org/apache/lucene/util/bkd/BKDWriter$BKDMergeQueue.*|org/apache/lucene/util/bkd/BKDWriter$PathSlice.*|org/apache/lucene/util/bkd/DocIdsWriter.*|org/apache/lucene/util/bkd/HeapPointReader.*|org/apache/lucene/util/bkd/HeapPointWriter.*|org/apache/lucene/util/bkd/MutablePointsReaderUtils.*|org/apache/lucene/util/bkd/OfflinePointReader.*|org/apache/lucene/util/bkd/OfflinePointWriter.*|org/apache/lucene/util/bkd/PointReader.*|org/apache/lucene/util/bkd/PointWriter.*}(count=0,optlevel=hot)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message