asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yingyi Bu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ASTERIXDB-1481) Array index out of bound in BTree split during feeds ingestion
Date Wed, 15 Jun 2016 06:36:09 GMT

    [ https://issues.apache.org/jira/browse/ASTERIXDB-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331236#comment-15331236
] 

Yingyi Bu commented on ASTERIXDB-1481:
--------------------------------------

I added some logging to the code as follows.

{noformat}
            if (getLargeFlag()) {
                ((BTreeNSMLeafFrame) rightFrame).growCapacity(extraPageBlockHelper, bufferCache,
                        buf.capacity() - rightFrame.getBuffer().capacity());
                System.out.println("Page re-sizing");
            }

            ByteBuffer right = rightFrame.getBuffer();
            if (buf.capacity() > right.capacity()) {
                System.out.println(
                        "incorrect resizing, buf size: " + buf.capacity() + " right size:
" + right.capacity());
            }
            // Copy entire page.
            System.arraycopy(buf.array(), 0, right.array(), 0, buf.capacity());
{noformat}

I got the following numbers before the exception:

{noformat}
incorrect resizing, buf size: 262144 right size: 131072
java.lang.ArrayIndexOutOfBoundsException
{noformat}

It looks the execution doesn't enter the if-block.


> Array index out of bound in BTree split during feeds ingestion
> --------------------------------------------------------------
>
>                 Key: ASTERIXDB-1481
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1481
>             Project: Apache AsterixDB
>          Issue Type: Bug
>            Reporter: Yingyi Bu
>            Assignee: Michael Blow
>            Priority: Critical
>
> I ran into the following exception when ingesting a large dataset.
> {noformat}
> Jun 14, 2016 4:28:06 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> java.lang.ArrayIndexOutOfBoundsException
> 	at java.lang.System.arraycopy(Native Method)
> 	at org.apache.hyracks.storage.am.btree.frames.BTreeNSMLeafFrame.split(BTreeNSMLeafFrame.java:257)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.performLeafSplit(BTree.java:448)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.insertLeaf(BTree.java:404)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.upsertLeaf(BTree.java:530)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:771)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:673)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.insertUpdateOrDelete(BTree.java:312)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.upsert(BTree.java:345)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.access$500(BTree.java:74)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsertIfConditionElseInsert(BTree.java:953)
> 	at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree.insert(LSMBTree.java:402)
> 	at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree.modify(LSMBTree.java:343)
> 	at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.modify(LSMHarness.java:372)
> 	at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.forceModify(LSMHarness.java:352)
> 	at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.forceInsert(LSMTreeIndexAccessor.java:158)
> 	at org.apache.asterix.runtime.operators.AsterixLSMPrimaryUpsertOperatorNodePushable.nextFrame(AsterixLSMPrimaryUpsertOperatorNodePushable.java:227)
> 	at org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.nextFrame(SyncFeedRuntimeInputHandler.java:46)
> 	at org.apache.asterix.external.operators.FeedMetaStoreNodePushable.nextFrame(FeedMetaStoreNodePushable.java:158)
> 	at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:349)
> 	at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.storage.am.btree.exceptions.BTreeException:
java.lang.ArrayIndexOutOfBoundsException
> 	at org.apache.asterix.runtime.operators.AsterixLSMPrimaryUpsertOperatorNodePushable.nextFrame(AsterixLSMPrimaryUpsertOperatorNodePushable.java:239)
> 	at org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.nextFrame(SyncFeedRuntimeInputHandler.java:46)
> 	at org.apache.asterix.external.operators.FeedMetaStoreNodePushable.nextFrame(FeedMetaStoreNodePushable.java:158)
> 	at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:349)
> 	at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hyracks.storage.am.btree.exceptions.BTreeException: java.lang.ArrayIndexOutOfBoundsException
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:827)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:673)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.insertUpdateOrDelete(BTree.java:312)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.upsert(BTree.java:345)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.access$500(BTree.java:74)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsertIfConditionElseInsert(BTree.java:953)
> 	at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree.insert(LSMBTree.java:402)
> 	at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree.modify(LSMBTree.java:343)
> 	at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.modify(LSMHarness.java:372)
> 	at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.forceModify(LSMHarness.java:352)
> 	at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.forceInsert(LSMTreeIndexAccessor.java:158)
> 	at org.apache.asterix.runtime.operators.AsterixLSMPrimaryUpsertOperatorNodePushable.nextFrame(AsterixLSMPrimaryUpsertOperatorNodePushable.java:227)
> 	... 7 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException
> 	at java.lang.System.arraycopy(Native Method)
> 	at org.apache.hyracks.storage.am.btree.frames.BTreeNSMLeafFrame.split(BTreeNSMLeafFrame.java:257)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.performLeafSplit(BTree.java:448)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.insertLeaf(BTree.java:404)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.upsertLeaf(BTree.java:530)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:771)
> 	... 18 more
> org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.storage.am.btree.exceptions.BTreeException:
java.lang.ArrayIndexOutOfBoundsException
> 	at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365)
> 	at org.apache.hyracks.control.nc.Task.run(Task.java:297)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.api.exceptions.HyracksDataException:
org.apache.hyracks.storage.am.btree.exceptions.BTreeException: java.lang.ArrayIndexOutOfBoundsException
> 	at org.apache.asterix.external.operators.FeedMetaStoreNodePushable.nextFrame(FeedMetaStoreNodePushable.java:161)
> 	at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:349)
> 	... 4 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.hyracks.storage.am.btree.exceptions.BTreeException:
java.lang.ArrayIndexOutOfBoundsException
> 	at org.apache.asterix.runtime.operators.AsterixLSMPrimaryUpsertOperatorNodePushable.nextFrame(AsterixLSMPrimaryUpsertOperatorNodePushable.java:239)
> 	at org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.nextFrame(SyncFeedRuntimeInputHandler.java:46)
> 	at org.apache.asterix.external.operators.FeedMetaStoreNodePushable.nextFrame(FeedMetaStoreNodePushable.java:158)
> 	... 5 more
> Caused by: org.apache.hyracks.storage.am.btree.exceptions.BTreeException: java.lang.ArrayIndexOutOfBoundsException
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:827)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:673)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.insertUpdateOrDelete(BTree.java:312)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.upsert(BTree.java:345)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.access$500(BTree.java:74)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree$BTreeAccessor.upsertIfConditionElseInsert(BTree.java:953)
> 	at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree.insert(LSMBTree.java:402)
> 	at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTree.modify(LSMBTree.java:343)
> 	at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.modify(LSMHarness.java:372)
> 	at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.forceModify(LSMHarness.java:352)
> 	at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.forceInsert(LSMTreeIndexAccessor.java:158)
> 	at org.apache.asterix.runtime.operators.AsterixLSMPrimaryUpsertOperatorNodePushable.nextFrame(AsterixLSMPrimaryUpsertOperatorNodePushable.java:227)
> 	... 7 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException
> 	at java.lang.System.arraycopy(Native Method)
> 	at org.apache.hyracks.storage.am.btree.frames.BTreeNSMLeafFrame.split(BTreeNSMLeafFrame.java:257)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.performLeafSplit(BTree.java:448)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.insertLeaf(BTree.java:404)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.upsertLeaf(BTree.java:530)
> 	at org.apache.hyracks.storage.am.btree.impls.BTree.performOp(BTree.java:771)
> 	... 18 more
> org.apache.hyracks.api.exceptions.HyracksDataException: java.lang.InterruptedException
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:218)
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:83)
> 	at org.apache.hyracks.control.nc.Task.run(Task.java:263)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.InterruptedException
> 	at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404)
> 	at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:212)
> 	... 5 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message