asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Luo (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ASTERIXDB-1872) java.lang.IndexOutOfBoundsException when ingesting data using small memory component size
Date Wed, 05 Apr 2017 16:24:41 GMT

     [ https://issues.apache.org/jira/browse/ASTERIXDB-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chen Luo updated ASTERIXDB-1872:
--------------------------------
    Description: 
I was testing LSM merge policies recently, and thus I used small memory component size (storage.memorycomponent.numpages=8,
storage.buffercache.pagesize=128kb). However, when ingesting some sample tweets, I kept receiving
java.lang.IndexOutOfBoundsException
{code}
java.lang.IndexOutOfBoundsException
	at java.nio.Buffer.checkIndex(Buffer.java:546)
	at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:365)
	at org.apache.hyracks.storage.am.common.frames.LIFOMetaDataFrame.isInner(LIFOMetaDataFrame.java:210)
	at org.apache.hyracks.storage.am.common.frames.LIFOMetaDataFrame.get(LIFOMetaDataFrame.java:174)
	at org.apache.hyracks.storage.am.common.freepage.AppendOnlyLinkedMetadataPageManager.get(AppendOnlyLinkedMetadataPageManager.java:332)
	at org.apache.asterix.common.ioopcallbacks.AbstractLSMIOOperationCallback.getTreeIndexLSN(AbstractLSMIOOperationCallback.java:106)
	at org.apache.asterix.common.ioopcallbacks.LSMBTreeIOOperationCallback.getComponentLSN(LSMBTreeIOOperationCallback.java:72)
	at org.apache.asterix.common.ioopcallbacks.AbstractLSMIOOperationCallback.putLSNIntoMetadata(AbstractLSMIOOperationCallback.java:100)
	at org.apache.asterix.common.ioopcallbacks.LSMBTreeIOOperationCallback.afterOperation(LSMBTreeIOOperationCallback.java:47)
	at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.merge(LSMHarness.java:519)
	at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.merge(LSMTreeIndexAccessor.java:112)
	at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeMergeOperation.call(LSMBTreeMergeOperation.java:83)
	at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeMergeOperation.call(LSMBTreeMergeOperation.java:1)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
{code}

Is this behavior expected?

Steps to reproduce
1. I started a simple cluster with one CC and one NC using org.apache.asterix.api.common.AsterixHyracksIntegrationUtil,
and the configuration file is attached.

2. The sample tweets I used is from "https://github.com/ISG-ICS/cloudberry".
The tweets can be ingested by entering the root directory of the project and typing "./script/ingestTwitterToLocalCluster.sh".
Note: please use the modified ingestTwitterToLocalCluster.sh attached.

  was:
I was testing LSM merge policies recently, and thus I used small memory component size (storage.memorycomponent.numpages=8,
storage.buffercache.pagesize=128kb). However, when ingesting some sample tweets, I kept receiving
java.lang.IndexOutOfBoundsException
<code>
java.lang.IndexOutOfBoundsException
	at java.nio.Buffer.checkIndex(Buffer.java:546)
	at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:365)
	at org.apache.hyracks.storage.am.common.frames.LIFOMetaDataFrame.isInner(LIFOMetaDataFrame.java:210)
	at org.apache.hyracks.storage.am.common.frames.LIFOMetaDataFrame.get(LIFOMetaDataFrame.java:174)
	at org.apache.hyracks.storage.am.common.freepage.AppendOnlyLinkedMetadataPageManager.get(AppendOnlyLinkedMetadataPageManager.java:332)
	at org.apache.asterix.common.ioopcallbacks.AbstractLSMIOOperationCallback.getTreeIndexLSN(AbstractLSMIOOperationCallback.java:106)
	at org.apache.asterix.common.ioopcallbacks.LSMBTreeIOOperationCallback.getComponentLSN(LSMBTreeIOOperationCallback.java:72)
	at org.apache.asterix.common.ioopcallbacks.AbstractLSMIOOperationCallback.putLSNIntoMetadata(AbstractLSMIOOperationCallback.java:100)
	at org.apache.asterix.common.ioopcallbacks.LSMBTreeIOOperationCallback.afterOperation(LSMBTreeIOOperationCallback.java:47)
	at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.merge(LSMHarness.java:519)
	at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.merge(LSMTreeIndexAccessor.java:112)
	at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeMergeOperation.call(LSMBTreeMergeOperation.java:83)
	at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeMergeOperation.call(LSMBTreeMergeOperation.java:1)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
<code>

Is this behavior expected?

Steps to reproduce
1. I started a simple cluster with one CC and one NC using org.apache.asterix.api.common.AsterixHyracksIntegrationUtil,
and the configuration file is attached.

2. The sample tweets I used is from "https://github.com/ISG-ICS/cloudberry".
The tweets can be ingested by entering the root directory of the project and typing "./script/ingestTwitterToLocalCluster.sh".
Note: please use the modified ingestTwitterToLocalCluster.sh attached.


> java.lang.IndexOutOfBoundsException when ingesting data using small memory component
size
> -----------------------------------------------------------------------------------------
>
>                 Key: ASTERIXDB-1872
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1872
>             Project: Apache AsterixDB
>          Issue Type: Bug
>          Components: Feeds, Storage
>            Reporter: Chen Luo
>            Priority: Minor
>         Attachments: asterix-build-configuration-lsm.xml, ingestTwitterToLocalCluster.sh
>
>
> I was testing LSM merge policies recently, and thus I used small memory component size
(storage.memorycomponent.numpages=8, storage.buffercache.pagesize=128kb). However, when ingesting
some sample tweets, I kept receiving java.lang.IndexOutOfBoundsException
> {code}
> java.lang.IndexOutOfBoundsException
> 	at java.nio.Buffer.checkIndex(Buffer.java:546)
> 	at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:365)
> 	at org.apache.hyracks.storage.am.common.frames.LIFOMetaDataFrame.isInner(LIFOMetaDataFrame.java:210)
> 	at org.apache.hyracks.storage.am.common.frames.LIFOMetaDataFrame.get(LIFOMetaDataFrame.java:174)
> 	at org.apache.hyracks.storage.am.common.freepage.AppendOnlyLinkedMetadataPageManager.get(AppendOnlyLinkedMetadataPageManager.java:332)
> 	at org.apache.asterix.common.ioopcallbacks.AbstractLSMIOOperationCallback.getTreeIndexLSN(AbstractLSMIOOperationCallback.java:106)
> 	at org.apache.asterix.common.ioopcallbacks.LSMBTreeIOOperationCallback.getComponentLSN(LSMBTreeIOOperationCallback.java:72)
> 	at org.apache.asterix.common.ioopcallbacks.AbstractLSMIOOperationCallback.putLSNIntoMetadata(AbstractLSMIOOperationCallback.java:100)
> 	at org.apache.asterix.common.ioopcallbacks.LSMBTreeIOOperationCallback.afterOperation(LSMBTreeIOOperationCallback.java:47)
> 	at org.apache.hyracks.storage.am.lsm.common.impls.LSMHarness.merge(LSMHarness.java:519)
> 	at org.apache.hyracks.storage.am.lsm.common.impls.LSMTreeIndexAccessor.merge(LSMTreeIndexAccessor.java:112)
> 	at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeMergeOperation.call(LSMBTreeMergeOperation.java:83)
> 	at org.apache.hyracks.storage.am.lsm.btree.impls.LSMBTreeMergeOperation.call(LSMBTreeMergeOperation.java:1)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> {code}
> Is this behavior expected?
> Steps to reproduce
> 1. I started a simple cluster with one CC and one NC using org.apache.asterix.api.common.AsterixHyracksIntegrationUtil,
and the configuration file is attached.
> 2. The sample tweets I used is from "https://github.com/ISG-ICS/cloudberry".
> The tweets can be ingested by entering the root directory of the project and typing "./script/ingestTwitterToLocalCluster.sh".
> Note: please use the modified ingestTwitterToLocalCluster.sh attached.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message