asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Luo (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ASTERIXDB-1812) OutofMemoryError when group by on a non-existing field with 300k records (tweets)
Date Sat, 25 Feb 2017 17:26:44 GMT

     [ https://issues.apache.org/jira/browse/ASTERIXDB-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chen Luo updated ASTERIXDB-1812:
--------------------------------
    Description: 
The dataset is a sample tweet dataset provided by Cloudberry, which contains 324000 tweets
(about 300M). When issuing the following query, I always get an OutofMemoryError.

Query:
select * from twitter.ds_tweet t
group by t.test;

Stacktrace:
org.apache.hyracks.api.exceptions.HyracksException: Job failed on account of:
HYR0003: java.lang.OutOfMemoryError: Java heap space

	at org.apache.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:211)
	at org.apache.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: HYR0003: java.lang.OutOfMemoryError:
Java heap space
	at org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:62)
	at org.apache.hyracks.control.nc.Task.run(Task.java:330)
	... 3 more
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: java.lang.OutOfMemoryError:
Java heap space
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:228)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:84)
	at org.apache.hyracks.control.nc.Task.run(Task.java:273)
	... 3 more
Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap
space
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:222)
	... 5 more
Caused by: java.lang.OutOfMemoryError: Java heap space
	at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
	at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
	at org.apache.hyracks.control.nc.resources.memory.FrameManager.allocateFrame(FrameManager.java:57)
	at org.apache.hyracks.control.nc.resources.memory.FrameManager.reallocateFrame(FrameManager.java:73)
	at org.apache.hyracks.control.nc.Joblet.reallocateFrame(Joblet.java:242)
	at org.apache.hyracks.control.nc.Task.reallocateFrame(Task.java:136)
	at org.apache.hyracks.api.comm.VSizeFrame.ensureFrameSize(VSizeFrame.java:53)
	at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.canHoldNewTuple(AbstractFrameAppender.java:104)
	at org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppender.append(FrameTupleAppender.java:49)
	at org.apache.hyracks.dataflow.common.comm.util.FrameUtils.appendToWriter(FrameUtils.java:159)
	at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendToFrameFromTupleBuilder(AbstractOneInputOneOutputOneFramePushRuntime.java:82)
	at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendToFrameFromTupleBuilder(AbstractOneInputOneOutputOneFramePushRuntime.java:78)
	at org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.nextFrame(AssignRuntimeFactory.java:150)
	at org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.nextFrame(AlgebricksMetaOperatorDescriptor.java:134)
	at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.write(AbstractFrameAppender.java:92)
	at org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppenderWrapper.write(FrameTupleAppenderWrapper.java:50)
	at org.apache.hyracks.dataflow.std.group.preclustered.PreclusteredGroupWriter.close(PreclusteredGroupWriter.java:189)
	at org.apache.hyracks.dataflow.std.group.preclustered.PreclusteredGroupOperatorNodePushable.close(PreclusteredGroupOperatorNodePushable.java:77)
	at org.apache.hyracks.dataflow.std.sort.AbstractExternalSortRunMerger.process(AbstractExternalSortRunMerger.java:165)
	at org.apache.hyracks.dataflow.std.sort.AbstractSorterOperatorDescriptor$MergeActivity$1.initialize(AbstractSorterOperatorDescriptor.java:181)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$initialize$0(SuperActivityOperatorNodePushable.java:86)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$17/1550206216.runAction(Unknown
Source)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$runInParallel$2(SuperActivityOperatorNodePushable.java:216)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$18/914923531.call(Unknown
Source)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	... 3 more

Feb 25, 2017 9:11:13 AM org.apache.asterix.api.http.servlet.APIServlet doPost
SEVERE: Job failed on account of:
HYR0003: java.lang.OutOfMemoryError: Java heap space

org.apache.hyracks.api.exceptions.HyracksException: Job failed on account of:
HYR0003: java.lang.OutOfMemoryError: Java heap space

	at org.apache.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:211)
	at org.apache.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: HYR0003: java.lang.OutOfMemoryError:
Java heap space
	at org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:62)
	at org.apache.hyracks.control.nc.Task.run(Task.java:330)
	... 3 more
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: java.lang.OutOfMemoryError:
Java heap space
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:228)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:84)
	at org.apache.hyracks.control.nc.Task.run(Task.java:273)
	... 3 more
Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap
space
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:222)
	... 5 more
Caused by: java.lang.OutOfMemoryError: Java heap space
	at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
	at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
	at org.apache.hyracks.control.nc.resources.memory.FrameManager.allocateFrame(FrameManager.java:57)
	at org.apache.hyracks.control.nc.resources.memory.FrameManager.reallocateFrame(FrameManager.java:73)
	at org.apache.hyracks.control.nc.Joblet.reallocateFrame(Joblet.java:242)
	at org.apache.hyracks.control.nc.Task.reallocateFrame(Task.java:136)
	at org.apache.hyracks.api.comm.VSizeFrame.ensureFrameSize(VSizeFrame.java:53)
	at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.canHoldNewTuple(AbstractFrameAppender.java:104)
	at org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppender.append(FrameTupleAppender.java:49)
	at org.apache.hyracks.dataflow.common.comm.util.FrameUtils.appendToWriter(FrameUtils.java:159)
	at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendToFrameFromTupleBuilder(AbstractOneInputOneOutputOneFramePushRuntime.java:82)
	at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendToFrameFromTupleBuilder(AbstractOneInputOneOutputOneFramePushRuntime.java:78)
	at org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.nextFrame(AssignRuntimeFactory.java:150)
	at org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.nextFrame(AlgebricksMetaOperatorDescriptor.java:134)
	at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.write(AbstractFrameAppender.java:92)
	at org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppenderWrapper.write(FrameTupleAppenderWrapper.java:50)
	at org.apache.hyracks.dataflow.std.group.preclustered.PreclusteredGroupWriter.close(PreclusteredGroupWriter.java:189)
	at org.apache.hyracks.dataflow.std.group.preclustered.PreclusteredGroupOperatorNodePushable.close(PreclusteredGroupOperatorNodePushable.java:77)
	at org.apache.hyracks.dataflow.std.sort.AbstractExternalSortRunMerger.process(AbstractExternalSortRunMerger.java:165)
	at org.apache.hyracks.dataflow.std.sort.AbstractSorterOperatorDescriptor$MergeActivity$1.initialize(AbstractSorterOperatorDescriptor.java:181)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$initialize$0(SuperActivityOperatorNodePushable.java:86)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$17/1550206216.runAction(Unknown
Source)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$runInParallel$2(SuperActivityOperatorNodePushable.java:216)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$18/914923531.call(Unknown
Source)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	... 3 more

Reproduce steps:
1. Install a local AsterixDB cluster from https://asterixdb.apache.org/docs/0.9.0/install.html#Section1SingleMachineAsterixDBInstallation.
2. Load sample data from CloudBerry.
-2.1 Download CloudBerry project from https://github.com/ISG-ICS/cloudberry
-2.2 Go to CloudBerry dir, and ingest sample tweets using "bin/ingestTwitterToLocalCluster.sh".
You might need to change the Asterix Cluster IP address at line 23, and the cluster instance
name at line 86.
3. Issue the following SQL++ query:
select * from twitter.ds_tweet t
group by t.test;

  was:
The dataset is a sample tweet dataset provided by Cloudberry, which contains 324000 tweets.
When issuing the following query, I always get an OutofMemoryError.

Query:
select * from twitter.ds_tweet t
group by t.test;

Stacktrace:
org.apache.hyracks.api.exceptions.HyracksException: Job failed on account of:
HYR0003: java.lang.OutOfMemoryError: Java heap space

	at org.apache.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:211)
	at org.apache.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: HYR0003: java.lang.OutOfMemoryError:
Java heap space
	at org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:62)
	at org.apache.hyracks.control.nc.Task.run(Task.java:330)
	... 3 more
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: java.lang.OutOfMemoryError:
Java heap space
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:228)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:84)
	at org.apache.hyracks.control.nc.Task.run(Task.java:273)
	... 3 more
Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap
space
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:222)
	... 5 more
Caused by: java.lang.OutOfMemoryError: Java heap space
	at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
	at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
	at org.apache.hyracks.control.nc.resources.memory.FrameManager.allocateFrame(FrameManager.java:57)
	at org.apache.hyracks.control.nc.resources.memory.FrameManager.reallocateFrame(FrameManager.java:73)
	at org.apache.hyracks.control.nc.Joblet.reallocateFrame(Joblet.java:242)
	at org.apache.hyracks.control.nc.Task.reallocateFrame(Task.java:136)
	at org.apache.hyracks.api.comm.VSizeFrame.ensureFrameSize(VSizeFrame.java:53)
	at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.canHoldNewTuple(AbstractFrameAppender.java:104)
	at org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppender.append(FrameTupleAppender.java:49)
	at org.apache.hyracks.dataflow.common.comm.util.FrameUtils.appendToWriter(FrameUtils.java:159)
	at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendToFrameFromTupleBuilder(AbstractOneInputOneOutputOneFramePushRuntime.java:82)
	at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendToFrameFromTupleBuilder(AbstractOneInputOneOutputOneFramePushRuntime.java:78)
	at org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.nextFrame(AssignRuntimeFactory.java:150)
	at org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.nextFrame(AlgebricksMetaOperatorDescriptor.java:134)
	at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.write(AbstractFrameAppender.java:92)
	at org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppenderWrapper.write(FrameTupleAppenderWrapper.java:50)
	at org.apache.hyracks.dataflow.std.group.preclustered.PreclusteredGroupWriter.close(PreclusteredGroupWriter.java:189)
	at org.apache.hyracks.dataflow.std.group.preclustered.PreclusteredGroupOperatorNodePushable.close(PreclusteredGroupOperatorNodePushable.java:77)
	at org.apache.hyracks.dataflow.std.sort.AbstractExternalSortRunMerger.process(AbstractExternalSortRunMerger.java:165)
	at org.apache.hyracks.dataflow.std.sort.AbstractSorterOperatorDescriptor$MergeActivity$1.initialize(AbstractSorterOperatorDescriptor.java:181)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$initialize$0(SuperActivityOperatorNodePushable.java:86)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$17/1550206216.runAction(Unknown
Source)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$runInParallel$2(SuperActivityOperatorNodePushable.java:216)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$18/914923531.call(Unknown
Source)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	... 3 more

Feb 25, 2017 9:11:13 AM org.apache.asterix.api.http.servlet.APIServlet doPost
SEVERE: Job failed on account of:
HYR0003: java.lang.OutOfMemoryError: Java heap space

org.apache.hyracks.api.exceptions.HyracksException: Job failed on account of:
HYR0003: java.lang.OutOfMemoryError: Java heap space

	at org.apache.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:211)
	at org.apache.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: HYR0003: java.lang.OutOfMemoryError:
Java heap space
	at org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:62)
	at org.apache.hyracks.control.nc.Task.run(Task.java:330)
	... 3 more
Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: java.lang.OutOfMemoryError:
Java heap space
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:228)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:84)
	at org.apache.hyracks.control.nc.Task.run(Task.java:273)
	... 3 more
Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap
space
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:222)
	... 5 more
Caused by: java.lang.OutOfMemoryError: Java heap space
	at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
	at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
	at org.apache.hyracks.control.nc.resources.memory.FrameManager.allocateFrame(FrameManager.java:57)
	at org.apache.hyracks.control.nc.resources.memory.FrameManager.reallocateFrame(FrameManager.java:73)
	at org.apache.hyracks.control.nc.Joblet.reallocateFrame(Joblet.java:242)
	at org.apache.hyracks.control.nc.Task.reallocateFrame(Task.java:136)
	at org.apache.hyracks.api.comm.VSizeFrame.ensureFrameSize(VSizeFrame.java:53)
	at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.canHoldNewTuple(AbstractFrameAppender.java:104)
	at org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppender.append(FrameTupleAppender.java:49)
	at org.apache.hyracks.dataflow.common.comm.util.FrameUtils.appendToWriter(FrameUtils.java:159)
	at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendToFrameFromTupleBuilder(AbstractOneInputOneOutputOneFramePushRuntime.java:82)
	at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendToFrameFromTupleBuilder(AbstractOneInputOneOutputOneFramePushRuntime.java:78)
	at org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.nextFrame(AssignRuntimeFactory.java:150)
	at org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.nextFrame(AlgebricksMetaOperatorDescriptor.java:134)
	at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.write(AbstractFrameAppender.java:92)
	at org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppenderWrapper.write(FrameTupleAppenderWrapper.java:50)
	at org.apache.hyracks.dataflow.std.group.preclustered.PreclusteredGroupWriter.close(PreclusteredGroupWriter.java:189)
	at org.apache.hyracks.dataflow.std.group.preclustered.PreclusteredGroupOperatorNodePushable.close(PreclusteredGroupOperatorNodePushable.java:77)
	at org.apache.hyracks.dataflow.std.sort.AbstractExternalSortRunMerger.process(AbstractExternalSortRunMerger.java:165)
	at org.apache.hyracks.dataflow.std.sort.AbstractSorterOperatorDescriptor$MergeActivity$1.initialize(AbstractSorterOperatorDescriptor.java:181)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$initialize$0(SuperActivityOperatorNodePushable.java:86)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$17/1550206216.runAction(Unknown
Source)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$runInParallel$2(SuperActivityOperatorNodePushable.java:216)
	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$18/914923531.call(Unknown
Source)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	... 3 more

Reproduce steps:
1. Install a local AsterixDB cluster from https://asterixdb.apache.org/docs/0.9.0/install.html#Section1SingleMachineAsterixDBInstallation.
2. Load sample data from CloudBerry.
-2.1 Download CloudBerry project from https://github.com/ISG-ICS/cloudberry
-2.2 Go to CloudBerry dir, and ingest sample tweets using "bin/ingestTwitterToLocalCluster.sh".
You might need to change the Asterix Cluster IP address at line 23, and the cluster instance
name at line 86.
3. Issue the following SQL++ query:
select * from twitter.ds_tweet t
group by t.test;


> OutofMemoryError when group by on a non-existing field with 300k records (tweets)
> ---------------------------------------------------------------------------------
>
>                 Key: ASTERIXDB-1812
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1812
>             Project: Apache AsterixDB
>          Issue Type: Bug
>          Components: AsterixDB, Hyracks
>         Environment: Linux 16.04
> Asterix 0.9.0 with 2 nc nodes and 1 cc node. (all using default configurations from https://asterixdb.apache.org/docs/0.9.0/install.html#Section1SingleMachineAsterixDBInstallation)
>            Reporter: Chen Luo
>
> The dataset is a sample tweet dataset provided by Cloudberry, which contains 324000 tweets
(about 300M). When issuing the following query, I always get an OutofMemoryError.
> Query:
> select * from twitter.ds_tweet t
> group by t.test;
> Stacktrace:
> org.apache.hyracks.api.exceptions.HyracksException: Job failed on account of:
> HYR0003: java.lang.OutOfMemoryError: Java heap space
> 	at org.apache.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:211)
> 	at org.apache.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:48)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: HYR0003: java.lang.OutOfMemoryError:
Java heap space
> 	at org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:62)
> 	at org.apache.hyracks.control.nc.Task.run(Task.java:330)
> 	... 3 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: java.lang.OutOfMemoryError:
Java heap space
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:228)
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:84)
> 	at org.apache.hyracks.control.nc.Task.run(Task.java:273)
> 	... 3 more
> Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java
heap space
> 	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> 	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:222)
> 	... 5 more
> Caused by: java.lang.OutOfMemoryError: Java heap space
> 	at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
> 	at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> 	at org.apache.hyracks.control.nc.resources.memory.FrameManager.allocateFrame(FrameManager.java:57)
> 	at org.apache.hyracks.control.nc.resources.memory.FrameManager.reallocateFrame(FrameManager.java:73)
> 	at org.apache.hyracks.control.nc.Joblet.reallocateFrame(Joblet.java:242)
> 	at org.apache.hyracks.control.nc.Task.reallocateFrame(Task.java:136)
> 	at org.apache.hyracks.api.comm.VSizeFrame.ensureFrameSize(VSizeFrame.java:53)
> 	at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.canHoldNewTuple(AbstractFrameAppender.java:104)
> 	at org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppender.append(FrameTupleAppender.java:49)
> 	at org.apache.hyracks.dataflow.common.comm.util.FrameUtils.appendToWriter(FrameUtils.java:159)
> 	at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendToFrameFromTupleBuilder(AbstractOneInputOneOutputOneFramePushRuntime.java:82)
> 	at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendToFrameFromTupleBuilder(AbstractOneInputOneOutputOneFramePushRuntime.java:78)
> 	at org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.nextFrame(AssignRuntimeFactory.java:150)
> 	at org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.nextFrame(AlgebricksMetaOperatorDescriptor.java:134)
> 	at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.write(AbstractFrameAppender.java:92)
> 	at org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppenderWrapper.write(FrameTupleAppenderWrapper.java:50)
> 	at org.apache.hyracks.dataflow.std.group.preclustered.PreclusteredGroupWriter.close(PreclusteredGroupWriter.java:189)
> 	at org.apache.hyracks.dataflow.std.group.preclustered.PreclusteredGroupOperatorNodePushable.close(PreclusteredGroupOperatorNodePushable.java:77)
> 	at org.apache.hyracks.dataflow.std.sort.AbstractExternalSortRunMerger.process(AbstractExternalSortRunMerger.java:165)
> 	at org.apache.hyracks.dataflow.std.sort.AbstractSorterOperatorDescriptor$MergeActivity$1.initialize(AbstractSorterOperatorDescriptor.java:181)
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$initialize$0(SuperActivityOperatorNodePushable.java:86)
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$17/1550206216.runAction(Unknown
Source)
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$runInParallel$2(SuperActivityOperatorNodePushable.java:216)
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$18/914923531.call(Unknown
Source)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	... 3 more
> Feb 25, 2017 9:11:13 AM org.apache.asterix.api.http.servlet.APIServlet doPost
> SEVERE: Job failed on account of:
> HYR0003: java.lang.OutOfMemoryError: Java heap space
> org.apache.hyracks.api.exceptions.HyracksException: Job failed on account of:
> HYR0003: java.lang.OutOfMemoryError: Java heap space
> 	at org.apache.hyracks.control.cc.job.JobRun.waitForCompletion(JobRun.java:211)
> 	at org.apache.hyracks.control.cc.work.WaitForJobCompletionWork$1.run(WaitForJobCompletionWork.java:48)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: HYR0003: java.lang.OutOfMemoryError:
Java heap space
> 	at org.apache.hyracks.control.common.utils.ExceptionUtils.setNodeIds(ExceptionUtils.java:62)
> 	at org.apache.hyracks.control.nc.Task.run(Task.java:330)
> 	... 3 more
> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: java.lang.OutOfMemoryError:
Java heap space
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:228)
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.initialize(SuperActivityOperatorNodePushable.java:84)
> 	at org.apache.hyracks.control.nc.Task.run(Task.java:273)
> 	... 3 more
> Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java
heap space
> 	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> 	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.runInParallel(SuperActivityOperatorNodePushable.java:222)
> 	... 5 more
> Caused by: java.lang.OutOfMemoryError: Java heap space
> 	at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
> 	at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
> 	at org.apache.hyracks.control.nc.resources.memory.FrameManager.allocateFrame(FrameManager.java:57)
> 	at org.apache.hyracks.control.nc.resources.memory.FrameManager.reallocateFrame(FrameManager.java:73)
> 	at org.apache.hyracks.control.nc.Joblet.reallocateFrame(Joblet.java:242)
> 	at org.apache.hyracks.control.nc.Task.reallocateFrame(Task.java:136)
> 	at org.apache.hyracks.api.comm.VSizeFrame.ensureFrameSize(VSizeFrame.java:53)
> 	at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.canHoldNewTuple(AbstractFrameAppender.java:104)
> 	at org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppender.append(FrameTupleAppender.java:49)
> 	at org.apache.hyracks.dataflow.common.comm.util.FrameUtils.appendToWriter(FrameUtils.java:159)
> 	at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendToFrameFromTupleBuilder(AbstractOneInputOneOutputOneFramePushRuntime.java:82)
> 	at org.apache.hyracks.algebricks.runtime.operators.base.AbstractOneInputOneOutputOneFramePushRuntime.appendToFrameFromTupleBuilder(AbstractOneInputOneOutputOneFramePushRuntime.java:78)
> 	at org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.nextFrame(AssignRuntimeFactory.java:150)
> 	at org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.nextFrame(AlgebricksMetaOperatorDescriptor.java:134)
> 	at org.apache.hyracks.dataflow.common.comm.io.AbstractFrameAppender.write(AbstractFrameAppender.java:92)
> 	at org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppenderWrapper.write(FrameTupleAppenderWrapper.java:50)
> 	at org.apache.hyracks.dataflow.std.group.preclustered.PreclusteredGroupWriter.close(PreclusteredGroupWriter.java:189)
> 	at org.apache.hyracks.dataflow.std.group.preclustered.PreclusteredGroupOperatorNodePushable.close(PreclusteredGroupOperatorNodePushable.java:77)
> 	at org.apache.hyracks.dataflow.std.sort.AbstractExternalSortRunMerger.process(AbstractExternalSortRunMerger.java:165)
> 	at org.apache.hyracks.dataflow.std.sort.AbstractSorterOperatorDescriptor$MergeActivity$1.initialize(AbstractSorterOperatorDescriptor.java:181)
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$initialize$0(SuperActivityOperatorNodePushable.java:86)
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$17/1550206216.runAction(Unknown
Source)
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable.lambda$runInParallel$2(SuperActivityOperatorNodePushable.java:216)
> 	at org.apache.hyracks.api.rewriter.runtime.SuperActivityOperatorNodePushable$$Lambda$18/914923531.call(Unknown
Source)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	... 3 more
> Reproduce steps:
> 1. Install a local AsterixDB cluster from https://asterixdb.apache.org/docs/0.9.0/install.html#Section1SingleMachineAsterixDBInstallation.
> 2. Load sample data from CloudBerry.
> -2.1 Download CloudBerry project from https://github.com/ISG-ICS/cloudberry
> -2.2 Go to CloudBerry dir, and ingest sample tweets using "bin/ingestTwitterToLocalCluster.sh".
You might need to change the Asterix Cluster IP address at line 23, and the cluster instance
name at line 86.
> 3. Issue the following SQL++ query:
> select * from twitter.ds_tweet t
> group by t.test;



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message