giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Claudio Martella (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (GIRAPH-516) out-of-core messages dies for ArrayIndexOutOfBoundsException when running out-of-core messages in UnsafeByteArrayOutputStream
Date Thu, 14 Feb 2013 19:17:13 GMT

     [ https://issues.apache.org/jira/browse/GIRAPH-516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Claudio Martella updated GIRAPH-516:
------------------------------------

    Description: 
Can reproduce with:

hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar org.apache.giraph.benchmark.PageRankBenchmark
-Dgiraph.useOutOfCoreMessages=true -Dgiraph.maxMessagesInMemory=100000 -Dgiraph.numComputeThreads=4
-w 100 -c 0 -e 100 -V 1000000 -v -s 10



2013-02-14 20:11:13,445 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException
occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@615e10ab
	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
	at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.IllegalStateException: waitFor: ExecutionException occurred while waiting
for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@615e10ab
	at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:145)
	at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:105)
	at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:67)
	at org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:739)
	at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:270)
	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
	... 7 more
Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException
	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232)
	at java.util.concurrent.FutureTask.get(FutureTask.java:91)
	at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:232)
	at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:137)
	... 12 more
Caused by: java.lang.ArrayIndexOutOfBoundsException
	at org.apache.giraph.utils.UnsafeByteArrayOutputStream.ensureSize(UnsafeByteArrayOutputStream.java:132)
	at org.apache.giraph.utils.UnsafeByteArrayOutputStream.writeDouble(UnsafeByteArrayOutputStream.java:248)
	at org.apache.hadoop.io.DoubleWritable.write(DoubleWritable.java:45)
	at org.apache.giraph.comm.messages.DiskBackedMessageStore.addVertexMessages(DiskBackedMessageStore.java:118)
	at org.apache.giraph.comm.messages.DiskBackedMessageStoreByPartition.addPartitionMessages(DiskBackedMessageStoreByPartition.java:91)
	at org.apache.giraph.comm.requests.SendWorkerMessagesRequest.doRequest(SendWorkerMessagesRequest.java:111)
	at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:422)
	at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:381)
	at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:182)
	at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)
2013-02-14 20:11:13,570 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task

  was:
Can reproduce with:

adoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar org.apache.giraph.benchmark.PageRankBenchmark
-Dgiraph.useOutOfCoreMessages=true -Dgiraph.maxMessagesInMemory=100000 -Dgiraph.numComputeThreads=4
-w 100 -c 0 -e 100 -V 1000000 -v -s 10



2013-02-14 20:11:13,445 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException
occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@615e10ab
	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
	at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.IllegalStateException: waitFor: ExecutionException occurred while waiting
for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@615e10ab
	at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:145)
	at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:105)
	at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:67)
	at org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:739)
	at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:270)
	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
	... 7 more
Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException
	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232)
	at java.util.concurrent.FutureTask.get(FutureTask.java:91)
	at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:232)
	at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:137)
	... 12 more
Caused by: java.lang.ArrayIndexOutOfBoundsException
	at org.apache.giraph.utils.UnsafeByteArrayOutputStream.ensureSize(UnsafeByteArrayOutputStream.java:132)
	at org.apache.giraph.utils.UnsafeByteArrayOutputStream.writeDouble(UnsafeByteArrayOutputStream.java:248)
	at org.apache.hadoop.io.DoubleWritable.write(DoubleWritable.java:45)
	at org.apache.giraph.comm.messages.DiskBackedMessageStore.addVertexMessages(DiskBackedMessageStore.java:118)
	at org.apache.giraph.comm.messages.DiskBackedMessageStoreByPartition.addPartitionMessages(DiskBackedMessageStoreByPartition.java:91)
	at org.apache.giraph.comm.requests.SendWorkerMessagesRequest.doRequest(SendWorkerMessagesRequest.java:111)
	at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:422)
	at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:381)
	at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:182)
	at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)
2013-02-14 20:11:13,570 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task

    
> out-of-core messages dies for ArrayIndexOutOfBoundsException when running out-of-core
messages in UnsafeByteArrayOutputStream
> -----------------------------------------------------------------------------------------------------------------------------
>
>                 Key: GIRAPH-516
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-516
>             Project: Giraph
>          Issue Type: Bug
>            Reporter: Claudio Martella
>
> Can reproduce with:
> hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar org.apache.giraph.benchmark.PageRankBenchmark
-Dgiraph.useOutOfCoreMessages=true -Dgiraph.maxMessagesInMemory=100000 -Dgiraph.numComputeThreads=4
-w 100 -c 0 -e 100 -V 1000000 -v -s 10
> 2013-02-14 20:11:13,445 WARN org.apache.hadoop.mapred.Child: Error running child
> java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException
occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@615e10ab
> 	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:264)
> Caused by: java.lang.IllegalStateException: waitFor: ExecutionException occurred while
waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@615e10ab
> 	at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:145)
> 	at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:105)
> 	at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:67)
> 	at org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:739)
> 	at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:270)
> 	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
> 	... 7 more
> Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException
> 	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232)
> 	at java.util.concurrent.FutureTask.get(FutureTask.java:91)
> 	at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:232)
> 	at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:137)
> 	... 12 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException
> 	at org.apache.giraph.utils.UnsafeByteArrayOutputStream.ensureSize(UnsafeByteArrayOutputStream.java:132)
> 	at org.apache.giraph.utils.UnsafeByteArrayOutputStream.writeDouble(UnsafeByteArrayOutputStream.java:248)
> 	at org.apache.hadoop.io.DoubleWritable.write(DoubleWritable.java:45)
> 	at org.apache.giraph.comm.messages.DiskBackedMessageStore.addVertexMessages(DiskBackedMessageStore.java:118)
> 	at org.apache.giraph.comm.messages.DiskBackedMessageStoreByPartition.addPartitionMessages(DiskBackedMessageStoreByPartition.java:91)
> 	at org.apache.giraph.comm.requests.SendWorkerMessagesRequest.doRequest(SendWorkerMessagesRequest.java:111)
> 	at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:422)
> 	at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:381)
> 	at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:182)
> 	at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:662)
> 2013-02-14 20:11:13,570 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the
task

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message