Return-Path: X-Original-To: apmail-giraph-dev-archive@www.apache.org Delivered-To: apmail-giraph-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A0F6EE4C1 for ; Thu, 14 Feb 2013 19:17:14 +0000 (UTC) Received: (qmail 35903 invoked by uid 500); 14 Feb 2013 19:17:13 -0000 Delivered-To: apmail-giraph-dev-archive@giraph.apache.org Received: (qmail 35799 invoked by uid 500); 14 Feb 2013 19:17:13 -0000 Mailing-List: contact dev-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@giraph.apache.org Delivered-To: mailing list dev@giraph.apache.org Received: (qmail 35601 invoked by uid 500); 14 Feb 2013 19:17:13 -0000 Delivered-To: apmail-incubator-giraph-dev@incubator.apache.org Received: (qmail 35529 invoked by uid 99); 14 Feb 2013 19:17:13 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 14 Feb 2013 19:17:13 +0000 Date: Thu, 14 Feb 2013 19:17:13 +0000 (UTC) From: "Claudio Martella (JIRA)" To: giraph-dev@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (GIRAPH-516) out-of-core messages dies for ArrayIndexOutOfBoundsException when running out-of-core messages in UnsafeByteArrayOutputStream MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/GIRAPH-516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Claudio Martella updated GIRAPH-516: ------------------------------------ Description: Can reproduce with: hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar org.apache.giraph.benchmark.PageRankBenchmark -Dgiraph.useOutOfCoreMessages=true -Dgiraph.maxMessagesInMemory=100000 -Dgiraph.numComputeThreads=4 -w 100 -c 0 -e 100 -V 1000000 -v -s 10 2013-02-14 20:11:13,445 WARN org.apache.hadoop.mapred.Child: Error running child java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@615e10ab at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323) at org.apache.hadoop.mapred.Child$4.run(Child.java:270) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177) at org.apache.hadoop.mapred.Child.main(Child.java:264) Caused by: java.lang.IllegalStateException: waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@615e10ab at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:145) at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:105) at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:67) at org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:739) at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:270) at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92) ... 7 more Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232) at java.util.concurrent.FutureTask.get(FutureTask.java:91) at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:232) at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:137) ... 12 more Caused by: java.lang.ArrayIndexOutOfBoundsException at org.apache.giraph.utils.UnsafeByteArrayOutputStream.ensureSize(UnsafeByteArrayOutputStream.java:132) at org.apache.giraph.utils.UnsafeByteArrayOutputStream.writeDouble(UnsafeByteArrayOutputStream.java:248) at org.apache.hadoop.io.DoubleWritable.write(DoubleWritable.java:45) at org.apache.giraph.comm.messages.DiskBackedMessageStore.addVertexMessages(DiskBackedMessageStore.java:118) at org.apache.giraph.comm.messages.DiskBackedMessageStoreByPartition.addPartitionMessages(DiskBackedMessageStoreByPartition.java:91) at org.apache.giraph.comm.requests.SendWorkerMessagesRequest.doRequest(SendWorkerMessagesRequest.java:111) at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:422) at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:381) at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:182) at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) 2013-02-14 20:11:13,570 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task was: Can reproduce with: adoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar org.apache.giraph.benchmark.PageRankBenchmark -Dgiraph.useOutOfCoreMessages=true -Dgiraph.maxMessagesInMemory=100000 -Dgiraph.numComputeThreads=4 -w 100 -c 0 -e 100 -V 1000000 -v -s 10 2013-02-14 20:11:13,445 WARN org.apache.hadoop.mapred.Child: Error running child java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@615e10ab at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323) at org.apache.hadoop.mapred.Child$4.run(Child.java:270) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177) at org.apache.hadoop.mapred.Child.main(Child.java:264) Caused by: java.lang.IllegalStateException: waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@615e10ab at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:145) at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:105) at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:67) at org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:739) at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:270) at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92) ... 7 more Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232) at java.util.concurrent.FutureTask.get(FutureTask.java:91) at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:232) at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:137) ... 12 more Caused by: java.lang.ArrayIndexOutOfBoundsException at org.apache.giraph.utils.UnsafeByteArrayOutputStream.ensureSize(UnsafeByteArrayOutputStream.java:132) at org.apache.giraph.utils.UnsafeByteArrayOutputStream.writeDouble(UnsafeByteArrayOutputStream.java:248) at org.apache.hadoop.io.DoubleWritable.write(DoubleWritable.java:45) at org.apache.giraph.comm.messages.DiskBackedMessageStore.addVertexMessages(DiskBackedMessageStore.java:118) at org.apache.giraph.comm.messages.DiskBackedMessageStoreByPartition.addPartitionMessages(DiskBackedMessageStoreByPartition.java:91) at org.apache.giraph.comm.requests.SendWorkerMessagesRequest.doRequest(SendWorkerMessagesRequest.java:111) at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:422) at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:381) at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:182) at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) 2013-02-14 20:11:13,570 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task > out-of-core messages dies for ArrayIndexOutOfBoundsException when running out-of-core messages in UnsafeByteArrayOutputStream > ----------------------------------------------------------------------------------------------------------------------------- > > Key: GIRAPH-516 > URL: https://issues.apache.org/jira/browse/GIRAPH-516 > Project: Giraph > Issue Type: Bug > Reporter: Claudio Martella > > Can reproduce with: > hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar org.apache.giraph.benchmark.PageRankBenchmark -Dgiraph.useOutOfCoreMessages=true -Dgiraph.maxMessagesInMemory=100000 -Dgiraph.numComputeThreads=4 -w 100 -c 0 -e 100 -V 1000000 -v -s 10 > 2013-02-14 20:11:13,445 WARN org.apache.hadoop.mapred.Child: Error running child > java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@615e10ab > at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323) > at org.apache.hadoop.mapred.Child$4.run(Child.java:270) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177) > at org.apache.hadoop.mapred.Child.main(Child.java:264) > Caused by: java.lang.IllegalStateException: waitFor: ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@615e10ab > at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:145) > at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:105) > at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:67) > at org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:739) > at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:270) > at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92) > ... 7 more > Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232) > at java.util.concurrent.FutureTask.get(FutureTask.java:91) > at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:232) > at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:137) > ... 12 more > Caused by: java.lang.ArrayIndexOutOfBoundsException > at org.apache.giraph.utils.UnsafeByteArrayOutputStream.ensureSize(UnsafeByteArrayOutputStream.java:132) > at org.apache.giraph.utils.UnsafeByteArrayOutputStream.writeDouble(UnsafeByteArrayOutputStream.java:248) > at org.apache.hadoop.io.DoubleWritable.write(DoubleWritable.java:45) > at org.apache.giraph.comm.messages.DiskBackedMessageStore.addVertexMessages(DiskBackedMessageStore.java:118) > at org.apache.giraph.comm.messages.DiskBackedMessageStoreByPartition.addPartitionMessages(DiskBackedMessageStoreByPartition.java:91) > at org.apache.giraph.comm.requests.SendWorkerMessagesRequest.doRequest(SendWorkerMessagesRequest.java:111) > at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:422) > at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:381) > at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:182) > at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > 2013-02-14 20:11:13,570 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira