giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexandre Fonseca <alexandrejorgefons...@gmail.com>
Subject Re: Exception with out-of-core enabled
Date Mon, 10 Feb 2014 17:56:08 GMT
Hello Sebastian,

Stumbled upon the same exception the other day. Check the following JIRA issue 
and try the applied patch. It appears to work correctly on my cluster.

https://issues.apache.org/jira/browse/GIRAPH-788

Cheers,
Alex

On Monday, February 10, 2014 06:42:42 PM Sebastian Schelter wrote:
> Hi,
> 
> I'm trying to process a huge graph on our cluster and therefore enabled
> out-of-core. Unfortunately, I run into a NullPointerException during the
> input phase.
> 
> What can I do to fix this? I'm using the following settings:
> 
> giraph.oneToAllMsgSending=true
> giraph.isStaticGraph=true
> giraph.numComputeThreads=15
> giraph.numInputThreads=15
> giraph.numOutputThreads=15
> giraph.maxNumberOfSupersteps=30
> giraph.useOutOfCoreGraph=true
> 
> Here's the log for the exception:
> 
> 
> java.lang.IllegalStateException: run: Caught an unrecoverable exception
> waitFor: ExecutionException occurred while waiting for
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@6e172129
> 	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja
> va:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.IllegalStateException: waitFor: ExecutionException
> occurred while waiting for
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@6e172129
> 	at
> org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:181
> ) at
> org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java
> :139) at
> org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java
> :124) at
> org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.
> java:87) at
> org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(Progressa
> bleUtils.java:221) at
> org.apache.giraph.worker.BspServiceWorker.loadInputSplits(BspServiceWorker.j
> ava:281) at
> org.apache.giraph.worker.BspServiceWorker.loadVertices(BspServiceWorker.java
> :325) at
> org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:506)
> 	at
> org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:244)
> 	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)
> 	... 7 more
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalStateException: getOrCreatePartition: cannot retrieve
> partition 14
> 	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:262)
> 	at java.util.concurrent.FutureTask.get(FutureTask.java:119)
> 	at
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(Progressabl
> eUtils.java:300) at
> org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:173
> ) ... 16 more
> Caused by: java.lang.IllegalStateException: getOrCreatePartition: cannot
> retrieve partition 14
> 	at
> org.apache.giraph.partition.DiskBackedPartitionStore.getOrCreatePartition(Di
> skBackedPartitionStore.java:243) at
> org.apache.giraph.comm.requests.SendWorkerVerticesRequest.doRequest(SendWork
> erVerticesRequest.java:110) at
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(Net
> tyWorkerClientRequestProcessor.java:482) at
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.sendVertexReq
> uest(NettyWorkerClientRequestProcessor.java:276) at
> org.apache.giraph.worker.VertexInputSplitsCallable.readInputSplit(VertexInpu
> tSplitsCallable.java:172) at
> org.apache.giraph.worker.InputSplitsCallable.loadInputSplit(InputSplitsCalla
> ble.java:267) at
> org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:2
> 11) at
> org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:6
> 0) at
> org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.jav
> a:51) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> 	at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11
> 10) at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6
> 03) at java.lang.Thread.run(Thread.java:722)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.NullPointerException
> 	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
> 	at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> 	at
> org.apache.giraph.partition.DiskBackedPartitionStore.getOrCreatePartition(Di
> skBackedPartitionStore.java:228) ... 13 more
> Caused by: java.lang.NullPointerException
> 	at
> org.apache.giraph.partition.DiskBackedPartitionStore$GetPartition.call(DiskB
> ackedPartitionStore.java:692) at
> org.apache.giraph.partition.DiskBackedPartitionStore$GetPartition.call(DiskB
> ackedPartitionStore.java:658) at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at
> java.util.concurrent.FutureTask.run(FutureTask.java:166)
> 	at
> org.apache.giraph.partition.DiskBackedPartitionStore$DirectExecutorService.e
> xecute(DiskBackedPartitionStore.java:972) at
> java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.
> java:132) ... 14 more

Mime
View raw message