giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nalezenec, Lukas" <Lukas.Naleze...@Teradata.com>
Subject Re: PartitionBalancing failure on Giraph 1.1.0
Date Mon, 12 Oct 2015 12:16:22 GMT
Hi,
There is a bug in partitioners - see ticket Giraph 886. The ticket contains patch, you can
try to push it to Giraph trunk (I will buy you beer). You can also make your own Partitioner
by coping the original ones and fix the bug only in your code.

Lukas




Hello,

When I change the Partition Balance Algorithm from static to
edges/vertices, Giraph 1.1 quickly crashes. I have checked my configuration
settings to make sure nothing else is out of place,
and the logs are below. Has anyone else gotten this feature working ? Or
have any thoughts
on the problem ?

Thanks,
Kenrick

======
Error logs

ERROR [compute-0] org.apache.giraph.utils.LogStacktraceCallable:
Execution of callable failed
java.lang.NullPointerException
        at org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:132)
        at org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:50)
        at org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:232)
        at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:161)
        at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:69)
        at org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
2015-09-21 20:58:02,575 ERROR [main]
org.apache.giraph.graph.GraphMapper: Caught an unrecoverable exception
waitFor: ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@4bb488d0
java.lang.IllegalStateException: waitFor: ExecutionException occurred
while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@4bb488d0
        at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:193)
        at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:151)
        at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:136)
        at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:99)
        at org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:233)
        at org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:838)
        at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:415)
        at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:93)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:202)
        at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:312)
        at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:185)
        ... 14 more
Caused by: java.lang.NullPointerException
        at org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:132)
        at org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:50)
        at org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:232)
        at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:161)
        at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:69)
        at org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
2015-09-21 20:58:02,575 ERROR [main]
org.apache.giraph.worker.BspServiceWorker: unregisterHealth: Got
failure, unregistering health on
/_hadoopBsp/job_1442858202015_0001/_applicationAttemptsDir/0/_superstepDir/0/_workerHealthyDir/n06_0
on superstep 0
2015-09-21 20:58:02,579 WARN [main]
org.apache.hadoop.mapred.YarnChild: Exception running child :
java.lang.IllegalStateException: run: Caught an unrecoverable
exception waitFor: ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@4bb488d0
        at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:104)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.IllegalStateException: waitFor:
ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@4bb488d0
        at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:193)
        at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:151)
        at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:136)
        at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:99)
        at org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:233)
        at org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:838)
        at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:415)
        at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:93)
        ... 7 more
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:202)
        at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:312)
        at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:185)
        ... 14 more
Caused by: java.lang.NullPointerException
        at org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:132)
        at org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:50)
        at org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:232)
        at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:161)
        at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:69)
        at org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

======


Mime
View raw message