giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ramesh Krishnan <ramesh.154...@gmail.com>
Subject Fwd: Out of core thread exception : java concurrent exception
Date Sat, 14 May 2016 19:43:56 GMT
Hi Team,

I have the latest build of giraph running on a 5 node cluster. When i try
to use OutofCore Graph option for a huge data set like 600Milion edges i am
running into

the following exception. Please find below the script being executed and
the exception logs. I have tried all possible ways and could not avoid this
issue , i am really in need of your help.

*Script:*hadoop jar
/usr/local/giraph.back.1.2.0/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.7.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner -Dmapreduce.task.timeout=12000000
-Dmapred.job.tracker=ip-172-31-42-220.eu-west-1.compute.internal:8021
-Dmapreduce.map.memory.mb=23480 -Dmapreduce.map.java.opts=-Xmx22480m
org.apache.giraph.examples.ConnectedComponentsComputation   -vif
org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip /test/input_10M
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/test/ouput_10M -w 5 -ca
giraph.userPartitionCount=150,giraph.SplitMasterWorker=true,giraph.isStaticGraph=true,giraph.maxPartitionsInMemory=10,mapred.map.max.attempts=2,giraph.maxMessagesInMemory=100,giraph.numOutputThreads=10,giraph.useOutOfCoreMessages=true,giraph.numOutputThreads=4,giraph.numInputThreads=4,giraph.useOutOfCoreGraph=true,giraph.cleanupCheckpointsAfterSuccess=true,giraph.checkpointFrequency=1




*Exception:hadoop jar
/usr/local/giraph.back.1.2.0/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.7.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner -Dmapreduce.task.timeout=12000000
-Dmapred.job.tracker=ip-172-31-42-220.eu-west-1.compute.internal:8021
-Dmapreduce.map.memory.mb=23480 -Dmapreduce.map.java.opts=-Xmx22480m
org.apache.giraph.examples.ConnectedComponentsComputation   -vif
org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip /test/input_10M
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/test/ouput_10M -w 5 -ca
giraph.userPartitionCount=150,giraph.SplitMasterWorker=true,giraph.isStaticGraph=true,giraph.maxPartitionsInMemory=10,mapred.map.max.attempts=2,giraph.maxMessagesInMemory=100,giraph.numOutputThreads=10,giraph.useOutOfCoreMessages=true,giraph.numOutputThreads=4,giraph.numInputThreads=4,giraph.useOutOfCoreGraph=true,giraph.cleanupCheckpointsAfterSuccess=true,giraph.checkpointFrequency=1*

*thanks*

*Ramesh*

Mime
View raw message