giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gustavo Enrique Salazar Torres <gsala...@ime.usp.br>
Subject Problems running PageRank benchmark
Date Tue, 11 Dec 2012 20:32:01 GMT
Hi:

I checked out release 0.1 and ,after compiling it, I tried to run this line:

hadoop jar giraph-0.1-jar-with-dependencies.jar
org.apache.giraph.benchmark.PageRankBenchmark -e 1 -s 3 -v -V 50000000 -w 2

but job took too long to finish:

Using org.apache.giraph.benchmark.PageRankBenchmark$PageRankHashMapVertex
12/12/11 18:15:49 WARN bsp.BspOutputFormat: checkOutputSpecs:
ImmutableOutputCommiter will not check anything
12/12/11 18:15:49 INFO mapred.JobClient: Running job: job_201212111806_0003
12/12/11 18:15:50 INFO mapred.JobClient:  map 0% reduce 0%
12/12/11 18:16:06 INFO mapred.JobClient:  map 33% reduce 0%
12/12/11 18:21:11 INFO mapred.JobClient: Job complete: job_201212111806_0003
12/12/11 18:21:11 INFO mapred.JobClient: Counters: 5
12/12/11 18:21:11 INFO mapred.JobClient:   Job Counters
12/12/11 18:21:11 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=619750
12/12/11 18:21:11 INFO mapred.JobClient:     Total time spent by all
reduces waiting after reserving slots (ms)=0
12/12/11 18:21:11 INFO mapred.JobClient:     Total time spent by all maps
waiting after reserving slots (ms)=0
12/12/11 18:21:11 INFO mapred.JobClient:     Launched map tasks=2
12/12/11 18:21:11 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=4181

I checked logs for one of the map tasks and found this:

2012-12-11 18:21:03,929 INFO org.apache.giraph.graph.BspServiceMaster:
checkWorkers: No response from partition 2 (could be master)
2012-12-11 18:21:03,929 ERROR org.apache.giraph.graph.BspServiceMaster:
checkWorkers: Did not receive enough processes in time (only 1 of 2
required).  This occurs if you do not have enough map tasks available
simultaneously on your Hadoop instance to fulfill the number of requested
workers.
2012-12-11 18:21:03,932 INFO org.apache.giraph.graph.BspServiceMaster:
setJobState:
{"_stateKey":"FAILED","_applicationAttemptKey":-1,"_superstepKey":-1} on
superstep -1
2012-12-11 18:21:04,018 FATAL org.apache.giraph.graph.BspServiceMaster:
failJob: Killing job job_201212111806_0003
2012-12-11 18:21:04,142 INFO org.apache.giraph.graph.BspServiceMaster:
cleanup: Notifying master its okay to cleanup with
/_hadoopBsp/job_201212111806_0003/_cleanedUpDir/0_master
2012-12-11 18:21:04,159 INFO org.apache.giraph.graph.BspServiceMaster:
cleanUpZooKeeper: Node /_hadoopBsp/job_201212111806_0003/_cleanedUpDir
already exists, no need to create.
2012-12-11 18:21:04,161 INFO org.apache.giraph.graph.BspServiceMaster:
cleanUpZooKeeper: Got 1 of 3 desired children from
/_hadoopBsp/job_201212111806_0003/_cleanedUpDir
2012-12-11 18:21:04,161 INFO org.apache.giraph.graph.BspServiceMaster:
cleanedUpZooKeeper: Waiting for the children of
/_hadoopBsp/job_201212111806_0003/_cleanedUpDir to change since only got 1
nodes.
2012-12-11 18:21:05,013 WARN org.apache.giraph.zk.ZooKeeperManager:
onlineZooKeeperServers: Forced a shutdown hook kill of the ZooKeeper
process.

My hadoop version is 1.0.3. Is there any special configuration to be done?
Can anybody help me?

Thanks
Gustavo

Mime
View raw message