giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Heitmann <benjamin.heitm...@deri.org>
Subject Re: Incomplete output when running PageRank example
Date Sat, 31 Mar 2012 10:45:30 GMT

Hi Robert, 

On 31 Mar 2012, at 09:42, Robert Davis wrote:

> Hello Giraphers,
> 
> I am new to Giraph. I just check out a version and ran it in the single
> machine mode. I got the following results which has no Giraph counter
> information (as those in the example output). I am wondering what has gone
> wrong. The hadoop I am using is 1.0

it looks like your Giraph job did not actually finish the calculation. 

As you say that you are new to Giraph, there might be a high chance that you ran into the
same issue which tripped me up a few weeks ago ;) 

(I am not sure where the following information should be documented, 
maybe this issue should be documented on the same page which describes how to run the pagerank
benchmark)

You provide the parameter "-w 30" to your job, which means that it will use 30 workers. Maybe
thats from the example on the Giraph web page, 
however there is one very important caveat for the number of workers: 
the number of workers needs to be smaller then mapred.tasktracker.map.tasks.maximum minus
one. 

Giraph will use one mapper task to start some sort of coordinating worker (probably something
zookeeper specific), 
and then it will start the number of workers which you specified using -w . If the total number
of workers is bigger then the maximum number of tasks, 
then your Giraph job will not finish actually calculating stuff. 
(There might be a config option for specifying how many workers need to be finished in order
to start the next superstep, 
but I did not try that personally.) 

If you are running Hadoop/Giraph on your personal machine, then I would recommend, using 3
workers, and you should edit your conf/mapred-site.xml 
to include some values for the following configuration parameters (and restart hadoop...)


  <property>
    <name>mapred.map.tasks</name>
    <value>4</value>
  </property>
  <property>
    <name>mapred.reduce.tasks</name>
    <value>4</value>
  </property>
  <property>
    <name>mapred.tasktracker.map.tasks.maximum</name>
    <value>4</value>
  </property>
  <property>
    <name>mapred.tasktracker.reduce.tasks.maximum</name>
    <value>4</value>
  </property>



Mime
View raw message