hama-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hama Wiki] Update of "WriteHamaGraphFile" by thomasjungblut
Date Thu, 30 Aug 2012 13:28:26 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hama Wiki" for change notification.

The "WriteHamaGraphFile" page has been changed by thomasjungblut:
http://wiki.apache.org/hama/WriteHamaGraphFile?action=diff&rev1=6&rev2=7

  }}}
  to your Hama cluster. 
  
- '''Troubleshooting'''
- 
- If your job does not execute, your cluster may not have enough resources (task slots). 

- Symptoms may look like this in the bsp master log:
- {{{
- 2012-05-27 20:00:51,228 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
- 2012-05-27 20:00:51,288 INFO org.apache.hama.bsp.JobInProgress: num BSPTasks: 16
- 2012-05-27 20:00:51,305 INFO org.apache.hama.bsp.JobInProgress: Job is initialized.
- 2012-05-27 20:00:51,313 ERROR org.apache.hama.bsp.SimpleTaskScheduler: Scheduling of job
Pagerank could not be done successfully. Killing it!
- 2012-05-27 20:01:08,334 INFO org.apache.hama.bsp.JobInProgress: num BSPTasks: 16
- 2012-05-27 20:01:08,339 INFO org.apache.hama.bsp.JobInProgress: Job is initialized.
- 2012-05-27 20:01:08,340 ERROR org.apache.hama.bsp.SimpleTaskScheduler: Scheduling of job
Pagerank could not be done successfully. Killing it!
- }}}
- 
- This was run on a 8 slot cluster, but it required 16 slots because of 64m chunk size of
HDFS.
- Either you can reupload the file with higher chunksize so the slots match the blocks or
you can increase the slots in your Hama cluster.
- 
  If you sort the result descending by pagerank you can see the following top 10 sites:
  
  {{{
@@ -250, +233 @@

  
  Note that you can decode the indices you may see with the [[http://users.on.net/~henry/pagerank/titles-sorted.zip|page
titels]] files.
  
+ '''Troubleshooting'''
+ 
+ If your job does not execute, your cluster may not have enough resources (task slots). 

+ Symptoms may look like this in the bsp master log:
+ {{{
+ 2012-05-27 20:00:51,228 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
+ 2012-05-27 20:00:51,288 INFO org.apache.hama.bsp.JobInProgress: num BSPTasks: 16
+ 2012-05-27 20:00:51,305 INFO org.apache.hama.bsp.JobInProgress: Job is initialized.
+ 2012-05-27 20:00:51,313 ERROR org.apache.hama.bsp.SimpleTaskScheduler: Scheduling of job
Pagerank could not be done successfully. Killing it!
+ 2012-05-27 20:01:08,334 INFO org.apache.hama.bsp.JobInProgress: num BSPTasks: 16
+ 2012-05-27 20:01:08,339 INFO org.apache.hama.bsp.JobInProgress: Job is initialized.
+ 2012-05-27 20:01:08,340 ERROR org.apache.hama.bsp.SimpleTaskScheduler: Scheduling of job
Pagerank could not be done successfully. Killing it!
+ }}}
+ 
+ This was run on a 8 slot cluster, but it required 16 slots because of 64m chunk size of
HDFS.
+ Either you can reupload the file with higher chunksize so the slots match the blocks or
you can increase the slots in your Hama cluster.
+ 

Mime
View raw message