hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From andronat_asf <andronat_...@hotmail.com>
Subject Write Hama GraphFile problem
Date Sat, 09 Mar 2013 18:32:19 GMT
Hello everyone,

I was reading the tutorial for Google Web dataset (local mode, pseudo distributed cluser)
at http://wiki.apache.org/hama/WriteHamaGraphFile.

I downloaded a Graph (~1GB) and uploaded it to hdfs. The file was splitted in 17 hdfs chunks.
(I remind that I am trying to run in pseudo distributed mode, I have everything in my laptop).

ls -l /tmp/hadoop/dfs/data/current/  | grep -v 'meta' | wc -l
     17

Then I tried to run a code I wrote based on the example but:

# tail -f hama-my-bspmaster-my.local.log 

2013-03-09 20:09:47,060 INFO org.apache.hama.bsp.JobInProgress: num BSPTasks: 17
2013-03-09 20:09:47,100 INFO org.apache.hama.bsp.JobInProgress: Job is initialized.
2013-03-09 20:09:47,103 ERROR org.apache.hama.bsp.SimpleTaskScheduler: Could not schedule
all tasks!
2013-03-09 20:09:47,103 ERROR org.apache.hama.bsp.SimpleTaskScheduler: Scheduling of job Hama
Graph Loader could not be done successfully. Killing it!

My configuration is:

HamaConfiguration conf = new HamaConfiguration(new Configuration());
conf.set("bsp.local.tasks.maximum", "1");

GraphJob graphJob = new GraphJob(conf, HamaGraphLoader.class);
graphJob.setNumBspTask(1);

I also tried to change the values of:

conf.set("bsp.tasks.maximum", "1");
conf.set("bsp.max.tasks.per.job", "1");
conf.set("mapred.map.tasks", "1");
conf.set("mapred.min.split.size", String.valueOf(Long.MAX_VALUE));

I even changed the according variables from hama-default.xml but nothing seems to be working.
The number of BSPTasks remain to 17.

I attach my full configuration just in case.

Thank you in advance,
Anastasis


Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message