giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Cornell <m...@matthewcornell.org>
Subject Re: Giraph 1.0 | Computation stuck at map 100% - reduce 0% for my algorithm only, at multi-node cluster
Date Tue, 30 Sep 2014 12:24:49 GMT
I'm new, but in my meager experience when it stops at map 100% it means
there was an error somewhere. In Giraph I've often found it difficult to
pin down what that error actually was (e.g., out of memory), but the logs
are the first place to look. Just to clarify re: not finding outputs: Are
you going to http://<your_host.com>:50030/jobtracker.jsp and clicking on
the failed job id (e.g., job_201409251209_0029 ->
http://<your_host.com>:50030/jobdetails.jsp?jobid=job_201409251209_0029&refresh=0
)? From there, click the "map" link in the table to see its tasks. (Giraph
runs entirely as a map task, IIUC.) You should see tasks for the master
plus your workers. If you click on one of them (e.g.,
task_201409251209_0029_m_000000 ->
http://<your_host.com>:50030/taskdetails.jsp?tipid=task_201409251209_0029_m_000000
) you should see what machine it ran on plus a link to the Task Logs. Click
on "All" and you should see three sections for stdout, stderr, and syslog,
the latter of which usually contains hints about what went wrong. You
should check all the worker logs.

Hope that helps.


On Tue, Sep 30, 2014 at 2:53 AM, Panagiotis Eustratiadis <
ep.pan.dit@gmail.com> wrote:

> Good morning,
>
> I have been having a problem the past few days which sadly I can't solve.
>
> First of all I set up a Hadoop 0.20.203.0 cluster of two nodes a master
> and a slave. I followed this tutorial for the settings:
> http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/
>
> Then I set up Giraph, and I built it properly with maven. When I run the
> SimpleShortestPathVertex with number of workers = 2 it runs properly, and
> gives me results which I can view from any of the two nodes. Also the
> jobtracker at master:50030 and slave:50030 and everything else is working
> as expected.
>
> However, when I try to run my own algorithm it hangs at map 100% reduce 0%
> forever. I looked at SimpleShortestPathVertex for any configurations and it
> has none. And the weird part is: the jobs at the jobtracker have no logs at
> stdout or stderr. The only thing readable is the map task info:
>
> task_201409300940_0001_m_000000 | 100.00% - MASTER_ZOOKEEPER_ONLY | 1
> finished out of 2 on superstep -1
> task_201409300940_0001_m_000001 | 100.00% | startSuperstep: WORKER_ONLY -
> Attempt=0, Superstep=-1
> task_201409300940_0001_m_000002 | 100.00% | startSuperstep: WORKER_ONLY -
> Attempt=0, Superstep=-1
>
> Is there anything I'm overlooking? I have Googled the obvious stack
> overflow solutions for two days now. Has anyone encountered anything
> similar?
>
> Regards,
> Panagiotis Eustratiadis.
>



-- 
Matthew Cornell | matt@matthewcornell.org | 413-626-3621 | 34 Dickinson
Street, Amherst MA 01002 | matthewcornell.org

Mime
View raw message