hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@yahoo-inc.com>
Subject Re: Finding per-task log files on the cluster
Date Tue, 12 Aug 2008 23:51:24 GMT

On Aug 12, 2008, at 11:52 AM, Stuart Sierra wrote:

> Hello, list,
> I've seen this question before, but haven't found an answer.  If I run
> a Hadoop job on a cluster (EC2), how can I download the stdout/stderr
> logs from each mapper/reducer task?  Are they stored somewhere in
> HDFS, or just on the machine that ran the task?

They are usually just present on the nodes, however if you running  
Hadoop on Demand (HoD - http://hadoop.apache.org/core/docs/current/hod.html) 
  you get that feature with it.

OTOH - you might be interested in Chukwa (http://issues.apache.org/jira/browse/HADOOP-3719

) - it's coming in hadoop-0.19 though, which is a while away.


View raw message