hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hemanth Yamijala <yhema...@thoughtworks.com>
Subject Re: Reg: Fetching TaskAttempt Details from a RunningJob
Date Tue, 08 Jan 2013 04:30:54 GMT
Hi,

In Hadoop 1.0, I don't think this information is exposed. The
TaskInProgress is an internal class and hence cannot / should not be used
from client applications. The only way out seems to be to screen scrape the
information from the Jobtracker web UI.

If you can live with completed events, then there is something called
TaskCompletionEvents that seem to provide some of this information. You
could look at JobClient.getTaskCompletionEvents.

Please note that in Hadoop 2.0 where mapreduce has been re-architected into
YARN, there are JSON APIs that seem to expose the information you require:

http://hadoop.apache.org/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html#Response_Examples

Look here for "taskAttempts"

Thanks
hemanth


On Sun, Jan 6, 2013 at 8:30 PM, Hadoop Learner
<hadooplearner1986@gmail.com>wrote:

> Hi All,
>
>   Working on a requirement of hadoop Job Monitoring. Requirement is to get
> every Task attempt details of a Running Job. Details are following :
>
> Task Attempt Start Time
> Task Tracker Name where Task Attempt is executing
> Errors or Exceptions in a running Task Attempt
>
> I have Implemented JobClient, RunningJob, JobStatus, TaskReport APIs to
> get the Task Attempt IDs( I have the task Attempt IDs for which I need
> required details). By using these APIs I was unable to get Task Attempt
> details as per my requirement mentioned above. Can you please help me with
> some ways to get required details using Java APIs.
>
> Any pointers will be helpful.
>
> (Note : I have tried using TaskInProgress API, But not sure how to get
> TaskInProgress instance from a task ID. Also noticed TaskInProgress class
> is not visible to my classes)
>
> Thanks and Regards,
> Shyam
>

Mime
View raw message