hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: Showing hadoop status
Date Wed, 11 Nov 2009 10:50:16 GMT
Mark N wrote:
> Currently  I was trying to read the log files generated by hadoop so that I
> could show the status of map/reduce jobs in user interfaces
> (consider that UI is a separate application )
> 
> I was looking at JobTracker APIs and was trying use following APIs
>  1. jobtracker.getAllJobs()  which returns array of jobstatus objects
> 
>  2. Then for each jobStatus we can call
>     mapProgress() , isJobCompleted()  methods
> 
> The problem is how do I link existing jobTracker ( which is already running
> )  with my  code ? i am trying to built something like  " hadoop UI ."
> 
> should I use jobClient  APIs ? Also is this a proper approach ?
> 
> thanks in advance . N mark.
> 

jobclient API is full featured, but brittle against versions. If you 
look at Hadoop Studio, they apparently have added a plug-in layer so 
they can work with different versions of hadoop more gracefully -you may 
want to use their code:  http://www.hadoopstudio.org/

Some of the JSP pages have status, there's a new one that pushes some 
XML content out, but nobody has yet sat down and written a stable, 
secure long-haul interface to the job tracker.

I've talked about it, got some ideas, but not started coding a proper 
RESTful front end
http://www.slideshare.net/steve_l/long-haul-hadoop

-steve

Mime
View raw message