hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Whitecross <swhitecr...@gmail.com>
Subject Re: Listing Hadoop Job History Statistics
Date Tue, 17 Aug 2010 02:06:03 GMT
Thanks for the answers Doug and Arun.   I'm assuming the job-history files
mentioned are in ./hadoop-0.20/logs/history/done/.  The files look like they
were serialized by a class in Hadoop?  (If I can read the files back into
the appropriate class, and then dump them out into a custom format, that'd
be great.)

Thanks.



On Thu, Aug 12, 2010 at 12:52 AM, Arun C Murthy <acm@yahoo-inc.com> wrote:

> Moving to mapreduce-user@, bcc general@.
>
> There isn't a direct way. One possible option is just use the per-job
> job-history file which is on HDFS (See
> http://hadoop.apache.org/common/docs/r0.20.0/mapred_tutorial.html#Job+Submission+and+Monitoringfor
info on job-history).
>
> Hope that helps.
>
> Arun
>
>
> On Aug 11, 2010, at 8:54 AM, Scott Whitecross wrote:
>
>  Hi -
>>
>> What's the best way to list and query information on Hadoop job histories?
>> For example, I'd like to see the job names from the past week against a
>> Hadoop cluster I'm using.   I don't see an API call or a way through the
>> command line to pull the information.  Is the best way writing a quick
>> script to process the job history files?
>>
>> Thanks.
>> Scott
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message