hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harish Mallipeddi <harish.mallipe...@gmail.com>
Subject Re: userlogs
Date Wed, 29 Jul 2009 17:20:02 GMT
On Wed, Jul 29, 2009 at 10:06 PM, Marc Limotte <mlimotte@feeva.com> wrote:

> A quick question about the user logs (hadoop 0.19.1).
>
> I've looked all over HDFS.  I can't locate the actual userlogs.  Where are
> they stored?  I can get to them through the job tracker web interface, but
> I'd like to be able to script some jobs to check the logs for errors.  I
> suppose I could use curl against the web interface, but that seems like a
> hack.  Alternatively, is there command line access to the userlogs through
> the hadoop command?  I couldn't find anything like that.
> Thanks for any tips...
>
>
All hadoop logs are stored on the local filesystem on each node (the
location is configurable). What you could do though is run a cron job which
periodically pushes log files to HDFS at the end of the day.

Here's another idea:
http://www.cloudera.com/blog/2008/11/02/configuring-and-using-scribe-for-hadoop-log-collection/

-- 
Harish Mallipeddi
http://blog.poundbang.in

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message