hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: How do I log from my map/reduce application?
Date Wed, 15 Dec 2010 12:20:57 GMT
On 13/12/10 17:29, W.P. McNeill wrote:
> I would like to use Hadoop's Log4j infrastructure to do logging from my
> map/reduce application.  I think I've got everything set up correctly, but I
> am still unable to specify the logging level I want.
> By default Hadoop is set up to log at level INFO.  The first line of its
> log4j.properties file looks like this:
> hadoop.root.logger=INFO,console
> I have an application whose reducer looks like this:
> package com.me;
> public class MyReducer<...>  extends Reducer<...>  {
>     private static Logger logger =
> Logger.getLogger(MyReducer.class.getName());
>     ...
>     protected void reduce(...) {
>         logger.debug("My message");
>         ...
>     }
> }
> I've added the following line to the Hadoop log4j.properties file:
> log4j.logger.com.me.MyReducer=DEBUG
> I expect the Hadoop system to log at level INFO, but my application to log
> at level DEBUG, so that I see "My message" in the logs for the reducer task.
>   However, my application does not produce any log4j output.  If I change the
> line in my reducer to read logger.info("My message") the message does get
> logged, so somehow I'm failing to specify that log level for this class.
> I've also tried changing the log4j line for my app to
> read log4j.logger.com.me.MyReducer=DEBUG,console and get the same result.
> I've been through the Hadoop and log4j documentation and I can't figure out
> what I'm doing wrong.  Any suggestions?

I'd start with your logger logging @info level which log4j.properties 
file it's finding on the classpath. The code I use to find a resource 
looks like this:

    public URL findResource(String resource) {
         return getClass().getClassLoader().getResource(resource);

Once you know which log4j file is being used to drive your code, then 
you can start tweaking it...

View raw message