hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <qwertyman...@gmail.com>
Subject Re: Message in mapreduce class not logged
Date Wed, 23 Feb 2011 13:02:01 GMT
The logs should appear in each of the ReduceTask's log. Are they not
appearing in them?

On Wed, Feb 23, 2011 at 6:27 PM, Thomas Anderson
<t.dt.aanderson@gmail.com> wrote:
> I have setup a cluster with several machines up and running. But I
> encounter a problem that my mapper reducer class does not log.
>
> The hadoop version I use is 0.20.2. The rootLogger in log4j.properties
> adds DRFA (daily rolling file appender) and RFA (rolling file
> appender).
>
> My map reduce class looks as below
>
> public class MyTest{
>        public static final Log LOG = LogFactory.getLog(MyTest.class);
>
>        public static class MiniMap extends Mapper<LongWritable, Text,
> Text, Text>{
>                private long count = 0;
>                public void map(LongWritable key, Text value,
> Mapper.Context ctx) throws IOException, InterruptedException{
>                        String valueString = value.toString();
> LOG.info("XXXXXXXXXXX value string obtained: "+valueString);
>                        ctx.write(new LongWritable(counter++),
> doSomething(valueString));
>                }
>
>        }
> ...
>        static String fetch(String urlpath){
>          ...
> LOG.info("xxxxxxxxxxxxxx");
>           ...
>        }
> }
>
> The error shows 0java.lang.NullPointerException
>
>    [java]      at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:849)
>     [java]     at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:541)
>     [java]     at
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
>     [java]     at myapp.MyTest.$MiniMap.map(MiniMap.java:51)
>     [java]     at myapp.MyTest.$MiniMap.map(MiniMap.java:44)
>
> The error message points out the line that goes wrong, but the problem
> is I do not see any message (in LOG.info(...)) get logged.
>
> What else I need to turn on or modify so that log message can be seen
> in e.g. hadoop/logs/hadoop-...log?
>
> I appreciate any suggestion.
>



-- 
Harsh J
www.harshj.com

Mime
View raw message