hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Anderson <t.dt.aander...@gmail.com>
Subject Re: Message in mapreduce class not logged
Date Fri, 25 Feb 2011 05:45:13 GMT
I notice that the job message indeed gets logged in the userlogs dir,
where contains sys* files.

Thanks for help.

On Wed, Feb 23, 2011 at 9:02 PM, Harsh J <qwertymaniac@gmail.com> wrote:
> The logs should appear in each of the ReduceTask's log. Are they not
> appearing in them?
>
> On Wed, Feb 23, 2011 at 6:27 PM, Thomas Anderson
> <t.dt.aanderson@gmail.com> wrote:
>> I have setup a cluster with several machines up and running. But I
>> encounter a problem that my mapper reducer class does not log.
>>
>> The hadoop version I use is 0.20.2. The rootLogger in log4j.properties
>> adds DRFA (daily rolling file appender) and RFA (rolling file
>> appender).
>>
>> My map reduce class looks as below
>>
>> public class MyTest{
>>        public static final Log LOG = LogFactory.getLog(MyTest.class);
>>
>>        public static class MiniMap extends Mapper<LongWritable, Text,
>> Text, Text>{
>>                private long count = 0;
>>                public void map(LongWritable key, Text value,
>> Mapper.Context ctx) throws IOException, InterruptedException{
>>                        String valueString = value.toString();
>> LOG.info("XXXXXXXXXXX value string obtained: "+valueString);
>>                        ctx.write(new LongWritable(counter++),
>> doSomething(valueString));
>>                }
>>
>>        }
>> ...
>>        static String fetch(String urlpath){
>>          ...
>> LOG.info("xxxxxxxxxxxxxx");
>>           ...
>>        }
>> }
>>
>> The error shows 0java.lang.NullPointerException
>>
>>    [java]      at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:849)
>>     [java]     at
>> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:541)
>>     [java]     at
>> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
>>     [java]     at myapp.MyTest.$MiniMap.map(MiniMap.java:51)
>>     [java]     at myapp.MyTest.$MiniMap.map(MiniMap.java:44)
>>
>> The error message points out the line that goes wrong, but the problem
>> is I do not see any message (in LOG.info(...)) get logged.
>>
>> What else I need to turn on or modify so that log message can be seen
>> in e.g. hadoop/logs/hadoop-...log?
>>
>> I appreciate any suggestion.
>>
>
>
>
> --
> Harsh J
> www.harshj.com
>

Mime
View raw message