Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of qwertymaniac@gmail.com
 designates 209.85.161.48 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=RPxqePvqnauWE7Z8f50ZcDE/uGYibiYNgop9HCD9GATS3NT+oZWQOm0Hm86JnX/QQD
         oFWljIu9vD8vPoNk4A2uYt2HSu0viBAXlxhk6I4+ocuHUlj0oZRk+fOr2wElsVJMA+35
         MnNssEfsM1Sq5G15Ajs5UkRi8Er5LElvUiNnQ=
MIME-Version: 1.0
In-Reply-To: <AANLkTik1x+TA-cg2_j-o0gcfH-g05+B2=egqeA19os9z@mail.gmail.com>
References: <AANLkTik1x+TA-cg2_j-o0gcfH-g05+B2=egqeA19os9z@mail.gmail.com>
From: Harsh J <qwertymaniac@gmail.com>
Date: Wed, 23 Feb 2011 18:32:01 +0530
Message-ID: <AANLkTim=QEm6EZRNOGVPuDOzqz09iASu3k+fwrAS3=0P@mail.gmail.com>
Subject: Re: Message in mapreduce class not logged
To: mapreduce-user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

The logs should appear in each of the ReduceTask's log. Are they not
appearing in them?

On Wed, Feb 23, 2011 at 6:27 PM, Thomas Anderson
<t.dt.aanderson@gmail.com> wrote:
> I have setup a cluster with several machines up and running. But I
> encounter a problem that my mapper reducer class does not log.
>
> The hadoop version I use is 0.20.2. The rootLogger in log4j.properties
> adds DRFA (daily rolling file appender) and RFA (rolling file
> appender).
>
> My map reduce class looks as below
>
> public class MyTest{
> =A0 =A0 =A0 =A0public static final Log LOG =3D LogFactory.getLog(MyTest.c=
lass);
>
> =A0 =A0 =A0 =A0public static class MiniMap extends Mapper<LongWritable, T=
ext,
> Text, Text>{
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0private long count =3D 0;
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0public void map(LongWritable key, Text val=
ue,
> Mapper.Context ctx) throws IOException, InterruptedException{
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0String valueString =3D val=
ue.toString();
> LOG.info("XXXXXXXXXXX value string obtained: "+valueString);
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ctx.write(new LongWritable=
(counter++),
> doSomething(valueString));
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0}
>
> =A0 =A0 =A0 =A0}
> ...
> =A0 =A0 =A0 =A0static String fetch(String urlpath){
> =A0 =A0 =A0 =A0 =A0...
> LOG.info("xxxxxxxxxxxxxx");
> =A0 =A0 =A0 =A0 =A0 ...
> =A0 =A0 =A0 =A0}
> }
>
> The error shows 0java.lang.NullPointerException
>
> =A0 =A0[java] =A0 =A0 =A0at org.apache.hadoop.mapred.MapTask$MapOutputBuf=
fer.collect(MapTask.java:849)
> =A0 =A0 [java] =A0 =A0 at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:54=
1)
> =A0 =A0 [java] =A0 =A0 at
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputC=
ontext.java:80)
> =A0 =A0 [java] =A0 =A0 at myapp.MyTest.$MiniMap.map(MiniMap.java:51)
> =A0 =A0 [java] =A0 =A0 at myapp.MyTest.$MiniMap.map(MiniMap.java:44)
>
> The error message points out the line that goes wrong, but the problem
> is I do not see any message (in LOG.info(...)) get logged.
>
> What else I need to turn on or modify so that log message can be seen
> in e.g. hadoop/logs/hadoop-...log?
>
> I appreciate any suggestion.
>


--=20
Harsh J
www.harshj.com