Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of harsh@cloudera.com designates
 209.85.223.180 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAFyZJnQ0AwtZpt_snc_UE5-GOyD16O3U4UUtE172OUKqZq2uRQ@mail.gmail.com>
References: 
 <CAFyZJnQJwCT6CG5WY9dqFHUxAGk8qKNhrx1feQs8D8ZBr-jKew@mail.gmail.com>
 <CAOcnVr2Rx6EB7edLh7ivB_oJsC1hACdVOqXK874L2A7yY=vyEQ@mail.gmail.com>
 <CAFyZJnQ0AwtZpt_snc_UE5-GOyD16O3U4UUtE172OUKqZq2uRQ@mail.gmail.com>
From: Harsh J <harsh@cloudera.com>
Date: Fri, 8 Feb 2013 00:16:55 +0530
Message-ID: 
 <CAOcnVr0gmYQ8MUOEXkvNj4Vk63uUSd4G9B7kGU=o=3CRMXNawg@mail.gmail.com>
Subject: Re: Secondary Sort example error
To: "<user@hadoop.apache.org>" <user@hadoop.apache.org>
Content-Type: text/plain; charset=ISO-8859-1

Thanks, I managed to correlate proper line numbers.

Are you using some form of custom serialization in your job code? That
is, are your keys non-Writable types and are of some other type? The
specific NPE is arising from the SerializationFactory not being able
to find a serializer for your Map-Output key class. You may want to
look into that direction, or share your code for the list to spot it
instead.

On Fri, Feb 8, 2013 at 12:11 AM, Ravi Chandran
<ravichandran777@gmail.com> wrote:
> hi,
>
> it is Hadoop 2.0.0-cdh4.1.1. the whole output is given below:
>
> Hadoop 2.0.0-cdh4.1.1
> Subversion
> file:///data/1/jenkins/workspace/generic-package-centos32-6/topdir/BUILD/hadoop-2.0.0-cdh4.1.1/src/hadoop-common-project/hadoop-common
> -r 581959ba23e4af85afd8db98b7687662fe9c5f20
>
>
>
> On Fri, Feb 8, 2013 at 12:04 AM, Harsh J <harsh@cloudera.com> wrote:
>>
>> Hey Ravi,
>>
>> What version of Hadoop is this exactly? (Type and send output of
>> "hadoop version" if unsure)
>>
>> On Thu, Feb 7, 2013 at 11:55 PM, Ravi Chandran
>> <ravichandran777@gmail.com> wrote:
>> > Hi,
>> >
>> > I am trying to do a name sorting using secondary sort. I have a working
>> > example, which I am taking as a reference. But I am getting a null
>> > pointer
>> > error in the MapTask class. I am not able to locate the reason. as the
>> > logic
>> > to create the Custom Object from a given file has been tested through a
>> > java
>> > class..
>> > I am getting this error:
>> >
>> > 13/02/07 12:23:42 WARN snappy.LoadSnappy: Snappy native library is
>> > available
>> > 13/02/07 12:23:42 INFO snappy.LoadSnappy: Snappy native library loaded
>> > 13/02/07 12:23:42 INFO mapred.FileInputFormat: Total input paths to
>> > process
>> > : 1
>> > 13/02/07 12:23:43 INFO mapred.JobClient: Running job:
>> > job_201301301056_0014
>> > 13/02/07 12:23:44 INFO mapred.JobClient:  map 0% reduce 0%
>> > 13/02/07 12:23:56 INFO mapred.JobClient: Task Id :
>> > attempt_201301301056_0014_m_000000_0, Status : FAILED
>> > java.lang.NullPointerException
>> >  at
>> >
>> > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:814)
>> >  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:385)
>> >  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
>> >  at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
>> >  at java.security.AccessController.doPrivileged(Native Method)
>> >  at javax.security.auth.Subject.doAs(Subject.java:396)
>> >  at
>> >
>> > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
>> >  at org.apache.hadoop.mapred.Child.main(Child.java:262)
>> > 13/02/07 12:23:57 INFO mapred.JobClient: Task Id :
>> > attempt_201301301056_0014_m_000001_0, Status : FAILED
>> > java.lang.NullPointerException
>> >  at
>> >
>> > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:814)
>> >  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:385)
>> >  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
>> >  at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
>> >  at java.security.AccessController.doPrivileged(Native Method)
>> >  at javax.security.auth.Subject.doAs(Subject.java:396)
>> >  at
>> >
>> > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
>> >  at org.apache.hadoop.mapred.Child.main(Child.java:262)
>> >
>> > I am giving the Mapper code below:
>> >
>> > import java.io.IOException;
>> > import org.apache.hadoop.io.LongWritable;
>> > import org.apache.hadoop.io.Text;
>> > import org.apache.hadoop.mapred.MapReduceBase;
>> > import org.apache.hadoop.mapred.Mapper;
>> > import org.apache.hadoop.mapred.OutputCollector;
>> > import org.apache.hadoop.mapred.Reporter;
>> > import org.apache.log4j.Logger;
>> > import com.pom.Name;
>> >
>> > public class StubMapper extends MapReduceBase implements
>> >     Mapper<LongWritable, Text, Name, Text> {
>> >
>> >   private static Logger logger =
>> >     Logger.getLogger(StubMapper.class.getName());
>> >
>> >  StringBuffer readLine = new StringBuffer();
>> >   private final Name name = new Name();
>> >   @Override
>> >   public void map(LongWritable key, Text value,
>> >       OutputCollector<Name, Text> output, Reporter reporter)
>> >       throws IOException {
>> >     String line = value.toString();
>> >     String[] packer = null;
>> >
>> >     packer = line.split(" ");
>> >
>> >     // create the object
>> >  if(packer.length>2)
>> >  {
>> >   // take everything except last name
>> >   for (int i = 0; i < packer.length-1; i++) {
>> >    readLine.append(packer[i]+" ");
>> >   }
>> >
>> >   name.setfName(readLine.toString());
>> >   name.setlName(packer[packer.length-1]);
>> >
>> >   //clear the variable
>> >   readLine.delete(0, readLine.length());
>> >  }
>> >  else if(packer.length>0)
>> >  {
>> >   name.setfName(packer[0]);
>> >      name.setlName(packer[1]);
>> >    }
>> >
>> >  output.collect(name, new Text(name.getlName()));
>> >
>> >   }
>> > }
>> >
>> > I am not able to figure out the possible cause..
>> >
>> > --
>> > Thanks & Regards
>> > Ravi
>>
>>
>>
>> --
>> Harsh J
>
>
>
>
> --
> Thanks & Regards
> Ravi


--
Harsh J