Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-dev@hadoop.apache.org
Date: Fri, 5 Aug 2011 14:34:27 +0000 (UTC)
From: "Joseph Shraibman (JIRA)" <jira@apache.org>
To: mapreduce-dev@hadoop.apache.org
Message-ID: 
 <647081993.11715.1312554867359.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1167675006.65967.1307115470388.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Resolved] (MAPREDUCE-2564) NullPointerException in
 WritableComparator
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/MAPREDUCE-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joseph Shraibman resolved MAPREDUCE-2564.
-----------------------------------------

       Resolution: Invalid
    Fix Version/s: 0.20.203.0

> NullPointerException in WritableComparator
> ------------------------------------------
>
>                 Key: MAPREDUCE-2564
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2564
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 0.20.203.0
>         Environment: java version "1.6.0_23"
> Java(TM) SE Runtime Environment (build 1.6.0_23-b05)
> Java HotSpot(TM) 64-Bit Server VM (build 19.0-b09, mixed mode)
> hadoop-0.20.203.0rc1
>            Reporter: Joseph Shraibman
>            Priority: Blocker
>             Fix For: 0.20.203.0
>
>
> java.lang.NullPointerException
>         at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:96)
>         at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:1110)
>         at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:70)
>         at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59)
>         at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1398)
>         at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1297)
>         at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:698)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:765)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
>         at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
> It is easy to see why this is happening.  The WritableComparator is created in JobConf line 776:
> {code:title=JobConf.java}
>    return WritableComparator.get(getMapOutputKeyClass().asSubclass(WritableComparable.class));
> }
> {code}
> which calls
> {code:title=WritableComparator.java|borderStyle=solid}
> protected WritableComparator(Class<? extends WritableComparable> keyClass) {
>     this(keyClass, false);
>   }
>   protected WritableComparator(Class<? extends WritableComparable> keyClass,
>       boolean createInstances) {
>     this.keyClass = keyClass;
>     if (createInstances) {
>       key1 = newKey();
>       key2 = newKey();
>       buffer = new DataInputBuffer();
>     } else {
>       key1 = key2 = null;
>       buffer = null;
>     }
>   }
> {code}
> key1, key2, and buffer end up being null. When compare() is called the NPE is thrown because buffer is null
> {code:title=WritableComparator.java|borderStyle=solid}
>  public int compare(byte[] b1, int s1, int l1, byte[] b2, int s2, int l2) {
>     try {
>       buffer.reset(b1, s1, l1);                   // parse key1
>       key1.readFields(buffer);
>       
>       buffer.reset(b2, s2, l2);                   // parse key2
>       key2.readFields(buffer);
>       
>     } catch (IOException e) {
>       throw new RuntimeException(e);
>     }
>     
>     return compare(key1, key2);                   // compare them
>   }
> {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira