hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mike Smith" <mike.smith....@gmail.com>
Subject Re: Exception in MergeQueue.merge
Date Tue, 06 Feb 2007 20:23:28 GMT
I do have the same problem in the trunk version when the map input is large.


java.lang.NullPointerException
	at org.apache.hadoop.io.SequenceFile$Sorter$MergeQueue.merge(SequenceFile.java:2392)
	at org.apache.hadoop.io.SequenceFile$Sorter.merge(SequenceFile.java:2087)
	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:498)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:191)
	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1372)


Mike.


On 2/6/07, Ion Badita <ion.badita@mcr.ro> wrote:
>
> I keep receiving the same exception when running simple tasks like
> wordcount example but with big data as input.
> After some System.err's i founded that the code causing this is:
>
>             Path outputFile = conf.getLocalPath("mapred.local.dir",
>                                   (outFile.suffix("." +
> passNo)).toString());
>
> In this code outFile is null.
>
> The full stacktrace is below:
>
> java.lang.NullPointerException
>         at org.apache.hadoop.io.SequenceFile$Sorter$MergeQueue.merge(
> SequenceFile.java:2397)
>         at org.apache.hadoop.io.SequenceFile$Sorter.merge(
> SequenceFile.java:2089)
>         at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(
> MapTask.java:498)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:191)
>         at org.apache.hadoop.mapred.TaskTracker$Child.main(
> TaskTracker.java:1372)
>
> I have a 6 nodes + 1 master cluster, running java 6 and hadoop 0.11
>
>
>
> John
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message