hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karam Singh (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-857) task fails with NPE when GzipCodec is used for mapred.map.output.compression.codec and native libary is not present
Date Thu, 13 Aug 2009 07:32:14 GMT
task fails with NPE  when GzipCodec is used for mapred.map.output.compression.codec and native
libary is not present

                 Key: MAPREDUCE-857
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-857
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: task
            Reporter: Karam Singh
             Fix For: 0.21.0

Ran a job with mapred.map.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec.
Whenmaps of job completes they with following NPE  -:
tasklog -:
2009-08-12 13:48:13,423 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 256
2009-08-12 13:48:13,611 INFO org.apache.hadoop.mapred.MapTask: data buffer = 204010944/214748368
2009-08-12 13:48:13,611 INFO org.apache.hadoop.mapred.MapTask: record buffer = 3187670/3355443
2009-08-12 13:49:45,473 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map output
2009-08-12 13:49:45,544 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
2009-08-12 13:49:45,545 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor
2009-08-12 13:49:45,546 WARN org.apache.hadoop.mapred.Child: Error running child : java.lang.NullPointerException
        at org.apache.hadoop.mapred.IFile$Writer.<init>(IFile.java:105)
        at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1248)
        at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1146)
        at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:528)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:604)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:318)
        at org.apache.hadoop.mapred.Child.main(Child.java:162)

Line 105 of IFile.java contains followings line in trunk code on which error was seen -:
Line 104: this.compressor = CodecPool.getCompressor(codec);
Line: this.compressor.reset();

If native is available job runs successfully without any failures 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message