hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Greg Roelofs (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-857) task fails with NPE when GzipCodec is used for mapred.map.output.compression.codec and native libary is not present
Date Sat, 18 Sep 2010 04:39:33 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12910893#action_12910893
] 

Greg Roelofs commented on MAPREDUCE-857:
----------------------------------------

Do you know if this still occurs?  I vaguely recall seeing some e-mail about something like
this a couple of months back; I wonder if it got fixed in a duplicate JIRA.

> task fails with NPE  when GzipCodec is used for mapred.map.output.compression.codec and
native libary is not present
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-857
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-857
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task
>    Affects Versions: 0.21.0
>            Reporter: Karam Singh
>
> Ran a job with mapred.map.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec.
> Whenmaps of job completes they with following NPE  -:
> tasklog -:
> 2009-08-12 13:48:13,423 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 256
> 2009-08-12 13:48:13,611 INFO org.apache.hadoop.mapred.MapTask: data buffer = 204010944/214748368
> 2009-08-12 13:48:13,611 INFO org.apache.hadoop.mapred.MapTask: record buffer = 3187670/3355443
> 2009-08-12 13:49:45,473 INFO org.apache.hadoop.mapred.MapTask: Starting flush of map
output
> 2009-08-12 13:49:45,544 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where applicable
> 2009-08-12 13:49:45,545 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor
> 2009-08-12 13:49:45,546 WARN org.apache.hadoop.mapred.Child: Error running child : java.lang.NullPointerException
>         at org.apache.hadoop.mapred.IFile$Writer.<init>(IFile.java:105)
>         at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1248)
>         at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1146)
>         at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:528)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:604)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:318)
>         at org.apache.hadoop.mapred.Child.main(Child.java:162)
> Line 105 of IFile.java contains followings line in trunk code on which error was seen
-:
> Line 104: this.compressor = CodecPool.getCompressor(codec);
> Line: this.compressor.reset();
> If native is available job runs successfully without any failures 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message