hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jinghui Wang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-11334) Mapreduce Job Failed due to failure fetching mapper output on the reduce side
Date Tue, 25 Nov 2014 20:38:12 GMT
Jinghui Wang created HADOOP-11334:
-------------------------------------

             Summary: Mapreduce Job Failed due to failure fetching mapper output on the reduce
side
                 Key: HADOOP-11334
                 URL: https://issues.apache.org/jira/browse/HADOOP-11334
             Project: Hadoop Common
          Issue Type: Bug
          Components: io
    Affects Versions: 2.4.1
            Reporter: Jinghui Wang


Running terasort with the following options hadoop jar hadoop-mapreduce-examples.jar terasort
*-Dio.native.lib.available=false -Dmapreduce.map.output.compress=true -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.GzipCodec*
 /tmp/tera-in /tmp/tera-out

The job failed with the reducer failed to fetching the output from mappers (see the following
stacktrace). The problem is that in JIRA MAPREDUCE-1784, it added support to handle null compressors
to default to non-compressed output. In this case, when the *io.native.lib.available* is set
to true, the compressor will be null. However, the decompressor has a Java implementation,
so when the reducer tries to read the mapper output, it uses the decompressor, but the output
does not have the Gzip header.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message