Return-Path: 
 <mapreduce-user-return-4407-apmail-hadoop-mapreduce-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id B02B39672
	for <apmail-hadoop-mapreduce-user-archive@minotaur.apache.org>;
 Fri, 17 Feb 2012 06:37:42 +0000 (UTC)
Received: (qmail 63420 invoked by uid 500); 17 Feb 2012 06:37:41 -0000
Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org
Received: (qmail 63346 invoked by uid 500); 17 Feb 2012 06:37:40 -0000
Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:mapreduce-user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:mapreduce-user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:mapreduce-user@hadoop.apache.org>
List-Id: <mapreduce-user.hadoop.apache.org>
Reply-To: mapreduce-user@hadoop.apache.org
Delivered-To: mailing list mapreduce-user@hadoop.apache.org
Received: (qmail 63286 invoked by uid 99); 17 Feb 2012 06:37:37 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Feb 2012 06:37:37 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of shijuwei@gmail.com designates
 209.85.213.176 as permitted sender)
Received: from [209.85.213.176] (HELO mail-yx0-f176.google.com)
 (209.85.213.176)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Feb 2012 06:37:31 +0000
Received: by yenl9 with SMTP id l9so2158452yen.35
        for <mapreduce-user@hadoop.apache.org>;
 Thu, 16 Feb 2012 22:37:11 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        bh=dQGHEs9VQL8VWNi/XYICB+UbHchHlfid11cMYA78c18=;
        b=g4hwgjEXFOiyIswEmTvbaB3AmHYJa4iMp11NFWWamisVeSQCLNgvkQwmYYuVOu0P+d
         v3rvivuC8EQ+wu+kpXTsKWgp6x1JuLp2vEnUu20/AeQZR4VVy4rjVCLPXkmioavHo3ux
         PhVGBSD+cE4lnGUrmmP0RQZ9ouCBnktxGFLXM=
MIME-Version: 1.0
Received: by 10.236.136.99 with SMTP id v63mr7648042yhi.46.1329460631049; Thu,
 16 Feb 2012 22:37:11 -0800 (PST)
Received: by 10.236.29.136 with HTTP; Thu, 16 Feb 2012 22:37:11 -0800 (PST)
Date: Fri, 17 Feb 2012 14:37:11 +0800
Message-ID: 
 <CAKjofVgDAEiS7UDoNkiS4VFzRWNn+vaGe9DTiWDm6HivV+6RGQ@mail.gmail.com>
Subject: reduce output compression of Terasort
From: Juwei Shi <shijuwei@gmail.com>
To: mapreduce-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=485b397dd125a816a904b9232c41

--485b397dd125a816a904b9232c41
Content-Type: text/plain; charset=ISO-8859-1

Hi,

I am benchmarking the cluster using the Terasort package of Hadoop 0.20.2.
I enabled compression for both map output (*mapred.compress.map.output*)
and reduce output (*mapred.output.compress*). I checked the parameter in
Job.xml, both are true. I can see that the compression for Map output
works, but it seems that the compression for reduce output does not work.
The output of the job on HDFS is also 1TB.

Thanks!

- Juwei

--485b397dd125a816a904b9232c41
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi,<br clear=3D"all"><br>I am benchmarking the cluster using the Terasort p=
ackage of Hadoop 0.20.2. I enabled compression for both map output (<b>mapr=
ed.compress.map.output</b>) and reduce output (<b>mapred.output.compress</b=
>). I checked the parameter in Job.xml, both are true. I can see that the c=
ompression for Map output works, but it seems that the compression for redu=
ce output does not work. The output of the job on HDFS is also 1TB. <br>
<br>Thanks!<br><br>- Juwei <br>

--485b397dd125a816a904b9232c41--