Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of qwertymaniac@gmail.com
 designates 209.85.161.48 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type;
        b=THU0Mpp5jzAvVbuFmR5I/pkBunF/kSXaX9OHH7uo6+cNRAkJDs3ZosdBmx/wYIHJVJ
         MavGFw43RiIl8ULiM59DlUyBxymXxCfbAjnOt9glrQvFshEZf3TpTyigORivIBUjv2a1
         MPQ7nPxlI/eqd3pWVCFVboxmv8EWPr5ipX5U0=
MIME-Version: 1.0
In-Reply-To: <AANLkTim-=ORFOv-RU14uNGomHF7CJKWwRUD34EA9D+Xo@mail.gmail.com>
References: <AANLkTinHHierhff4DBPU=4VQFhJgp9P_-A5sH4ea_kfN@mail.gmail.com>
 <AANLkTinahKqNwN_eRN8gehoc2NB5Y8L5bO9z=7QFHyGS@mail.gmail.com>
 <AANLkTim-=ORFOv-RU14uNGomHF7CJKWwRUD34EA9D+Xo@mail.gmail.com>
From: Harsh J <qwertymaniac@gmail.com>
Date: Mon, 14 Feb 2011 23:46:52 +0530
Message-ID: <AANLkTi=7ps0ZbMB+tm2WUrf1xb_-17a=hJ26fx8kNsEH@mail.gmail.com>
Subject: Re: Map output files are SequenceFileFormat
To: mapreduce-user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1

Hello,

On Mon, Feb 14, 2011 at 11:37 PM, Pedro Costa <psdc1978@gmail.com> wrote:
> And when the data of the map-intermediate files is compressed, it's
> still an IFile?

Yes. From my understanding, if compression is turned ON for IFile, the
output stream for writing the IFile is itself set as a compressing one
and all data written to the stream is compressed.

In contrast, in SequenceFiles, compression is done in blocks (of a
sizes set upon the Writer creation), and keys are left uncompressed.

-- 
Harsh J
www.harshj.com