Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
Sender: niels@basj.es
In-Reply-To: 
 <CAHxBGfJ2APo4rT-mSDHdY_uLVupVGWmgUWVY6W6_smSyk3TTBg@mail.gmail.com>
References: 
 <CAHxBGfJ2APo4rT-mSDHdY_uLVupVGWmgUWVY6W6_smSyk3TTBg@mail.gmail.com>
Date: Wed, 7 Nov 2012 13:47:22 +0100
Message-ID: 
 <CADoiZqqQThCdJrQGxM-FH=LruFGKS=iBJxdjgBMSPWBFMgYVYw@mail.gmail.com>
Subject: Re: Doubts on compressed file
From: Niels Basjes <Niels@basjes.nl>
To: user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1

Hi,

> If a zip file(Gzip) is loaded into HDFS will it get splitted into Blocks and
> store in HDFS?

Yes.

> I understand that a single mapper can work with GZip as it reads the entire
> file from beginning to end... In that case if the GZip file size is larget
> than 128 MB will it get splitted into blocks and stored in HDFS?

Yes, and then the mapper will read the other parts of the file over the network.
So what I do is I upload such files with a bigger HDFS blocksize so
the mapper has "the entire file" locally.

-- 
Best regards / Met vriendelijke groeten,

Niels Basjes