hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niels Basjes <Ni...@basjes.nl>
Subject Gzip progress during map phase.
Date Sat, 24 Dec 2011 14:23:33 GMT
Hi,

I noticed that the mapper progress indication in the hadoop cdh3
distribution jumps from 0% to 100% for each gzipped input file. So when
running with big gzipped input files the job appears to be stuck.

I was unable to find a jira issue that describes this effect.
Before I dive into this I have a few questions to you guys:
1) is this a known effect for the 0.20 version? If so what is the jira
issue?
2) is this specific to gzip?
3) is this effect still present in the MRv2/yarn version of Hadoop?

Thanks.
-- 
Met vriendelijke groet,
Niels Basjes
(Verstuurd vanaf mobiel )

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message