hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niels Basjes <Ni...@basjes.nl>
Subject Gzip progress during map phase.
Date Sat, 24 Dec 2011 14:23:33 GMT

I noticed that the mapper progress indication in the hadoop cdh3
distribution jumps from 0% to 100% for each gzipped input file. So when
running with big gzipped input files the job appears to be stuck.

I was unable to find a jira issue that describes this effect.
Before I dive into this I have a few questions to you guys:
1) is this a known effect for the 0.20 version? If so what is the jira
2) is this specific to gzip?
3) is this effect still present in the MRv2/yarn version of Hadoop?

Met vriendelijke groet,
Niels Basjes
(Verstuurd vanaf mobiel )

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message