hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: How to interpret the progress meter?
Date Fri, 11 Jan 2013 06:01:05 GMT
The map side percentage is as the map's record reader reports its
progress. The reduce side is divided into 3 phases of 33~% each ->
shuffle (fetch data), sort and finally user-code (reduce). It is
normal to see jumps between these values, depending on the work to be
done, etc.

On Fri, Jan 11, 2013 at 9:32 AM, Roy Smith <roy@panix.com> wrote:
> I'm running a job that looks like it's going to take about 12 hours on 4 EC2
> instances.  I don't really understand the "complete" percentages reported by
> http://localhost:9100/jobtasks.jsp.  They are extremely non-linear.  For my
> reduce steps, they ramp up to 40-60% in just a few minutes, then take hours
> to slowly inch their way up the rest of the way to 100%.
>
> What does the "complete" percentage really mean?
>
> --
> Roy Smith
> roy@panix.com
>



-- 
Harsh J

Mime
View raw message