hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Schubert Zhang <zson...@gmail.com>
Subject Re: MapReduce task progress (0.20.0)
Date Sat, 05 Sep 2009 18:58:18 GMT
After reading of code, I find it is because NewTrackingRecordReader class in
MapTask should be modified.

private final TaskReporter reporter;

this.reporter = reporter;

    @Override
    public boolean nextKeyValue() throws IOException, InterruptedException {
      boolean result = real.nextKeyValue();
      if (result) {
        inputRecordCounter.increment(1);
      }
      reporter.setProgress(getProgress()); //adding
      return result;
    }

and then, we I checkout the latest 0.20.0 branch, the above modification has
in the branch.

Schubert
On Sun, Sep 6, 2009 at 2:07 AM, Schubert Zhang <zsongbo@gmail.com> wrote:

> In 0.19.x, the WEB GUI can show the progress of each map or reduce task
> (before it complete, it is from the RecordReader).
>
> But in 0.20.0, we cannot show the progress (always 0%) before map or reduce
> task complete.
>
> Can anybody tell me how to let it show the progress?

Mime
View raw message