hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lei liu <liulei...@gmail.com>
Subject how does hadoop handle the counter of the failed task and speculative task
Date Sun, 26 Dec 2010 06:03:05 GMT
I define the counter to count the bad records, there is below code in map
task;
 reporter.incrCounter("bad',"records', 1),

When the job is completed, the pritnt the result to use below code:
long total = counters.findCounter("bad","records").getCounter();


But I have two questions about the counter:
1. If the map task is retried 4 times, and the last map task is successful,
I think the counter of the other three map tasks should be not included in
ultima result, is that right?
2. If there are speculative tasks,  I think the counter of speculative tasks
should be not included in ultima result, is that right?


Thanks,

LiuLei

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message