hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-5499) JobTracker metrics do not match job counters
Date Mon, 16 Mar 2009 11:40:51 GMT
JobTracker metrics do not match job counters

                 Key: HADOOP-5499
                 URL: https://issues.apache.org/jira/browse/HADOOP-5499
             Project: Hadoop Core
          Issue Type: Bug
          Components: mapred, metrics
    Affects Versions: 0.20.0
            Reporter: Chris Douglas

After one run of a 10 map, 3 reduce sort job, the mapred metrics report:
mapred.tasktracker: hostName=snip, sessionId=, mapTaskSlots=3, maps_running=0, \
  reduceTaskSlots=3, reduces_running=0, tasks_completed=15, tasks_failed_ping=0, tasks_failed_timeout=0
mapred.jobtracker: hostName=snip, sessionId=, jobs_completed=1, jobs_submitted=1, \
  maps_completed=10, maps_launched=12, reduces_completed=3, reduces_launched=3

After the second (w/ one manually killed reduce)
mapred.tasktracker: hostName=pineapple-lm, sessionId=, mapTaskSlots=3, maps_running=0, \
   reduceTaskSlots=3, reduces_running=0, tasks_completed=32, tasks_failed_ping=0, tasks_failed_timeout=0
mapred.jobtracker: hostName=pineapple-lm, sessionId=, jobs_completed=2, jobs_submitted=2,
  maps_completed=20, maps_launched=24, reduces_completed=6, reduces_launched=8

The counters report the expected 10/3 map/reduce completions, the second job reporting a single

# The {{maps_launched}} and {{reduces_launched}} counts are likely recording setup and cleanup
tasks as well
# After being recorded among launched tasks, it looks like setup and cleanup are _not_ included
among completed tasks
# {{tasks_completed}} on the TaskTracker should include only user tasks
# The {{reduces_launched}} count makes little sense to me. With three reduces launched per
job and one failed, what's the other launched reduce counting?

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message